A Total Variation Regularization Based Super-Resolution Reconstruction Algorithm for Digital Video
© Michael K. Ng et al. 2007
Received: 13 September 2006
Accepted: 21 April 2007
Published: 27 June 2007
Super-resolution (SR) reconstruction technique is capable of producing a high-resolution image from a sequence of low-resolution images. In this paper, we study an efficient SR algorithm for digital video. To effectively deal with the intractable problems in SR video reconstruction, such as inevitable motion estimation errors, noise, blurring, missing regions, and compression artifacts, the total variation (TV) regularization is employed in the reconstruction model. We use the fixed-point iteration method and preconditioning techniques to efficiently solve the associated nonlinear Euler-Lagrange equations of the corresponding variational problem in SR. The proposed algorithm has been tested in several cases of motion and degradation. It is also compared with the Laplacian regularization-based SR algorithm and other TV-based SR algorithms. Experimental results are presented to illustrate the effectiveness of the proposed algorithm.
- Tsai RY, Huang TS: Multi-frame image restoration and registration. Advances in Computer Vision and Image Processing 1984,1(2):317-339.
- Kim SP, Bose NK, Valenzuela HM: Recursive reconstruction of high resolution image from noisy undersampled multiframes. IEEE Transactions on Acoustics, Speech, and Signal Processing 1990,38(6):1013-1027. 10.1109/29.56062View Article
- Kim SP, Su W-Y: Recursive high-resolution reconstruction of blurred multiframe images. IEEE Transactions on Image Processing 1993,2(4):534-539. 10.1109/83.242363View Article
- Rhee S, Kang MG: Discrete cosine transform based regularized high-resolution image reconstruction algorithm. Optical Engineering 1999,38(8):1348-1356. 10.1117/1.602177View Article
- Chan RH, Chan TF, Shen L, Shen Z: Wavelet algorithms for high-resolution image reconstruction. SIAM Journal of Scientific Computing 2003,24(4):1408-1432. 10.1137/S1064827500383123MathSciNetView ArticleMATH
- Ng MK, Sze CK, Yung SP: Wavelet algorithms for deblurring models. International Journal of Imaging Systems and Technology 2004,14(3):113-121. 10.1002/ima.20014View Article
- Nguyen N, Milanfar P: A wavelet-based interpolation-restoration method for superresolution (wavelet superresolution). Circuits, Systems, and Signal Processing 2000,19(4):321-338. 10.1007/BF01200891View ArticleMATH
- Ur H, Gross D: Improved resolution from subpixel shifted pictures. CVGIP: Graphical Models and Image Processing 1992,54(2):181-186. 10.1016/1049-9652(92)90065-6
- Irani M, Peleg S: Improving resolution by image registration. CVGIP: Graphical Models and Image Processing 1991,53(3):231-239. 10.1016/1049-9652(91)90045-L
- Stark H, Oskoui P: High-resolution image recovery from image-plane arrays, using convex projections. Journal of the Optical Society of America A: Optics and Image Science, and Vision 1989,6(11):1715-1726. 10.1364/JOSAA.6.001715View Article
- Tekalp AM, Ozkan MK, Sezan MI: High-resolution image reconstruction from lower-resolution image sequences and space-varying image restoration. Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '92), March 1992, San Francisco, Calif, USA 3: 169-172.
- Patti AJ, Sezan MI, Tekalp AM: High-resolution image reconstruction from a low-resolution image sequence in the presence of time-varying motion blur. Proceedings of IEEE International Conference Image Processing (ICIP '94), November 1994, Austin, Tex, USA 1: 343-347.
- Patti AJ, Sezan MI, Tekalp AM: Superresolution video reconstruction with arbitrary sampling lattices and nonzero aperture time. IEEE Transactions on Image Processing 1997,6(8):1064-1076. 10.1109/83.605404View Article
- Tom BC, Katsaggelos AK: Reconstruction of a high-resolution image from multiple-degraded misregistered low-resolution images. Visual Communications and Image Processing, September 1994, Chicago, Ill, USA, Proceedings of SPIE 2308: 971-981.View Article
- Schultz RR, Stevenson RL: Extraction of high-resolution frames from video sequences. IEEE Transactions on Image Processing 1996,5(6):996-1011. 10.1109/83.503915View Article
- Hardie RC, Tuinstra TR, Bognar J, Barnard KJ, Armstrong EE: High resolution image reconstruction from digital video with global and non-global scene motion. Proceedings of IEEE International Conference on Image Processing (ICIP '97), October 1997, Santa Barbara, Calif, USA 1: 153-156.View Article
- Elad M, Feuer A: Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images. IEEE Transactions on Image Processing 1997,6(12):1646-1658. 10.1109/83.650118View Article
- Elad M, Feuer A: Superresolution restoration of an image sequence: adaptive filtering approach. IEEE Transactions on Image Processing 1999,8(3):387-395. 10.1109/83.748893View Article
- Chung J, Haber E, Nagy J: Numerical methods for coupled super-resolution. Inverse Problems 2006,22(4):1261-1272. 10.1088/0266-5611/22/4/009MathSciNetView ArticleMATH
- Hardie RC, Barnard KJ, Armstrong EE: Joint MAP registration and high-resolution image estimation using a sequence of undersampled images. IEEE Transactions on Image Processing 1997,6(12):1621-1633. 10.1109/83.650116View Article
- Woods NA, Galatsanos NP, Katsaggelos AK: Stochastic methods for joint registration, restoration, and interpolation of multiple undersampled images. IEEE Transactions on Image Processing 2006,15(1):201-213.MathSciNetView Article
- Shen H, Zhang L, Huang B, Li P: A MAP approach for joint motion estimation, segmentation, and super resolution. IEEE Transactions on Image Processing 2007,16(2):479-490.MathSciNetView Article
- Sasahara R, Hasegawa H, Yamada I, Sakaniwa K: A color super-resolution with multiple nonsmooth constraints by hybrid steepest descent method. Proceedings of IEEE International Conference on Image Processing (ICIP '05), September 2005, Genova, Italy 1: 857-860.
- Farsiu S, Elad M, Milanfar P: Multiframe demosaicing and super-resolution of color images. IEEE Transactions on Image Processing 2006,15(1):141-159.View Article
- Akgun T, Altunbasak Y, Mersereau RM: Super-resolution reconstruction of hyperspectral images. IEEE Transactions on Image Processing 2005,14(11):1860-1875.View Article
- Segall CA, Katsaggelos AK, Molina R, Mateos J: Bayesian resolution enhancement of compressed video. IEEE Transactions on Image Processing 2004,13(7):898-910. 10.1109/TIP.2004.827230View Article
- Segall CA, Molina R, Katsaggelos AK: High-resolution images from low-resolution compressed video. IEEE Signal Processing Magazine 2003,20(3):37-48. 10.1109/MSP.2003.1203208View Article
- Capel D, Zisserman A: Super-resolution enhancement of text image sequences. Proceedings of the 15th International Conference on Pattern Recognition (ICPR '00), September 2000, Barcelona, Spain 1: 600-605.View Article
- Han YB, Wu LN: Super resolution reconstruction of video sequence based on total variation. Proceedings of International Symposium on Intelligent Multimedia, Video and Speech Processing (ISIMP '04), October 2004, Hong Kong 575-578.
- Vazquez C, Aly H, Dubois E, Mitiche A: Motion compensated super-resolution of video by level sets evolution. Proceedings of IEEE International Conference on Image Processing (ICIP '04), October 2004, Singapore 3: 1767-1770.
- Farsiu S, Robinson MD, Elad M, Milanfar P: Fast and robust multiframe super resolution. IEEE Transactions on Image Processing 2004,13(10):1327-1344. 10.1109/TIP.2004.834669View Article
- Chan TF, Shen J: Mathematical models for local nontexture inpaintings. SIAM Journal on Applied Mathematics 2002,62(3):1019-1043. 10.1137/S0036139900368844MathSciNetView ArticleMATH
- Borman S, Stevenson RL: Spatial resolution enhancement of low-resolution image sequences: a comprehensive review with directions for future research. Laboratory for Image and Signal Analysis (LISA), University of Notre Dame, Notre Dame, Ind, USA; 1998.
- Park SC, Park MK, Kang MG: Super-resolution image reconstruction: a technical overview. IEEE Signal Processing Magazine 2003,20(3):21-36. 10.1109/MSP.2003.1203207View Article
- Nguyen N, Milanfar P, Golub G: Efficient generalized cross-validation with applications to parametric image restoration and resolution enhancement. IEEE Transactions on Image Processing 2001,10(9):1299-1308. 10.1109/83.941854MathSciNetView ArticleMATH
- Capel D, Zisserman A: Computer vision applied to super resolution. IEEE Signal Processing Magazine 2003,20(3):75-86. 10.1109/MSP.2003.1203211View Article
- Schultz RR, Meng L, Stevenson RL: Subpixel motion estimation for super-resolution image sequence enhancement. Journal of Visual Communication and Image Representation 1998,9(1):38-50. 10.1006/jvci.1997.0370View Article
- Tekalp AM: Digital Video Processing. Prentice-Hall, Englewood Clliffs, NJ, USA; 1995.
- Farsiu S, Robinson D, Elad M, Milanfar P: Advances and challenges in super-resolution. International Journal of Imaging Systems and Technology 2004,14(2):47-57. 10.1002/ima.20007View Article
- Rudin L, Osher S, Fatemi E: Nonlinear total variation based noise removal algorithms. Physica D 1992,60(1–4):259-268.View ArticleMathSciNetMATH
- Vogel CR, Oman ME: Fast, robust total variation-based reconstruction of noisy, blurred images. IEEE Transactions on Image Processing 1998,7(6):813-824. 10.1109/83.679423MathSciNetView ArticleMATH
- Chambolle A: An algorithm for total variation minimization and applications. Journal of Mathematical Imaging and Vision 2004,20(1-2):89-97.MathSciNet
- Li Y, Santosa F: A computational algorithm for minimizing total variation in image restoration. IEEE Transactions on Image Processing 1996,5(6):987-995. 10.1109/83.503914View Article
- Bioucas-Dias JM, Figueiredo MAT, Oliveira JP: Total variation-based image deconvolution: a majorization-minimization approach. Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '06), May 2006, Toulouse, France 2: 861-864.
- Vogel CR, Oman ME: Iterative methods for total variation denoising. SIAM Journal of Scientific Computing 1996,17(1):227-238. 10.1137/0917016MathSciNetView ArticleMATH
- Vogel CR: Computational Methods for Inverse Problems, Frontiers in Applied Mathematics. SIAM, Philadelphia, Pa, USA; 2002.View Article
- Lin F-R, Ng MK, Ching W-K: Factorized banded inverse preconditioners for matrices with Toeplitz structure. SIAM Journal of Scientific Computing 2005,26(6):1852-1870. 10.1137/030601272MathSciNetView ArticleMATH
- Chan RH, Chan TF, Wong C-K: Cosine transform based preconditioners for total variation deblurring. IEEE Transactions on Image Processing 1999,8(10):1472-1478. 10.1109/83.791976View Article
- Ng MK, Chan RH, Chan TF, Yip AM: Cosine transform preconditioners for high resolution image reconstruction. Linear Algebra and Its Applications 2000,316(1–3):89-104.MathSciNetView ArticleMATH
- Kolotilina LY, Yeremin AY: Factorized sparse approximate inverse preconditionings I: theory. SIAM Journal on Matrix Analysis and Applications 1993,14(1):45-58. 10.1137/0614004MathSciNetView ArticleMATH
- Nguyen N, Milanfar P, Golub G: A computationally efficient superresolution image reconstruction algorithm. IEEE Transactions on Image Processing 2001,10(4):573-583. 10.1109/83.913592MathSciNetView ArticleMATH
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.