Skip to main content
EURASIP Journal on Advances in Signal Processing
Download PDF
  • Research Article
  • Open access
  • Published:

Low-Cost Super-Resolution Algorithms Implementation Over a HW/SW Video Compression Platform

EURASIP Journal on Advances in Signal Processing volume 2006, Article number: 084614 (2006) Cite this article

Abstract

Two approaches are presented in this paper to improve the quality of digital images over the sensor resolution using super-resolution techniques: iterative super-resolution (ISR) and noniterative super-resolution (NISR) algorithms. The results show important improvements in the image quality, assuming that sufficient sample data and a reasonable amount of aliasing are available at the input images. These super-resolution algorithms have been implemented over a codesign video compression platform developed by Philips Research, performing minimal changes on the overall hardware architecture. In this way, a novel and feasible low-cost implementation has been obtained by using the resources encountered in a generic hybrid video encoder. Although a specific video codec platform has been used, the methodology presented in this paper is easily extendable to any other video encoder architectures. Finally a comparison in terms of memory, computational load, and image quality for both algorithms, as well as some general statements about the final impact of the sampling process on the quality of the super-resolved (SR) image, are also presented.

References

  1. Komatsu T, Igarashi T, Aizawa K, Saito T: Very high resolution imaging scheme with multiple different-aperture cameras. Signal Processing: Image Communication 1993, 5(5–6):511–526. 10.1016/0923-5965(93)90014-K

    Google Scholar 

  2. Ghiglia DC: Space-invariant deblurring given N independently blurred images of a common object. Journal of Optical Society of America A 1984, 1(4):398–402. 10.1364/JOSAA.1.000398

    Article  Google Scholar 

  3. Kim SP, Su W-Y: Recursive high-resolution reconstruction of blurred multiframe images. IEEE Transactions on Image Processing 1993, 2(4):534–539. 10.1109/83.242363

    Article  Google Scholar 

  4. Cheeseman P, Kanefsky B, Kraft R, Stutz J, Hanson R: Super-resolved surface reconstruction from multiple images. In Maximum Entropy and Bayesian Methods. Edited by: Heidbreder GR. Kluwer Academic, Dordrecht, The Netherlands; 1996:293–308.

    Chapter  Google Scholar 

  5. Avrin V, Dinstein I: Restoration and resolution enhancement of video sequences. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '97), April 1997, Munich, Germany 4: 2549–2552.

    Google Scholar 

  6. Eggleston P: Quality photos from digital video: salient Stills' algorithms provide a missing link. Advanced Imaging Magazine 2000, 15(5):39–41.

    Google Scholar 

  7. Schultz RR: Super-resolution enhancement of native digital video versus digitized NTSC sequences. Proceedings of the 5th IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI '02), April 2002, Sante Fe, NM, USA 193–197.

    Chapter  Google Scholar 

  8. Sale D, Schultz RR, Szczerba RJ: Super-resolution enhancement of night vision image sequences. Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, October 2000, Nashville, Tenn, USA 3: 1633–1638.

    Article  Google Scholar 

  9. Gunturk BK, Batur AU, Altunbasak Y, Hayes MH III, Mersereau RM: Eigenface-based super-resolution for face recognition. Proceedings of the IEEE International Conference on Image Processing (ICIP '02), September 2002, Rochester, NY, USA 2: 845–848.

    Google Scholar 

  10. Capel D, Zisserman A: Super-resolution enhancement of text image sequences. Proceedings of the 15th International Conference on Pattern Recognition (ICPR '00), September 2000, Barcelona, Spain 1: 600–605.

    Article  Google Scholar 

  11. Goyette JA, Lapin GD, Kang MG, Katsaggelos AK: Improving autoradiograph resolution using image restoration techniques. IEEE Engineering in Medicine and Biology Magazine 1994, 13(3):571–574.

    Google Scholar 

  12. Katsaggelos AK, Galatsanos NP (Eds): Signal Recovery Techniques for Image and Video Compression and Transmission. Kluwer Academic, Dordrecht, The Netherlands; 1998.

    Google Scholar 

  13. Chen D, Schultz RR: Extraction of high-resolution video stills from MPEG image sequences. Proceedings of the IEEE International Conference on Image Processing (ICIP '98), October 1998, Chicago, Ill, USA 2: 465–469.

    Article  Google Scholar 

  14. Erickson KJ, Schultz RR: MPEG-1 super-resolution decoding for the analysis of video still images. Proceedings of the 4th IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI '00), April 2000, Austin, Tex, USA 13–17.

    Chapter  Google Scholar 

  15. Martins B, Forchhammer S: A unified approach to restoration, deinterlacing and resolution enhancement in decoding MPEG-2 video. IEEE Transactions on Circuits and Systems for Video Technology 2002, 12(9):803–811. 10.1109/TCSVT.2002.803227

    Article  Google Scholar 

  16. Altunbasak Y, Patti AJ, Mersereau RM: Super-resolution still and video reconstruction from MPEG-coded video. IEEE Transactions on Circuits and Systems for Video Technology 2002, 12(4):217–226. 10.1109/76.999200

    Article  Google Scholar 

  17. Segall CA, Molina R, Katsaggelos AK: High-resolution images from low-resolution compressed video. IEEE Signal Processing Magazine 2003, 20(3):37–48. 10.1109/MSP.2003.1203208

    Article  Google Scholar 

  18. Candocia FM, Principe JC: Method using multiple models to superresolve SAR imagery. Algorithms for Synthetic Aperture Radar Imagery V, April 1998, Orlando, Fla, USA, Proceedings of SPIE 3370: 197–207.

    Article  Google Scholar 

  19. Cheng Y, Lu Y, Lin Z: A super resolution SAR imaging method based on CSA. Proceedings of the IEEE International Geoscience and Remote Sensing Symposium (IGARSS '02), June 2002, Toronto, Canada 6: 3671–3673.

    Article  Google Scholar 

  20. Pastina D, Lombardo P, Farina A, Daddi P: Super-resolution of polarimetric SAR images of a ship. Proceedings of the IEEE International Geoscience and Remote Sensing Symposium (IGARSS '01), July 2001, Sydney, NSW, Australia 5: 2343–2345.

    MATH  Google Scholar 

  21. Wu R, Li J: Autofocus and super-resolution synthetic aperture radar image formation. IEE Proceedings—Radar, Sonar and Navigation 2000, 147(5):217–223. 10.1049/ip-rsn:20000616

    Article  Google Scholar 

  22. Yamada H, Yamaguchi Y, Rodriguez E, Kim Y, Boerner WM: Polarimetric SAR interferometry for forest canopy analysis by using the super-resolution method. Proceedings of the IEEE International Geoscience and Remote Sensing Symposium (IGARSS '01), July 2001, Sydney, NSW, Australia 3: 1101–1103.

    Google Scholar 

  23. Liang X, Tao R, Zhou S, Wang Y: Multidimensional real aperture radar imaging. Proceedings of CIE International Conference on Radar, October 2001, Beijing, China 675–678.

    Google Scholar 

  24. Tatem AJ, Lewis HG, Atkinson PM, Nixon MS: Super-resolution mapping of multiple-scale land cover features using a Hopfield neural network. Proceedings of the IEEE International Geoscience and Remote Sensing Symposium (IGARSS '01) , July 2001, Sydney, NSW, Australia 7: 3200–3202.

    Google Scholar 

  25. Freeman WT, Jones TR, Pasztor EC: Example-based super-resolution. IEEE Computer Graphics and Applications 2002, 22(2):56–65. 10.1109/38.988747

    Article  Google Scholar 

  26. Nagahara H, Yagi Y, Yachida M: Super-resolution from an omnidirectional image sequence. Proceedings of the 26th Annual Conference of the IEEE Industrial Electronics Society (IECON '00), October 2000, Nagoya, Japan 4: 2559–2564.

    Google Scholar 

  27. Nagahara H, Yagi Y, Yachida M: Resolution improving method for a 3D environment modeling using omnidirectional image sensor. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA '02), May 2002, Washington, DC, USA 1: 900–907.

    Google Scholar 

  28. Elad M, Feuer A: Super-resolution reconstruction of image sequences. IEEE Transactions on Pattern Analysis and Machine Intelligence 1999, 21(9):817–834. 10.1109/34.790425

    Article  Google Scholar 

  29. Elad M, Feuer A: Superresolution restoration of an image sequence: adaptive filtering approach. IEEE Transactions on Image Processing 1999, 8(3):387–395. 10.1109/83.748893

    Article  Google Scholar 

  30. Huang TS, Tsay RY: Multiple frame image restoration and registration. In Advances in Computer Vision and Image Processing. Volume 1. Edited by: Huang TS. JAI Press, Greenwich, Conn, USA; 1984:317–339.

    Google Scholar 

  31. Yen JL: On nonuniform sampling of bandwidth-limited signals. IRE Transactions on Circuit Theory 1956, 3(4):251–257.

    Article  Google Scholar 

  32. Papoulis A: Generalized sampling expansion. IEEE Transactions on Circuits and Systems 1977, 24(11):652–654. 10.1109/TCS.1977.1084284

    Article  MathSciNet  Google Scholar 

  33. Huang TS (Ed): Image Sequence Processing and Dynamic Scene Analysis. Springer, Berlin, Germany; 1983.

    Google Scholar 

  34. Hunt BR, Kubler O: Karhunen-Loeve multispectral image restoration, part I: theory. IEEE Transactions on Acoustics, Speech, Signal Processing 1984, 32(3):592–600. 10.1109/TASSP.1984.1164363

    Article  Google Scholar 

  35. Zervakis ME: Optimal restoration of multichannel images based on constrained mean-square estimation. Journal of Visual Communication and Image Representation 1992, 3(4):392–411. 10.1016/1047-3203(92)90042-R

    Article  MathSciNet  Google Scholar 

  36. Katsaggelos AK: A multiple input image restoration approach. Journal of Visual Communication and Image Representation 1990, 1(1):93–103. 10.1016/1047-3203(90)90019-R

    Article  Google Scholar 

  37. Katsaggelos AK, Driessen JN, Efstratiadis SN, Lagendijk RL: Spatiotemporal motion-compensated noise filtering of image sequences. Visual Communications and Image Processing IV, November 1989, Philadelphia, Pa, USA, Proceedings of SPIE 1199: 61–70.

    Article  Google Scholar 

  38. Katsaggelos AK, Kleihorst RP, Efstratiadis SN, Lagendijk RL: Adaptive image sequence noise filtering methods. Visual Communications and Image Processing '91: Image Processing, November 1991, Boston, Mass, USA, Proceedings of SPIE 1606: 716–727.

    Google Scholar 

  39. Ho S-J, Lee YH: Nonlinear spatio-temporal noise suppression techniques with applications in image sequence processing. Proceedings of the IEEE International Sympoisum on Circuits and Systems (ISCAS '91), June 1991, Singapore 1: 662–665.

    Google Scholar 

  40. Patti AJ, Tekalp AM, Sezan MI: Image sequence restoration and deinterlacing by motion-compensated Kalman filtering. Image and Video Processing, February 1993, San Jose, Calif, USA, Proceedings of SPIE 1903: 59–70.

    Article  Google Scholar 

  41. Patti AJ, Sezan MI, Tekalp AM: High-resolution image reconstruction from a low-resolution image sequence in the presence of time-varying motion blur. Proceedings of the IEEE International Conference on Image Processing (ICIP '94), November 1994, Austin, Tex, USA 1: 343–347.

    Article  Google Scholar 

  42. Elad M, Feuer A: Recursive optical flow estimation-adaptive filtering approach. Proceedings of the 19th IEEE Convention of Electrical and Electronics Engineers in Israel, November 1996, Jerusalem, Israel 387–390.

    Chapter  Google Scholar 

  43. Schultz RR, Stevenson RL: Improved definition video frame enhancement. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '95), May 1995, Detroit, Mich, USA 4: 2169–2172.

    Google Scholar 

  44. Patti AJ, Sezan MI, Tekalp AM: High resolution standards conversion of low resolution video. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP '95), May 1995, Detroit, Mich, USA 4: 2197–2200.

    Google Scholar 

  45. Schultz RR, Stevenson RL: Extraction of high-resolution frames from video sequences. IEEE Transactions on Image Processing 1996, 5(6):996–1011. 10.1109/83.503915

    Article  Google Scholar 

  46. Elad M, Feuer A: Restoration of single super resolution image from several blurred, noisy, and undersampled measured images. IEEE Transactions on Image Processing 1997, 6(12):1646–1658. 10.1109/83.650118

    Article  Google Scholar 

  47. Kang MG, Katsaggelos AK: Simultaneous iterative image restoration and evaluation of the regularization parameter. IEEE Transactions on Signal Processing 1992, 40(9):2329–2334. 10.1109/78.157234

    Article  Google Scholar 

  48. Lagendijk RL, Biemond J: Iterative Identification and Restoration of Images. Kluwer Academic, Boston, Mass, USA; 1991.

    Book  Google Scholar 

  49. Jain AK: Fundamentals of Digital Image Processing. Prentice-Hall, Upper Saddle River, NJ, USA; 1989.

    MATH  Google Scholar 

  50. Pratt WK: Digital Image Processing. John Wiley & Sons, New York, NY, USA; 1991.

    MATH  Google Scholar 

  51. Zomet A, Rav-Acha A, Peleg S: Robust super-resolution. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '01), December 2001, Kauai Marriott, Hawaii, USA 1: 645–650.

    Google Scholar 

  52. Youla DC: Generalized image restoration by the method of alternating orthogonal projections. IEEE Transactions on Circuits and Systems 1978, 25(9):694–702. 10.1109/TCS.1978.1084541

    Article  MathSciNet  Google Scholar 

  53. Zhao W, Sawhney H, Hansen M, Samarasekera S: Super-fusion: a super-resolution method based on fusion. Proceedings of the 16th International Conference on Pattern Recognition (ICPR '02), August 2002, Quebec, Canada 2: 269–272.

    Article  Google Scholar 

  54. Irani M, Peleg S: Motion analysis for image enhancement: resolution, occlusion, and transparency. Journal of Visual Communication and Image Representation 1993, 4(4):324–335. 10.1006/jvci.1993.1030

    Article  Google Scholar 

  55. Freeman WT, Pasztor EC: Learning low-level vision. Proceedings of the 7th IEEE International Conference on Computer Vision (ICCV '99), September 1999, Kerkyra, Greece 2: 1182–1189.

    Google Scholar 

  56. Capel D, Zisserman A: Super-resolution from multiple views using learnt image models. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '01), December 2001, Kauai Marriott, Hawaii, USA 2: 627–634.

    Google Scholar 

  57. Baker S, Kanade T: Limits on super-resolution and how to break them. IEEE Transactions on Pattern Analysis and Machine Intelligence 2002, 24(9):1167–1183. 10.1109/TPAMI.2002.1033210

    Article  Google Scholar 

  58. Chaudhuri S (Ed): Super-Resolution Imaging. Kluwer Academic, Boston, Mass, USA; 2001.

    Google Scholar 

  59. Farsiu S, Robinson D, Elad M, Milanfar P: Fast and robust multi-frame super-resolution. IEEE Transactions on Image Processing 2004, 13(10):1327–1344. 10.1109/TIP.2004.834669

    Article  Google Scholar 

  60. de Beeck MO, Kleihorst RP: Super-resolution of regions of interest in a hybrid video encoder. Proceedings of Philips Conference on Digital Signal Processing (DSP '99), November 1999, Veldhoven, The Netherlands

    Google Scholar 

  61. Callicó GM, Llopis RP, Núñez A, Sethuraman R: Mapping of real-time and low-cost super-resolution algorithms onto a hybrid video encoder. VLSI Circuits and Systems, May 2003, Maspalomas, Spain, Proceedings of SPIE 5117: 42–52.

    Article  Google Scholar 

  62. Llopis RP, Sethuraman R, Pinto CA, Peters H, Maul S, Oosterhuis M: A low-cost and low-power multi-standard video encoder. Proceedings of the 1st IEEE/ACM/IFIP International Conference on Hardware/Software Codesign and System Synthesis, October 2003, Newport Beach, Calif, USA 97–102.

    Google Scholar 

  63. Callicó GM, Núñez A, Llopis RP, Sethuraman R, de Beeck MO: A low-cost implementation of super-resolution based on a video encoder. Proceedings of the 28th Annual Conference of the IEEE Industrial Electronics Society (IECON '02), November 2002, Sevilla, Spain 2: 1439–1444.

    Google Scholar 

  64. Llopis RP, Oosterhuis M, Ramanathan S, et al.: HW-SW co-design and verification of a multi-standard video and image codec. Proceedings of the 2nd IEEE International Symposium on Quality of Electronic Design (ISQED '01), March 2001, San Jose, Calif, USA 393–398.

    Google Scholar 

  65. Callicó GM, Núñez A, Llopis RP, Sethuraman R: Low-cost and real-time super-resolution over a video encoder IP. Proceedings of the 4th IEEE International Symposium on Quality Electronic Design (ISQED '03), March 2003, San Jose, Calif, USA 79–84.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute for Applied Microelectronics (IUMA), The University of Las Palmas de Gran Canaria, Tafira Baja, 35017, Spain

    Gustavo M. Callicó, Sebastian López, José Fco López, Antonio Núñez & Roberto Sarmiento

  2. Philips Consumer Electronics, SFJ-6, P.O. Box 80002, 5600, JB, The Netherlands

    Rafael Peset Llopis

  3. Philips Research Laboratories, WDC 3.33, Professor Holstlaan 4, Eindhoven, 5656, AA, The Netherlands

    Ramanathan Sethuraman

Authors
  1. Gustavo M. Callicó
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rafael Peset Llopis
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sebastian López
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. José Fco López
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Antonio Núñez
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ramanathan Sethuraman
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Roberto Sarmiento
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gustavo M. Callicó.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Callicó, G.M., Llopis, R.P., López, S. et al. Low-Cost Super-Resolution Algorithms Implementation Over a HW/SW Video Compression Platform. EURASIP J. Adv. Signal Process. 2006, 084614 (2006). https://doi.org/10.1155/ASP/2006/84614

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1155/ASP/2006/84614

Keywords