Skip to main content
  • Research Article
  • Open access
  • Published:

A MAP Estimator for Simultaneous Superresolution and Detector Nonunifomity Correction

Abstract

During digital video acquisition, imagery may be degraded by a number of phenomena including undersampling, blur, and noise. Many systems, particularly those containing infrared focal plane array (FPA) sensors, are also subject to detector nonuniformity. Nonuniformity, or fixed pattern noise, results from nonuniform responsivity of the photodetectors that make up the FPA. Here we propose a maximum a posteriori (MAP) estimation framework for simultaneously addressing undersampling, linear blur, additive noise, and bias nonuniformity. In particular, we jointly estimate a superresolution (SR) image and detector bias nonuniformity parameters from a sequence of observed frames. This algorithm can be applied to video in a variety of ways including using a moving temporal window of frames to process successive groups of frames. By combining SR and nonuniformity correction (NUC) in this fashion, we demonstrate that superior results are possible compared with the more conventional approach of performing scene-based NUC followed by independent SR. The proposed MAP algorithm can be applied with or without SR, depending on the application and computational resources available. Even without SR, we believe that the proposed algorithm represents a novel and promising scene-based NUC technique. We present a number of experimental results to demonstrate the efficacy of the proposed algorithm. These include simulated imagery for quantitative analysis and real infrared video for qualitative analysis.

References

  1. Milton AF, Barone FR, Kruer MR: Influence of nonuniformity on infrared focal plane array performance. Optical Engineering 1985,24(5):855-862.

    Article  Google Scholar 

  2. Gross W, Hierl T, Schultz M: Correctability and long-term stability of infrared focal plane arrays. Optical Engineering 1999,38(5):862-869. 10.1117/1.602055

    Article  Google Scholar 

  3. Perry DL, Dereniak EL: Linear theory of nouniformity correction in infrared staring sensors. Optical Engineering 1993,32(8):1854-1859. 10.1117/12.145601

    Article  Google Scholar 

  4. Nelson MD, Johnson JF, Lomheim TS: General noise processes in hybrid infrared focal plane arrays. Optical Engineering 1991,30(11):1682-1700. 10.1117/12.55996

    Article  Google Scholar 

  5. El Gamal A, Eltoukhy H: CMOS image sensors. IEEE Circuits and Devices Magazine 2005,21(3):6-20. 10.1109/MCD.2005.1438751

    Article  Google Scholar 

  6. Narendra PM, Foss NA: Shutterless fixed pattern noise correction for infrared imaging arrays. Technical Issues in Focal Plane Development, April 1981, Washington, DC, USA, Proceedings of SPIE 282: 44–51.

    Article  Google Scholar 

  7. Harris JG: Continuous-time calibration of VLSI sensors for gain and offset variations. In Smart Focal Plane Arrays and Focal Plane Array Testing, April 1995, Orlando, Fla, USA, Proceedings of SPIE Edited by: Wigdor M, Massie MA. 2474: 23–33.

    Chapter  Google Scholar 

  8. Harris JG, Chiang Y-M: Nonuniformity correction using the constant-statistics constraint: analog and digital implementations. In Infrared Technology and Applications XXIII, April 1997, Orlando, Fla, USA, Proceedings of SPIE Edited by: Andresen BF, Strojnik M. 3061: 895–905.

    Google Scholar 

  9. Chiang Y-M, Harris JG: An analog integrated circuit for continuous-time gain and offset calibration of sensor arrays. Analog Integrated Circuits and Signal Processing 1997,12(3):231-238. 10.1023/A:1008297408871

    Article  Google Scholar 

  10. Scribner DA, Sarkady KA, Caulfield JT, et al.: Nonuniformity correction for staring IR focal plane arrays using scene-based techniques. In Infrared Detectors and Focal Plane Arrays, April 1990, Orlando, Fla, USA, Proceedings of SPIE Edited by: Dereniak EL, Sampson RE. 1308: 224–233.

    Google Scholar 

  11. Scribner DA, Sarkady KA, Kruer MR, Caulfield JT, Hunt JD, Herman C: Adaptive nonuniformity correction for IR focal-plane arrays using neural networks. In Infrared Sensors: Detectors, Electronics, and Signal Processing, July 1991, San Diego, Calif, USA, Proceedings of SPIE Edited by: Jayadev TS. 1541: 100–109.

    Chapter  Google Scholar 

  12. Scribner DA, Sarkady KA, Kruer MR, et al.: Adaptive retina-like preprocessing for imaging detector arrays. Proceedings of IEEE International Conference on Neural Networks, March-April 1993, San Francisco, Calif, USA 3: 1955–1960.

    Article  Google Scholar 

  13. Narayanan B, Hardie RC, Muse RA: Scene-based nonuniformity correction technique that exploits knowledge of the focal-plane array readout architecture. Applied Optics 2005,44(17):3482-3491. 10.1364/AO.44.003482

    Article  Google Scholar 

  14. Hayat MM, Torres SN, Armstrong EE, Cain SC, Yasuda B: Statistical algorithm for nonuniformity correction in focal-plane arrays. Applied Optics 1999,38(5):772-780. 10.1364/AO.38.000772

    Article  Google Scholar 

  15. Torres SN, Hayat MM: Kalman filtering for adaptive nonuniformity correction in infrared focal-plane arrays. Journal of the Optical Society of America A 2003,20(3):470-480. 10.1364/JOSAA.20.000470

    Article  Google Scholar 

  16. Hardie RC, Hayat MM: A nonlinear-filter based approach to detector nonuniformity correction. Proceedings of IEEE-EURASIP Workshop on Nonlinear Signal and Image Processing, June 2001, Baltimore, Md, USA 66–85.

    Google Scholar 

  17. O'Neil WF: Dithered scan detector compensation. Proceedings of the Infrared Information Symposium (IRIS) Specialty Group on Passive Sensors, 1993, Ann Arbor, Mich, USA

    Google Scholar 

  18. O'Neil WF: Experimental verification of dither scan non-uniformity correction. Proceedings of the Infrared Information Symposium (IRIS) Specialty Group on Passive Sensors, 1997, Monterey, Calif, USA 1: 329–339.

    Google Scholar 

  19. Hardie RC, Hayat MM, Armstrong EE, Yasuda B: Scene-based nonuniformity correction with video sequences and registration. Applied Optics 2000,39(8):1241-1250. 10.1364/AO.39.001241

    Article  Google Scholar 

  20. Ratliff BM, Hayat MM, Hardie RC: An algebraic algorithm for nonuniformity correction in focal-plane arrays. Journal of the Optical Society of America A 2002,19(9):1737-1747. 10.1364/JOSAA.19.001737

    Article  Google Scholar 

  21. Ratliff BM, Hayat MM, Tyo JS: Radiometrically accurate scene-based nonuniformity correction for array sensors. Journal of the Optical Society of America A 2003,20(10):1890-1899. 10.1364/JOSAA.20.001890

    Article  Google Scholar 

  22. Ratliff BM, Hayat MM, Tyo JS: Generalized algebraic scene-based nonuniformity correction algorithm. Journal of the Optical Society of America A 2005,22(2):239-249. 10.1364/JOSAA.22.000239

    Article  Google Scholar 

  23. Sakoglu U, Hardie RC, Hayat MM, Ratliff BM, Tyo JS: An algebraic restoration method for estimating fixed-pattern noise in infrared imagery from a video sequence. Applications of Digital Image Processing XXVII, August 2004, Denver, Colo, USA, Proceedings of SPIE 5558: 69–79.

    Article  Google Scholar 

  24. Park SC, Park MK, Kang MG: Super-resolution image reconstruction: a technical overview. IEEE Signal Processing Magazine 2003,20(3):21-36. 10.1109/MSP.2003.1203207

    Article  Google Scholar 

  25. Borman S: Topics in multiframe superresolution restoration, Ph.D. dissertation.

  26. Schultz RR, Stevenson RL: A Bayesian approach to image expansion for improved definition. IEEE Transactions on Image Processing 1994,3(3):233-242. 10.1109/83.287017

    Article  Google Scholar 

  27. Cheeseman P, Kanefsky B, Kraft R, Stutz J, Hanson R: Super-resolved surface reconstruction from multiple images. In Tech. Rep. FIA-94-12. NASA, Moffett Field, Calif, USA; 1994.

    Google Scholar 

  28. Cain SC, Hardie RC, Armstrong EE: Restoration of aliased video sequences via a maximum-likelihood approach. Proceedings of National Infrared Information Symposium (IRIS) on Passive Sensors, March 1996, Monterey, Calif, USA 230–251.

    Google Scholar 

  29. Hardie RC, Barnard KJ, Armstrong EE: Joint MAP registration and high-resolution image estimation using a sequence of undersampled images. IEEE Transactions on Image Processing 1997,6(12):1621-1633. 10.1109/83.650116

    Article  Google Scholar 

  30. Segall CA, Katsaggelos AK, Molina R, Mateos J: Bayesian resolution enhancement of compressed video. IEEE Transactions on Image Processing 2004,13(7):898-910. 10.1109/TIP.2004.827230

    Article  Google Scholar 

  31. Armstrong EE, Hayat MM, Hardie RC, Torres SN, Yasuda BJ: Nonuniformity correction for improved registration and high-resolution image reconstruction in IR imagery. In Applications of Digital Image Processing XXII, July 1999, Denver, Colo, USA, Proceedings of SPIE Edited by: Tescher AG. 3808: 150–161.

    Chapter  Google Scholar 

  32. Armstrong EE, Hayat MM, Hardie RC, Torres SN, Yasuda B: The advantage of non-uniformity correction pre-processing on infrared image registration. Application of Digital Image Processing XXII, July 1999, Denver, Colo, USA, Proceedings of SPIE 3808:

    Google Scholar 

  33. Cain S, Armstrong EE, Yasuda B: Joint estimation of image, shifts, and nonuniformities from IR images. In Infrared Information Symposium (IRIS) on Passive Sensors, 1997, Ann Arbor, Mich, USA. Infrared Information Analysis Center, ERIM International;

    Google Scholar 

  34. Geman S, Geman D: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence 1984,6(6):721-741.

    Article  MATH  Google Scholar 

  35. Besag J: Spatial interaction and the statistical analysis of lattice systems. Journal of the Royal Statistical Society B 1974,36(2):192-236.

    MathSciNet  MATH  Google Scholar 

  36. Derin H, Elliott E: Modeling and segmentation of noisy and textured images using Gibbs random fields. IEEE Transactions on Pattern Analysis and Machine Intelligence 1987,9(1):39-55.

    Article  Google Scholar 

  37. Cain SC, Hayat MM, Armstrong EE: Projection-based image registration in the presence of fixed-pattern noise. IEEE Transactions on Image Processing 2001,10(12):1860-1872. 10.1109/83.974571

    Article  MATH  Google Scholar 

  38. Hardie RC, Barnard KJ, Bognar JG, Armstrong EE, Watson EA: High-resolution image reconstruction from a sequence of rotated and translated frames and its application to an infrared imaging system. Optical Engineering 1998,37(1):247-260. 10.1117/1.601623

    Article  Google Scholar 

  39. Irani M, Peleg S: Improving resolution by image registration. CVGIP: Graphical Models and Image Processing 1991,53(3):231-239. 10.1016/1049-9652(91)90045-L

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Russell C. Hardie.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://doi.org/creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Hardie, R.C., Droege, D.R. A MAP Estimator for Simultaneous Superresolution and Detector Nonunifomity Correction. EURASIP J. Adv. Signal Process. 2007, 089354 (2007). https://doi.org/10.1155/2007/89354

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1155/2007/89354

Keywords