Skip to main content
  • Research Article
  • Open access
  • Published:

Unsupervised Performance Evaluation of Image Segmentation

Abstract

We present in this paper a study of unsupervised evaluation criteria that enable the quantification of the quality of an image segmentation result. These evaluation criteria compute some statistics for each region or class in a segmentation result. Such an evaluation criterion can be useful for different applications: the comparison of segmentation results, the automatic choice of the best fitted parameters of a segmentation method for a given image, or the definition of new segmentation methods by optimization. We first present the state of art of unsupervised evaluation, and then, we compare six unsupervised evaluation criteria. For this comparative study, we use a database composed of 8400 synthetic gray-level images segmented in four different ways. Vinet's measure (correct classification rate) is used as an objective criterion to compare the behavior of the different criteria. Finally, we present the experimental results on the segmentation evaluation of a few gray-level natural images.

References

  1. Freixenet J, Muñoz X, Raba D, Marti J, Cufi X: Yet another survey on image segmentation: region and boundary information integration. Proceedings of the European Conference on Computer Vision (ECCV '02), May 2002, Copenhagen, Denmark 408–422.

    MATH  Google Scholar 

  2. Haralick RM, Shapiro LG: Image segmentation techniques. Computer Vision, Graphics, & Image Processing 1985, 29(1):100–132. 10.1016/S0734-189X(85)90153-7

    Article  Google Scholar 

  3. Zhang YJ: A survey on evaluation methods for image segmentation. Pattern Recognition 1996, 29(8):1335–1346. 10.1016/0031-3203(95)00169-7

    Article  Google Scholar 

  4. Nasab NM, Analoui M, Delp EJ: Robust and efficient image segmentation approaches using Markov random field models. Journal of Electronic Imaging 2003, 12(1):50–58. 10.1117/1.1525280

    Article  Google Scholar 

  5. Baddeley AJ: An error metric for binary images. In Robust Computer Vision. Wichmann, Karlsruhe, Germany; 1992:59–78.

    Google Scholar 

  6. Vinet L: Segmentation et mise en correspondance de régions de paires d'images stéréoscopiques, M.S. thesis. Université de Paris IX Dauphine, Paris, France; 1991.

    Google Scholar 

  7. Huttenlocher DP, Rucklidge WJ: Multi-resolution technique for comparing images using the Hausdorff distance. Proceedings of IEEE Computer Vision and Pattern Recognition (CVPR '93), June 1993, New York, NY, USA 705–706.

    Chapter  Google Scholar 

  8. Martin D, Fowlkes C, Tal D, Malik J: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. Proceedings of the IEEE International Conference on Computer Vision (ICCV '01), July 2001, Vancouver, BC, Canada 2: 416–423.

    Google Scholar 

  9. Weszka JS, Rosenfeld A: Threshold evaluation techniques. IEEE Transactions on Systems, Man and Cybernetics 1978, 8(8):622–629.

    Article  Google Scholar 

  10. Levine MD, Nazif AM: Dynamic measurement of computer generated image segmentations. IEEE Transactions on Pattern Analysis and Machine Intelligence 1985, 7(2):155–164.

    Article  Google Scholar 

  11. Sezgin M, Sankur B: Survey over image thresholding techniques and quantitative performance evaluation. Journal of Electronic Imaging 2004, 13(1):146–168. 10.1117/1.1631315

    Article  Google Scholar 

  12. Cochran WG:Some methods for strengthening the common tests. Biometrics 1954, 10: 417–451. 10.2307/3001616

    MathSciNet  MATH  Google Scholar 

  13. Pal NR, Pal SK: Entropic thresholding . Signal Processing 1989, 16(2):97–108. 10.1016/0165-1684(89)90090-X

    Article  MathSciNet  Google Scholar 

  14. Rosenberger C: Mise en oeuvre d'un système adaptatif de segmentation d'images, M.S. thesis. Université de Rennes 1, Rennes, France; 1999.

    Google Scholar 

  15. Borsotti M, Campadelli P, Schettini R: Quantitative evaluation of color image segmentation results. Pattern Recognition Letters 1998, 19(8):741–747. 10.1016/S0167-8655(98)00052-X

    Article  MATH  Google Scholar 

  16. Liu J, Yang Y-H: Multiresolution color image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 1994, 16(7):689–700. 10.1109/34.297949

    Article  Google Scholar 

  17. Zeboudj R: Filtrage, seuillage automatique, contraste et contours: du pré-traitement à l'analyse d'image, M.S. thesis. Université de Saint Etienne, Saint Etienne, France; 1988.

    Google Scholar 

  18. Chabrier S, Rosenberger C, Laurent H, Emile B, Marché P: Evaluating the segmentation result of a gray-level image. Proceedings of 12th European Signal Processing Conference (EUSIPCO '04), September 2004, Vienna, Austria 953–956.

    Google Scholar 

  19. Chabrier S, Emile B, Laurent H, Rosenberger C, Marché P: Unsupervised evaluation of image segmentation application to multi-spectral images. Proceedings of International Conference on Pattern Recognition (ICPR '04), August 2004, Cambridge, UK 1: 576–579.

    Article  Google Scholar 

  20. Krishnapuram R, Keller JM:Possibilistic-means algorithm: insights and recommendations. IEEE Transactions on Fuzzy Systems 1996, 4(3):385–393. 10.1109/91.531779

    Google Scholar 

  21. Comaniciu D, Meer P: Mean shift: a robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 2002, 24(5):603–619. 10.1109/34.1000236

    Article  Google Scholar 

  22. Monawer HA: Image vector quantization using a modified LBG algorithm with approximated centroids. Electronics Letters 1995, 31(3):174–175. 10.1049/el:19950100

    Article  Google Scholar 

  23. Erdem ÇE, Sankur B, Tekalp AM: Performance measures for video object segmentation and tracking. IEEE Transactions on Image Processing 2004, 13(7):937–951. 10.1109/TIP.2004.828427

    Article  Google Scholar 

  24. Nazif AM, Levine MD: Low level image segmentation: an expert system. IEEE Transactions on Pattern Analysis and Machine Intelligence 1984, 6(5):555–577.

    Article  Google Scholar 

  25. Krishnapuram R, Keller JM: Possibilistic approach to clustering. IEEE Transactions on Fuzzy Systems 1993, 1(2):98–110. 10.1109/91.227387

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Sebastien Chabrier.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Chabrier, S., Emile, B., Rosenberger, C. et al. Unsupervised Performance Evaluation of Image Segmentation. EURASIP J. Adv. Signal Process. 2006, 096306 (2006). https://doi.org/10.1155/ASP/2006/96306

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1155/ASP/2006/96306

Keywords