Statistical Region Merging

1. Statistical Region Merging R. Nock and F. Nielsen IEEE Transactions on pattern analysis and machine intelligence, Vol 26, Issue 11, p.p. 1452-1458, Nov. 2004

2. Outline • 1. Introduction • 2. The model of image generation • 3. Theoretical analysis and algorithms • 4. Experimental results • 5. Conclusion

3. 1. Introduction • Segmentation is a tantalizing and central problem for image processing. • A prominent trend in grouping focuses on graph theorem. • The authors proposed a different strategy which belongs to region growing/merging techniques. • Regions are sets of pixels with homogeneous properties and are iteratively grown by combining smaller regions. • Region growing/merging techniques usually work with a statistical test to decide the merging of regions. • A good region merging algorithm has to find a good balance between preserving the perceptual units and the risk of overmerging for the remaining region.

4. 1. Introduction • A novel model of image generation and the segmentation approach are proposed. • To reconstruct the true region from the observed region. • With high probability, it suffers only the overmerging problem in segmentation. • With high probability, it has small overmerging error. • Fast and easily implementable. • Can be used to images with many channels. • Can handling noise and occlusions.

5. 2 The model of image generation • 1. Introduction • 2. The model of image generation • 2.1 The model of image generation • 3. Theoretical analysis and algorithms • 4. Experimental results • 5. Conclusion

6. 2.1 The model of image generation • The observed image, I, contains |I| pixels, each containing RGB values and belonging to the set {1,2,...,g}

7. 2.1 The model of image generation • The observed color channel is sampled from a family of Q distributions at each pixel of a perfect scene, I*. (Range of the Q distributions are bounded by g/Q)

8. 2.1 The model of image generationAn example • Example of some true image I* (expectation) and the observed image I.

9. 2.1 The model of image generationhomogeneity property • In I*, the optimal regions share a homogeneity property: • Inside a region, the statistical pixels have the same expectation for every color channel. • Different regions have different expectations for at least one color channel. • Inside a region, all distributions associated to each pixel can be different, as long as the homogeneity property is satisfied.

10. 3. Theoretical analysis and algorithms • 1. Introduction • 2. The model of image generation • 3. Theoretical analysis and algorithms • 3.1 Theoretical analysis • Merging predicate • Order in merging • 3.2 Other properties of the proposed approach • 3.3 Proposed algorithm: SRM • 4. Experimental results • 5. Conclusion

11. 3.1 Theoretical analysis and algorithmsTheoretical analysis • Two essential components in defining a region merging algorithm: • Merging predicate: define how to merge to undetermined region. • Order in merging: define an order to be followed to check the merging predicate.

12. 3.1 Theoretical analysis and algorithmsMerging predicate • Theorem 1 (The independent bounded difference inequality). Let be a vector of n R.V.s. Suppose the real-valued function f satisfies whenever vectors x and x’ differ only in kth coordinate. Then, for any , where is the expected value of the R.V. f(X)

13. 3.1 Theoretical analysis and algorithmsMerging predicate • From thm 1, we obtain the result on the deviation of observed differences between regions of I. • Corollary 1. Consider a fixed couple (R,R’) of regions of I. , the probability is no more than that

14. 3.1 Theoretical analysis and algorithmsMerging predicate • In the same statistical region, and with a high probability that does not exceed . • Merging predicate : merge R and R’ iff

15. 3.1 Theoretical analysis and algorithmsOrder in merging • Ideally, the order to test the merging of regions is: • when any test between two true regions occurs, that means that all tests inside each of the two true regions have previously occurred.

16. 3.2 Theoretical analysis and algorithmsOther properties of the proposed approach • The proposed approach is proved that only overmerging occurs, with high probability. • The proposed approach has been shown to have an upperbound on the error incurred w.r.t. the optimal sementation, with high probability. • The proposed approach is easily extended to numerical channels, such as RGB.

17. 3.3 Theoretical analysis and algorithmsProposed algorithm: SRM • To choose a merging predicate and order in merging to approximate the ideal segmentation method. • Merging predicate: merge R and R’ iff • Order in merging: choose a real-valued function f and radix sort f(.,.) to approximate the order in merging. ( O(|I|*log(g)) )

18. 4. Experimental results • 1. Introduction • 2. The model of image generation • 3. Theoretical analysis and algorithms • 4. Experimental results • 4.1 Choice of f • 4.2 Noise handling • 4.3 Enhance the noise handling ability • 4.4 Handling occlusions • 4.5 Controlling the scale of the segmentation • 5. Conclusion

19. 4.1 Experimental resultsChoice of f • Choose , where and are the pixel channel values. • The preordering can manage dramatic improvements over conventional scanning.

20. 4.1 Experimental resultsChoice of f • A second choice of f is to use and in Sobel filters, where smoothing filter is performed by [1 2 1] and derivative filter is [-1 0 0 1].

21. 4.1 Experimental resultsChoice of f • Comparison of the two choices of f

22. 4.2 Experimental resultsNoise handling • Two noise types to be handled: • Transmission noise t(q): chosen uniformly in {1,2,...,g} • Salt and pepper noise s(q): chosen uniformly in {1,g}

23. 4.3 Experimental resultsEnhance the noise handling ability • By integrating the moving average operators, the first kind of f: is replaced by • For the second kind of f, the smoothing filter is extended to be [1 2 ... △+1 △ ... 1], and the derivative filter is extended to be [-△ -△+1 ... △].

24. 4.3 Experimental resultsEnhance the noise handling ability • Noise handling ability of the extended SRM methods.

25. 4.4 Experimental resultsHandling occlusions • First run SRM as already presented. • In a second stage, run SRM again with the modification of to , and 4-connexity to clique connexity. • Radix sorting with f has an overall time complexity O( (|I|+k2)logg) ).

26. 4.4 Experimental resultsHandling occlusions • SRM with occlusion handling.

27. 4.5 Experimental resultsControlling the scale of the segmentation • The objective of multiscale segmentation is to get a hierarchy of segmentations at different scales. • In SRM, scale is controlled by tuning of parameter Q: as Q increases, the regions found are getting smaller.

28. 5. Conclusion • 1. Introduction • 2. The model of image generation • 3. Theoretical analysis and algorithms • 4. Experimental results • 5. Conclusion

29. 5. Conclusion • A novel model of image generation is proposed, which captures the idea that grouping is an inference problem. • A simple merging predicate and ordering in merging are provided. • SRM suffers only overmerging problems and achieves low error in segmentation, both with high probability. • SRM is very fast (segments a 512x512 image is in about one second on an Intel Pentium 4 2.4G processor) • SRM is able to cope with significant noise corruption, handling occlusions, and perform scale-sensitive segmentations.