Boundary Extraction in Natural Images Using Ultrametric Contour Maps

1. Boundary Extraction in Natural Images Using Ultrametric Contour Maps Pablo Arbeláez Université Paris Dauphine Presented by Derek Hoiem

2. What is segmentation?

3. What is segmentation? • Segmentation is a result

4. Face Woman What is segmentation? • Segmentation is a result • Segmentation is a process

5. What is segmentation? • Segmentation is a result • Segmentation is a process • Segmentation is a guide

6. Segmentation as a Guide • Multiple Segmentations

7. Segmentation as a Guide • Multiple Segmentations • Hierarchy of Segmentations

8. Key Concepts/Contributions • Hierarchical segmentation by iterative merging • Ultrametric dissimilarities • Thorough evaluation on BSDS

9. λ Hierarchical Segmentation 3 Region Image Dendrogram Contour Image

10. λ Ultrametric Contour Map • Ultrametric • Definition: D(x,y) <= max{ D(x,z), D(z,y) } The union R12 of two regions R1 and R2 must have >= distance to adjacent region R3 than either R1or R2

11. Ultrametric Contour Map

12. Region Dissimilarity • Dc(R1, R2): mean boundary contrast • contrast(x) = max L*a*b* diff within radius of x • Dg(R1, R2): mean boundary gradient • gradient(x) = Pb(x) • Da(R1): Area + α3 Scatter (in color space) α2 D(R1, R2) = [Dc(R1, R2) + α1 Dg(R1, R2)] · min{ Da(R1), Da(R2)} Learned Parameters: xi = 4.5 α1 = 5 α2 = 0.2 α3 = 0

13. Examples Contrast Contrast + Gradient Contrast + Gradient + Region

14. Algorithm Summary • Create Initial Contours: • Extrema in gray channel form regions • Assign pixels to regions based on above ultrametric • Iteratively merge regions • Keep adjacency/distance matrix

15. Comparison • Martin et al. (Pb) • Canny edge detector • Hierarchical watersheds (using MFM for gradient) [Najman and Schmitt 1996] • Variational (global energy minimization)

16. Pb Brightness Gradient Oriented Edges Color Gradient Texture Gradient No Boundary Boundary [Martin Fowlkes Malik 2004]

17. Pb

18. Variational Method Originally Wavelet-based Textons [Koepfler Lopez Morel 1994]

19. Comparison • MFM: Martin et al. (Pb) • Canny: Canny edge detector • WS: Hierarchical watersheds (using MFM for gradient) [Najman and Schmitt 1996] • MS: Variational (global energy minimization) Edge-Based Region-Based

20. Comparison

21. Results

22. Results

23. Best Results http://www.ceremade.dauphine.fr/~arbelaez/results-UCM/main.html

24. Best Results http://www.ceremade.dauphine.fr/~arbelaez/results-UCM/main.html

25. Best Results http://www.ceremade.dauphine.fr/~arbelaez/results-UCM/main.html

26. Best Results http://www.ceremade.dauphine.fr/~arbelaez/results-UCM/main.html

27. Median Results

28. Median Results

29. Median Results

30. Median Results

31. Worst Results

32. Worst Results

33. Worst Results

34. Worst Results

35. Hierarchies vs. Multiple Segmentations

36. Revising Segmentation