Chapter 14: SEGMENTATION BY CLUSTERING

1. Chapter 14:SEGMENTATION BYCLUSTERING

3. Outline • Introduction • Human Vision & Gestalt Properties • Applications • Background Subtraction • Shot Boundary Detection • Segmentation by Clustering • Simple Clustering Methods • K-means • Segmentation by Graph-Theoretic Clustering • The Overall Approach • Affinity Measures • Normalized Cut

4. Introduction

5. General Ideas • Tokens • Whatever we need to group (pixels, points, surface elements, etc., etc.) • Top down segmentation • Tokens belong together because they lie on the same object • Bottom up segmentation • Tokens belong together because they are locally coherent

6. Why & What is segmentation • Emphasize the property • Make main idea interesting • There are too many pixels in a single image • We need some form of compact, summary representation

7. What segmentation can do • Summarizing video • Finding machined parts • Finding people • Finding buildings in satellite images • Searching a collection of images

8. Model Problems • Forming image segments • Roughly coherent color and texture • Finding body segments • Arms, torso, etc. • Fitting lines to edge points • To fit a line to a set of points

9. Segmentation as Clustering • Partitioning • Decompose an image into regions • Decompose an image into extended blobs • Take a video sequence and decompose it into shots • Grouping

10. Human Vision & Gestalt Properties

11. Figure-ground discrimination • Grouping can be seen in terms of allocating some elements to a figure, some to ground • Can be based on local bottom-up cues or high level recognition

12. Figure-ground discrimination

13. A series of factors affect whether elements should be grouped together

14. A series of factors affect whether elements should be grouped together

17. Grouping by occlusions

18. Grouping by invisible completions

19. Outline • Introduction • Human Vision & Gestalt Properties • Applications • Background Subtraction • Shot Boundary Detection • Segmentation by Clustering • Simple Clustering Methods • K-means • Segmentation by Graph-Theoretic Clustering • The Overall Approach • Affinity Measures • Normalized Cut

20. ApplicationBackground Subtraction • Anything that doesn’t look like a known background is interesting

21. Take a picture? • Not good… • The background changes slowly overtime…

22. Estimate the value of background pixels using a moving average. • Ideally, the moving average should track the changes in the background.

24. Shot Boundary Detection • Where substantial changes in a video are interesting.

26. Computing distance • Frame differencing • Histogram-based • Block comparison • Edge differencing pixel-by-pixel histogram-by-histogram box-by-box edge map-by-edge map

27. Outline • Introduction • Human Vision & Gestalt Properties • Applications • Background Subtraction • Shot Boundary Detection • Segmentation by Clustering • Simple Clustering Methods • K-means • Segmentation by Graph-Theoretic Clustering • The Overall Approach • Affinity Measures • Normalized Cut

28. Segmentation by ClusteringSimple Clustering Methods • Divisive clustering

29. Divisive clustering p, q, r, s, t r, s, t p, q s, t s t r p q

30. Agglomerative clustering

31. Agglomerative clustering p, q, r, s, t r, s, t p, q s, t p q r s t

32. Q: what the distance of A:

33. Single-link clustering

34. Complete-link clustering

35. Group average clustering

36. K-means

37. 37

39. Outline • Introduction • Human Vision & Gestalt Properties • Applications • Background Subtraction • Shot Boundary Detection • Segmentation by Clustering • Simple Clustering Methods • K-means • Segmentation by Graph-Theoretic Clustering • The Overall Approach • Affinity Measures • Normalized Cut

40. Segmentation by Graph-theoretic Clustering • Terminology for Graph • Vertices • Edges • Directed graph • Undirected graph • Weighted graph • Self-loop • Connected graph • Connected components 2 4 3

41. The Overall Approach • Weighted graph represented by a square matrix Weighted matrix Undirected weighted graph

42. Set of well separated points Affinity matrix

43. Affinity Measures • Affinity by Distance

44. Affinity by Intensity • Affinity by Color • Affinity by Texture

45. Eigenvectors and segmentation • Extracting a Single Good Cluster :Element affinities { association of element I with cluster n } × {affinity between I and j}× { association of element j with cluster n} :The vector of weight linking elements to the th cluster

46. Scaling by scales • Normalizes the weights by requiring that • Differentiation and dropping Lagrangian :eigenvector of

47. Small

48. Extracting Weights for a Set of Clusters • quite tight • distinct. The three largest eigenvalues of the affinity matrix for the dataset

50. Outline • Introduction • Human Vision & Gestalt Properties • Applications • Background Subtraction • Shot Boundary Detection • Segmentation by Clustering • Simple Clustering Methods • K-means • Segmentation by Graph-Theoretic Clustering • The Overall Approach • Affinity Measures • Normalized Cut