گروه بینایی ماشین و پردازش تصویر Machine Vision and Image Processing Group

1. گروه بینایی ماشین و پردازش تصویر Machine Vision and Image Processing Group Mean Shift ;Theory and Applications Presented by: Reza Hemati دی 89 December 2010

2. Mean ShiftTheory and Applications Yaron Ukrainitz & Bernard Sarel

3. Mean Shift Theory • What is Mean Shift ? • Density Estimation Methods • Deriving the Mean Shift • Mean shift properties • Applications • Clustering • Discontinuity Preserving Smoothing • Segmentation • Object Tracking • Object Contour Detection Agenda

4. Mean Shift Theory

5. Region of interest Center of mass Intuitive Description Mean Shift vector Objective : Find the densest region Distribution of identical billiard balls

6. Region of interest Center of mass Intuitive Description Mean Shift vector Objective : Find the densest region Distribution of identical billiard balls

7. Region of interest Center of mass Intuitive Description Mean Shift vector Objective : Find the densest region Distribution of identical billiard balls

8. Region of interest Center of mass Intuitive Description Mean Shift vector Objective : Find the densest region Distribution of identical billiard balls

9. Region of interest Center of mass Intuitive Description Mean Shift vector Objective : Find the densest region Distribution of identical billiard balls

10. Region of interest Center of mass Intuitive Description Mean Shift vector Objective : Find the densest region Distribution of identical billiard balls

11. Region of interest Center of mass Intuitive Description Objective : Find the densest region Distribution of identical billiard balls

12. Data A tool for: Finding modes in a set of data samples, manifesting an underlying probability density function (PDF) in RN What is Mean Shift ? • PDF in feature space • Color space • Scale space • Actually any feature space you can conceive • … Non-parametric Density Estimation Discrete PDF Representation Non-parametric Density GRADIENT Estimation (Mean Shift) PDF Analysis

13. Assumption : The data points are sampled from an underlying PDF Non-Parametric Density Estimation Data point density implies PDF value ! Assumed Underlying PDF Real Data Samples

14. Non-Parametric Density Estimation Assumed Underlying PDF Real Data Samples

15. Non-Parametric Density Estimation ? Assumed Underlying PDF Real Data Samples

16. Assumption : The data points are sampled from an underlying PDF Parametric Density Estimation Estimate Assumed Underlying PDF Real Data Samples

17. Data A function of some finite number of data points x1…xn Kernel Density Estimation Parzen Windows - Function Forms In practice one uses the forms: or Same function on each dimension Function of vector length only

18. Data A function of some finite number of data points x1…xn Kernel Density EstimationVarious Kernels • Examples: • Epanechnikov Kernel • Uniform Kernel • Normal Kernel

19. Gradient Give up estimating the PDF ! Estimate ONLYthe gradient Kernel Density Estimation Using the Kernel form: We get : Size of window

20. Computing The Mean Shift Gradient Kernel Density Estimation

21. Computing The Mean Shift Yet another Kernel density estimation ! • Simple Mean Shift procedure: • Compute mean shift vector • Translate the Kernel window by m(x)

22. What happens if we reach a saddle point ? Mean Shift Mode Detection Perturb the mode position and check if we return back • Updated Mean Shift Procedure: • Find all modes using the Simple Mean Shift Procedure • Prune modes by perturbing them (find saddle points and plateaus) • Prune nearby – take highest mode in the window

23. Automatic convergence speed – the mean shift vector size depends on the gradient itself. • Near maxima, the steps are small and refined • For Uniform Kernel ( ), convergence is achieved in a finite number of steps • Normal Kernel ( ) exhibits a smooth trajectory, but is slower than Uniform Kernel ( ). Mean Shift Properties Adaptive Gradient Ascent

24. Real Modality Analysis Tessellate the space with windows Run the procedure in parallel

25. Real Modality Analysis The blue data points were traversed by the windows towards the mode

26. Real Modality AnalysisAn example Window tracks signify the steepest ascent directions

27. Strengths : • Application independent tool • Suitable for real data analysis • Does not assume any prior shape (e.g. elliptical) on data clusters • Can handle arbitrary feature spaces • Only ONE parameter to choose • h (window size) has a physical meaning, unlike K-Means • Weaknesses : • The window size (bandwidth selection) is not trivial • Inappropriate window size can cause modes to be merged, or generate additional “shallow” modes  Use adaptive window size Mean Shift Strengths & Weaknesses

28. Mean Shift Applications

29. Cluster : All data points in the attraction basin of a mode Clustering Attraction basin : the region for which all trajectories lead to the same mode Mean Shift : A robust Approach Toward Feature Space Analysis, by Comaniciu, Meer

30. ClusteringSynthetic Examples Simple Modal Structures Complex Modal Structures

31. Feature space: L*u*v representation Initial window centers ClusteringReal Example Modes found Modes after pruning Final clusters

32. ClusteringReal Example L*u*v space representation

33. 2D (L*u) space representation Final clusters ClusteringReal Example

34. Feature space : Joint domain = spatial coordinates + color space Discontinuity Preserving Smoothing Meaning : treat the image as data points in the spatial and gray level domain Image Data (slice) Mean Shift vectors Smoothing result Mean Shift : A robust Approach Toward Feature Space Analysis, by Comaniciu, Meer

35. z y Discontinuity Preserving Smoothing Flat regions induce the modes !

36. The effect of window size in spatial and range spaces Discontinuity Preserving Smoothing

37. Discontinuity Preserving SmoothingExample

38. Discontinuity Preserving SmoothingExample

39. Segment = Cluster, or Cluster of Clusters • Algorithm: • Run Filtering (discontinuity preserving smoothing) • Cluster the clusters which are closer than window size Segmentation Image Data (slice) Mean Shift vectors Segmentation result Smoothing result Mean Shift : A robust Approach Toward Feature Space Analysis, by Comaniciu, Meer http://www.caip.rutgers.edu/~comanici

40. SegmentationExample …when feature space is only gray levels…

41. SegmentationExample

42. SegmentationExample

43. SegmentationExample

44. SegmentationExample

45. SegmentationExample

46. SegmentationExample

47. Non-Rigid Object Tracking … …

48. Choose a reference model in the current frame Choose a feature space Represent the model in the chosen feature space … … Current frame Mean-Shift Object TrackingGeneral Framework: Target Representation

49. Search in the model’s neighborhood in next frame Find best candidate by maximizing a similarity func. Model Candidate … … Current frame Mean-Shift Object TrackingGeneral Framework: Target Localization Start from the position of the model in the current frame Repeat the same process in the next pair of frames

50. Choose a reference target model Choose a feature space Represent the model by its PDF in the feature space Quantized Color Space Mean-Shift Object TrackingTarget Representation Kernel Based Object Tracking, by Comaniniu, Ramesh, Meer