Real-Time Tracking with Mean Shift

1. Real-Time Tracking with Mean Shift Presented by: Qiuhua Liu May 6, 2005

2. Outline • Color model for the target • Introduction to mean-shift • Tracking algorithm with mean shift • Compassion with Particle Filter algorithm with the similar color model

3. Color Model for the Target • The target is represented by an ellipsoidal region in the image, normalized to a unit circle. Let be the normalized pixel locations in the region centered at 0. • The probability of the feature(color) of the target was modeled by the its histogram with kernel : The kernel has a convex and monotonic decreasing kernel profile , assigning small weights to pixels farther away from the center.

4. Target Candidate • The profile of kernel is defined as a function such that • Let be the normalized pixel locations of the target candidates, centered at yin the current frame. The target candidate is modeled as:

5. Similarity Function • The similarity function is defined as the metric distance between the candidate and the target model: • Choose as the Bhattacharyya coefficients (it is a divergence type measure) • Minimizing the distance is equivalent to maximizing .

6. where Maximization with Mean Shift • Assume the target candidate histogram does not change drastically, using Taylor expansion around the values at location : • Only need to maximize the second term, which is the density estimate with kernel profile k(x) at y in the current frame, with the data being weighted by wi.

7. d: dimension of data; h: band width. Mean Shift • First Introduced by Fukunaga and Hostetler in 1975 [1], Mean shift is a non-parametric, iterative procedure to find the mode of a density function represented by a set of samples and a Kernel K : With the definition of the Profile of a kernel:

8. where Mean Shift • With mean shift method, the kernel is recursively moved from the current location to the new location until converge with: • For a kernel with a convex and monotonic decreasing kernel profile, it is guaranteed to converge (to local maxima)

9. One Normally Used Kernel • The Epanechnikov kernel has a profile: • Then where cd is the volume of the unit d -dimensional sphere. (*)

10. Tracking Algorithm with Mean Shift • Very Simple: Given the target model and its location in the previous frame. 1. Initialize the location at the current frame with . 2. Compute the next location according to (*). 3. Iterate 1 and 2 until converge.

11. Tracked Result:

12. Mean Shift Maximization:

13. Summary and Comparison to Particle Filter Method • Advantage: • Good color histogram model and distance measure. • Deterministic method: the mean shift usually converged at 2 to 3 iterations –Fast. • Disadvantage: • Sometimes get stuck at local minimum. • Difficult to handle abrupt motion: Due to use of the kernels, the center of the target in the current frame has to be covered by the target model in the previous frame. Otherwise, the local maximum of the Bhattacharyya coefficient would not be a reliable indicator.

14. Connection to Particle Filter Tracking • Adopting the same distance measure, Jaco Vermaak [4][5] proposed the following observation likelihood function for probabilistic tracking with particle filters and VB inference : The histogram does not necessarily need a kernel.

15. Comparison Top: Deterministic with Mean-shift Bottom: Probabilistic with particle filters

16. References [1] Fukunaga et al, “The Estimation of the Gradient of a Density Function, with Applications in Pattern Recognition”, IEEE Trans. on Information Theory, 1975 [2] Dorin Comaniciu et al, “Real-time Tracking of Non-Rigid Objects Using Mean Shift”, CVPR 2000. [3] Dorin Comaniciu et al, “Kernel-Based Object Tracking”, IEEE Trans . On Pattern Analysis and Machine Learning , May 2003. [4] Jaco Vermaak et al [5] Jaco Vermaak et al, “Variational Inference for Visual Tracking”, CVPR, 2003