Granularity and Elasticity Adaptation in Visual Tracking

1. Granularity and Elasticity Adaptation in Visual Tracking Ming Yang, Ying Wu CVPR 2006 New York City

2. Motivation • General targets exhibit enormous variability and unpredictable changes. • rotation and scale changes • different degrees of deformations • partial occlusions • Most observation models tend to focus on certain characteristics of targets. • Adaptation of more aspects of target observation models is preferable. CVPR 2008 Anchorage, Alaska

3. Appearance-based visual tracking • Two key aspects in designing appearance based observation models: • the abstraction level of features, • how to take into account the geometrical structures of targets. CVPR 2008 Anchorage, Alaska

4. Granularity vs. Elasticity • Feature Granularity: the abstraction level of features. • e.g. features describe attributes of a pixel, a blob region or a whole object. • Model Elasticity: the ability that the model can tolerate geometrical changes among components. CVPR 2008 Anchorage, Alaska

5. Comparisons • Comparisons of different tracking approaches in terms of their relative granularity and elasticity. CVPR 2008 Anchorage, Alaska

6. The paradigm • We propose a general tracking paradigm. • The target is represented by a MRF of interest regions. • Adaptation of the feature granularity and model elasticity to maximize the likelihood of the MRF. CVPR 2008 Anchorage, Alaska

7. Target observation model • An MRF model of interest regions • X={xi}: the initial interest regions • Y={yi}: the detected interest regions in every frame • Substantialize to different models • Every pixel is an interest region => Template • The target is one interest region => Meanshift CVPR 2008 Anchorage, Alaska

8. Target model construction • Harris-Laplace interest region detection • Represented by the location, characteristic scale, and shape matrix • MRF model: pair-wise potential among overlapped interest regions. CVPR 2008 Anchorage, Alaska

9. Model the granularity and elasticity • The pair-site potential is defined based on the relative angles. • The parameter models the elasticity. • The likelihood of individual interest region is defined using the Bahattachaya coefficient of feature histograms • The scale ratio r regulates the image region to extract features so as to models the granularity. CVPR 2008 Anchorage, Alaska

10. Motion estimation • Coarse motion estimation • The motion parameters are estimated independently based on the detected pair-wise cliques. • Motion parameters refinement • Jointly sample the motion parameters and evaluate the posteriors of the hypotheses CVPR 2008 Anchorage, Alaska

11. Feature granularity adaptation • Update the scale ratio by searching rt until a local maximum of • Rigid and stable targets => large ratio r can yield good matching • Partial occlusion or deformation happens => small ratio r may be appropriate. CVPR 2008 Anchorage, Alaska

12. Model elasticity adaptation • Update the parameter in the pair-site potential function by maximizing the likelihood of the current tracking result: • The optimal is the variance of the observed angle differences. CVPR 2008 Anchorage, Alaska

13. Experiment settings • Up to 12 integration scales used in Harris-Laplace interest region detection. • Features for the interest regions are 2D histograms in Normalized-RG space with 24*24 bins. • Interest regions matching: • Runs at 2-10 fps on a Pentium 3GHz desktop. CVPR 2008 Anchorage, Alaska

14. Illustration CVPR 2008 Anchorage, Alaska

15. More tracking results CVPR 2008 Anchorage, Alaska

16. Conclusion • A novel perspective of adapting target observation models. • able to automatically tune the observation model’s focus on target’s appearances and structures. • flexible to incorporate different interest region detection and features extraction. CVPR 2008 Anchorage, Alaska