Visual Tracking Decomposition

1. Visual Tracking Decomposition Junseok Kwon* and Kyoung Mu lee Computer Vision Lab. Dept. of EECS Seoul National University, Korea Homepage: http://cv.snu.ac.kr

2. Goal of Visual Tracking • Robustly tracks target in real-world scenarios Frame #60 Frame #1

3. Real-World Scenarios Mixed Pose variations Occlusions Illumination changes Abrupt motions

4. Previous Works Our method MIL Tracker [1] OAL Tracker [2] [1] Babenko et. al.Visual tracking with onlinemultiple instance learning. CVPR 2009. In the real-world scenarios, conventional tracking methods frequently fail. [2] Ross et. al.Incremental learning forrobust visual tracking. IJCV 2007.

5. Bayesian Tracking Approach edge color Position, scale Maximum a Posteriori (MAP) estimate

6. Bayesian Tracking Approach • Update rule • Observation model • Motion model

7. Compound Model Pose variation Smooth Clutters Occlusion Abrupt Illumination change Compound Observation Model Compound Motion Model • Need for real-world scenarios • But difficult to design

8. Our Approach Basic Observation Model 1 Basic Observation Model 2 Basic Observation Model r + + + • Observation Model Decomposition Compound Observation Model

9. Our Approach Basic Motion Model s Basic Motion Model 1 Basic Motion Model 2 + + + • Motion Model Decomposition Compound Motion Model

10. Our Approach Basic Observation Model 2 Basic Observation Model 1 Basic Observation Model 1 Basic Observation Model 1 Basic Motion Model 1 Basic Motion Model 1 Basic Motion Model 2 Basic Motion Model 2 Basic Observation Model r Basic Observation Model r Basic Motion Model s Basic Motion Model s • Tracker Decomposition Basic Tracker 1 Basic Tracker 2 Basic Tracker rs

11. Our Approach • Tracker Decomposition • Each tracker takes charge of a certain change in the object. Basic Tracker 1 Basic Tracker 2 Basic Tracker rs

12. Our Approach Basic Motion Model j Basic Motion Model j Basic Observation Model i Basic Observation Model i • Sampling based Tracker • Markov Chain Monte Carlo (MCMC) Basic Tracker Sampling…

13. Remaining Tasks Basic Observation Model 1 Basic Motion Model 1 Basic Motion Model s Basic Observation Model 1 Basic Observation Model r Basic Motion Model 1 Basic Observation Model r Basic Motion Model s • How to determine the basic models ? • How to estimate weights of the models ?

14. Remaining Tasks • How to determine the basic models ? • Sparse PCA [1] • How to estimate weights of the models ? • Interactive MCMC [2] [1] A. d’Aspremont et. al., A directformulation for sparse PCA using semidefinite programming. SIAMReview, 49(3), 2007. [2] J. Corander et. al. Parallell interacting MCMC for learning of topologies of graphical models. Data Min. Knowl.Discov., 2005.

15. Design of Basic Observation Models Template set 1 initial frame 4 recent frames Hue Saturation Value Edge Object models A subset of the template set Basic observation models Diffusion distance

16. Object Model • Three conditions • Representativeness • The model has to cover most appearance changes in an object over time. • Compactness • The formation of it should be as compact as possible. • Complementary relation • The relations between models should be complementary.

17. Object Model : Gram matrix of the template set : Principal component • Sparse Principal Component Analysis (SPCA) PCA Sparseness

18. Object Model Template set Template set

19. Object Model Sparse PC 1 0 0 0 0 0 0 0 0 Object model 1 Representativeness Sparse PC 2 0 0 0 0 0 0 0 0 0 0 Object model 2 Compactness Sparse PC r 0 0 0 0 0 0 0 0 0 Object model r Complementary relation

20. Basic Observation Model Object model • Diffusion distance [3] Saturation Hue Edge Value Edge Diffusion distance [3] [3] H. Ling and K. Okada. Diffusion distance for histogram comparison.CVPR, 2006.

21. Design of Basic Motion Models • Two conditions • Exploitation ( for smooth motions ) • Further simulating the seemingly good moves near the local minima • Exploration ( for abrupt motions ) • Further simulating moves that have not been explored much Exploitation Exploration

22. Weights of Basic Models Parallel Mode Interaction Mode Basic Observation Model 1 Basic Observation Model r Basic Motion Model 1 Basic Motion Model s Basic Motion Model 2 Basic Observation Model 1 Basic Tracker 1 State Basic Tracker 2 State Basic Tracker rs

23. Experimental Results • The number of models • Basic observation models : #4 • Basic motion models : #2 • Basic tracker models: #8(=4X2) • Settings for comparison • Standard MCMC (MC) : 800 samples • Mean Shift (MS) • On-line Appearance Learning (OAL) : 800 samples • Multiple Instance Learning (MIL) OAL : Ross et. al.Incremental learning forrobust visual tracking. IJCV 2007. MIL : Babenko et. al.Visual tracking with onlinemultiple instance learning. CVPR 2009.

24. Experimental Results

25. Abrupt Motions and Illumination Changes

26. Illumination Changes and Pose Variations

27. Occlusions and Pose Variations

28. Background Clutters

29. Quantitative Results - Average center location errors in pixels MS : Comaniciu et. al.Real-time tracking of nonrigidobjects using mean shift. CVPR 2000. OAL : Ross et. al.Incremental learning forrobust visual tracking. IJCV 2007. MIL : Babenko et. al.Visual tracking with onlinemultiple instance learning. CVPR 2009.

30. Summary • Visual tracking decomposition (VTD) • Our method successfully tracks an object whose motion and appearance change at the same time • Since VTD is easy to extend by adding new features or trackers, our method can be more improved.

31. http://cv.snu.ac.kr/paradiso