Multi-scale Visual Tracking by Sequential Belief Propagation

1. Multi-scale Visual Tracking by Sequential Belief Propagation Gang Hua, Ying Wu Dept. Electrical & Computer Engr. Northwestern University Evanston, IL 60208 {yingwu,ganghua}@ece.northwestern.edu CVPR'2004

2. Abrupt Motion • sudden changes of target dynamics • frame dropping • large camera motion • etc. CVPR'2004

3. Challenges • Most existing visual tracking methods assume either small motion or accurate motion models • Abrupt motion violates them • Hierarchical search is not enough • Unidirectional information flow • Error accumulation from coarse to fine • No mechanism to recover failure in coarse scales CVPR'2004

4. Our Idea • Different scales provide different salient visual features • Bi-directional information flow among different scales should help • Different scales “collaborate” CVPR'2004

5. Our Formulation • A Markov network • X={Xi ,i=1..L}—target state in different scales • Z={Zi ,i=1..L}—Image observation of the target in different scales • Undirected link— Potential function Ψij(fi(Xi),fj(Xj)), • Directed link—Observation function Pi(Zi|Xi) • The task is to infer Pi (Xi|Z), i=1..L Fig.1. Markov Network (MN) CVPR'2004

6. Belief propagation (BP) • The joint posterior • Belief propagation [Pearl’88, Freeman’99] CVPR'2004

7. Dynamic Markov Network • Xt={Xt,i ,i=1..L}—Target states at time t • Zt={Zt,i ,i=1..L}—Image observations at time t • P(Xt,i|Xt-1,i)—Dynamic model in the ith scale • Zt={Zk, k=1..t}—Image observation up to time t Fig.2. Dynamic Markov Network (DMN) modeling target dynamics CVPR'2004

8. Bayesian inference in DMN • Markovian assumption • The Bayesian inference is • Independent dynamics model CVPR'2004

9. Sequential BP • Message Passing in DMN • Belief update in DMN CVPR'2004

10. Sequential BP Monte Carlo • To handle non-Gaussian densities • Monte Carlo implementation • A set of collaborative particle filters CVPR'2004

11. Algorithm CVPR'2004

12. Experiments: bouncing ball • Sudden dynamics changes fail the single particle filters The tracking result of the Sequential BP CVPR'2004

13. Experiments: dropping frames • Dropping 9/10 of the video frames BP iteration at a specific time instant CVPR'2004

14. Experiments: shaking camera CVPR'2004

15. Experiments: scale changes CVPR'2004

16. Conclusion& future work • Contributions • A new multi-scale tracking approach • A rigorous statistical formulation • A sequential BP algorithm with Monte Carlo • Future work • Theoretic study& comparison of the BP with the mean field variational approach • Learning model parameters CVPR'2004