B ra MBL e: The B ayesian M ultiple- BL ob Tracker

1. BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003

2. Problem • The goal is to track an unknown number of blobs from static camera video.

3. Number, Positions, Shapes, Velocities, … Solution • The Bayesian Multiple-BLob (BraMBLe) tracker is a Bayesian solution. • It estimates State at frame t Image Sequence

4. Bayes Rule Prior Posterior State Distribution Observation Likelihood

5. Sequential Bayes Prior Posterior State Distribution Observation Likelihood Instead of modeling directly, BraMBLe models and .

6. Update Algorithm

7. Outline • Observation likelihood model . • Prediction model . • Estimation of posterior . • Results. • Discussion.

8. Outline • Observation likelihood model . • Prediction model . • Estimation of posterior . • Results. • Discussion.

9. Image Observations • We want to choose our observations so that we can compute quickly: Individual observations are conditionally independent.

10. Y Gaussian Cr Cb Mexican Hat Filter plots from http://www.cs.jhu.edu/~wolff/course600.461/week3.2/sld012.htm Image Observations • We want . • A bank of filters is applied at each grid point.

11. Image Observations • We want to choose our model so that we can compute quickly: Observation depends on membership of grid point.

12. Image Observations • We want to choose our model so that we can compute quickly: • We can precompute and quickly evaluate any state x.

13. Appearance Models • The appearance models are learned from training data. Training Data

14. Observation Likelihood

15. Observation Likelihood Review • We defined our image observations so that • We defined our observation model so that • We can precompute and quickly evaluate for many choices of x.

16. Outline • Observation likelihood model . • Prediction model . • Estimation of posterior . • Results. • Discussion.

17. Object Model • The blob configuration is Number of objects Object State

18. Object Model • The blob configuration is • The object state is Identity Velocity Location Shape

19. Calibrated Camera Person Model Generalized-Cylinder Model

20. Prediction Model • The number of objects can change: • Each object has a constant probability of remaining in the scene. • There is a constant probability that a new object will enter the scene. In this formulation, hypotheses with different numbers of objects can be compared directly.

21. Prediction Model • Damped constant location velocity:

22. Prediction Model • Damped constant location velocity: • Auto-regressive shape:

23. Model Review • The observation likelihood is fast to compute for different hypotheses . • The prediction model allows generation of from • Estimating requires an efficient way of • Representing . • Computing the multiplications and integrations.

24. Outline • Observation likelihood model . • Prediction model . • Estimation of posterior . • Results. • Discussion.

25. Efficient Representation • is represented by a set of particles, : • Sampling from the set using the weights approximates sampling from N Points: N Weights:

26. Efficient Representation • is represented by a set of particles, : • Sampling from the set using the weights approximates sampling from

27. Efficient Computation • The particle set representing is computed from by CONDENSATION: • Apply dynamics to the particle set: • Multiply by the observation likelihood:

28. Applying Dynamics • Given particle set , compute Image from http://www.hpl.hp.com/personal/John_MacCormick/WOMOT03/cal.giftalk/page.018.gif

29. Applying Dynamics • Given particle set , compute • Resample into Image from http://www.hpl.hp.com/personal/John_MacCormick/WOMOT03/cal.giftalk/page.020.gif

30. Applying Dynamics • Given particle set , compute • Resample into • Predict, generating to give

31. Multiplication by Likelihood • Given particle set , compute • Weight particles, setting Image from http://www.hpl.hp.com/personal/John_MacCormick/WOMOT03/cal.giftalk/page.021.gif

32. Efficient Computation Review • The particle set representing is computed from by CONDENSATION: Reweight Resample Predict Image from http://www.hpl.hp.com/personal/John_MacCormick/WOMOT03/cal.giftalk/page.022.gif

33. Outline • Observation likelihood model . • Prediction model . • Estimation of posterior . • Results. • Discussion.

34. People Tracking

35. People Tracking • Tracking was successful in real time on this 53s clip except when two people crossed in front of a third.

36. Outline • Observation likelihood model . • Prediction model . • Estimation of posterior . • Results. • Discussion.

37. Algorithm Summary • The models chosen • Are a smooth integration of foreground and background models. • Allow hypotheses with differing numbers of objects to be compared directly. • Can be quickly evaluated in a particle filtering implementation.

38. Relationship to Previous Work • Static camera blob tracking: • Classifies pixels as foreground or background. Application of Stauffer and Grimson’s Adaptive Background Subtraction to video with compression artifacts Video from http://image.pirl.umd.edu/knkim/research/BGS/compressed_video.htm

39. Relationship to Previous Work • Static camera blob tracking: • Classifies pixels as foreground or background. • Static camera blob tracking: • Finds the position in the search area • Made up of foreground pixels. • Matching the blob in the previous frame. Predicted position in frame t + 1 Search area Frame t - 1 Frame t Frame t + 1

40. Relationship to Previous Work • Improvements over blob tracking: • Integrates the foreground and background modeling. • Multiple objects can be tracked through occlusions. Video from http://www.robots.ox.ac.uk/~jmac/research/thesis/thesis.html Video from http://robotics.stanford.edu/~birch/headtracker/

41. Weaknesses • The algorithm is sensitive to reflections. • The algorithm sometimes switches the labels when one object passes in front of another. • There are a lot of parameters to assign.