TRACKING Particle Filter

1. TRACKING Particle Filter Computer Vision Seminar – Hagit Hel-Or ItzikBerrebi RotemKlorin

2. Tracking • Tracking - the process of locating a moving object (or multiple objects) over time using a camera. • Variety of uses: human-computer interaction, security and surveillance, video communication, traffic control…

3. Tracking - Goal • Finding the objects. • Giving them an identity. • Follow the object as they move around while maintaining their identity – even if the objects collide, disappear for a short time, etc.

4. Particle Filter

5. Main differences with Kalman filter: Kalmanfilter: • Assumes uni-modal (Gaussian) distribution. • Predicts single new state for each object tracked. • Updates state based on error between predicted state and observed data.

6. Main differences with Kalman filter: Particle filter: • Can work for multi-modal distribution. • Predicts multiple possible states for each object tracked. • Each possible state has a different probability. • Estimates probabilities of predicted states based on observed data.

7. Particle Vs Kalman

8. Particle Filter • Particle filter or Sequential Monte Carlo (SMC) methods are a set of algorithms that estimate the posterior density of the state variables given the observation variables.

9. Particle Filter Predicts multiple possible next states: • Achieved using sampling or drawing samples from the probability density functions.

10. Recursive Filter • The particle filter is a Bayesian sequential importance sampling technique, which recursively approximates the posterior distribution using a finite set of weighted samples. • Each state is estimated by the information from the last state.

11. Players • Observations and hidden states. • Particles and weights. • Probability

12. Observations and Hidden States • The particle filter is designed for a Hidden Markov Model, where the system consists of hidden and observable variables.

13. Observations and Hidden StatesExample - hand tracking Observations: • Images (R,G,B matrices)

14. Observations: • Binary contour images (0/1 matrices)

15. Observations: • Edge detection images

16. Observations: • Harris corners (lists of all detected Harris corners) • Other visual features collected from an image

17. Hidden States: • Hand contour images

18. Hidden States: • List of splines

19. Hidden States: • List of phalanx • List of angles between phalanxes

20. Hidden States: • One of the following: { paper, rock, scissors, lizard, spock}

21. Assumptions • - Hidden states • - Observations • Each depends only on

22. Assumptions • - Hidden states • - Observations • Each depends only on • is a Markov Chain:

23. Example Robot plays rock-paper-scissors DEMO

24. Players • Observations and hidden states. • Particles and weights. • Probability

25. Particles and Weights • Particles - Samples of hypotheses. • The hypotheses are estimated positions of the object we are tracking. • The more particles we sample, the faster we find the object.

26. Particles and Weights • Each particle is given a weight according to its probability of being the next state. • Particles with a bigger chance to be the next state will get higher weights.

27. Particle and Weights • High probability samples should be drawn more frequently. • Low probability samples should be drawn less frequently.

28. Particle Filter - Example DEMO

29. Players • Observations and hidden states. • Particles and weights. • Probability

30. HMM – Hidden Markov Models • P(X=H | O=T) = ? • P(X)= • Observation Model: P(O|X)= • Transition Model:

31. HMM – Hidden Markov Models

32. HMM – Hidden Markov Models • Bayes’ update: • Prediction:

33. Particle Filter • Posterior: What is the probability that the object is at the location for all possible locations if the history of observations is ? • Prior: The motion model – where will the object be at time instant t given that it was previously at ? • Likelihood: The likelihood of making the observation given that the object is at the location

34. Goal – Posterior Density • We would like to calculate: • Given the data:

35. Particle Filter Algorithm

36. Particle Filter - Algorithm • Sample the particles using a model. • Compute weights according to the model. • Resampling with respect to the weights: replace unlikely samples by more likely ones.

38. Particle Filter - Example DEMO

39. Aircraft example -Assumptions: • can be estimated from • can be calculated from • Velocity = 800 to 1000 km/h

40. In each iterator: • Sampling - picking big set of possible plane locations. • Prediction - assigning weights using a map: | • Update - updating locations and weights. |

41. A weighted set of N particles: P={(, )|1≤i≤N} In each iteration: • create new set P with respect to the weights in old P. • set the weight of each particle in new P. • update the locations and weights of particles in P using the model.

42. Particle Filter • Particle filterconsists of essentially two steps: Prediction and update.

43. Prediction • Given all available observations: up to time t-1, the prediction stage uses the probabilistic system transition model: to predict the posterior at time t as:

44. Update At time t, the observation is available, the state can be updated using Bayes’ rule: Where is described by the observation equation.

45. Particles and weights The posterior: is approximated by a finite set of N samples with importance weights .

46. Particles and weights The candidate samples are drawn from an importance distribution = And the weight of the samples are: =

47. Numerical Example Tracking a robot: • We attempt to track the position of a robot across two states using four particles. • States: A,B (=A means that particle i is at position A at time t)

48. Numerical Example Step 1 – model: • , The robot starts out as equally likely to be in either state.

49. Numerical Example • The motion model: , Any object has probability of moving to or staying in B of and any object of moving to or staying in A of .

50. Numerical Example • The observation model: If an objects is in state s in {A,B}, the probability that our observation is accurate is 80%.