Real Time Motion Capture Using a Single Time-Of-Flight Camera

1. Real Time Motion CaptureUsing a Single Time-Of-Flight Camera VarunGanapathi, Christian Plagemann, Daphne Koller, Sebastian Thrun CVPR 2010 Q36981123 邱碁森

2. Outline • Introduction • Probabilistic Model • Inference • Experiments • Conclusions

3. Introduction • Motion capture is used to human-machine interaction, smart surveillance and so on. • Time-of-flight sensors offers rich sensory information, not sensitive to changes in lighting, shadows, and some other problems. • This paper propose an efficient filtering algorithm for tracking human pose for fast operation at video frame.

4. What is Probabilistic Model? • A tree-shaped kinematic chain (skeleton) • Human body is modeled as 15 body parts • The transformations of the body Xt at time tis a set: Xt= {Xi}, i = 1~15 • X1: the root of tree → the pelvis part • root(pelvis): could freely rotate and translate • other parts: connected to the their parent, allow to rotate (not to translate)

5. What is Probabilistic Model? (cont.) • The absolute orientation of a body part i: Wi(X) • multiplying the transformations of its ancestors in the kinematic chain • Wi(X) = X1∙X2· ...· Xparent(i) ∙ Xi

6. Why need the Probabilistic Model? • Determine the most likely stateat at time t • the pose set Xt • the first discrete-time derivative set Vt(velocities) • zt: the recorded range measurements • The system is modeled as a dynamic Bayesian network (DBN)

7. Probabilistic Model • The measured range scan is denoted by z = {zk} k=1M • where zkgivesthe measured depth of the pixel at coordinate k.

8. Probabilistic Model • Assumption: the accelerations in our system are drawn from a Gaussian distribution with zero mean

9. Inference • How to perform efficient inference at each frame? • Model Based Hill Climbing Search (HC) • A component locally optimizes the likelihood function • Evidence Propagation (EP) • An inference procedure generate likely states which are used to initialize the HC

10. Model Based Hill Climbing Search • coarse-to-fine • The procedure can then potentially be applied to a smaller interval about the value chosen at the coarser level • hill-climbing • Start from the base of kinematic chain which includes the largest body parts, and proceed toward the limbs 3 2 sample: 0.5 0.45 0.4 ... -0.35 -0.4 -0.45 -0.5 1 then chose the best one optimize the X axis

11. Evidence Propagation • Problem: • fast motion cause motion blur • occlusion cause the estimate of the state of hidden parts to drift • the likelihood function has ridges (difficult to navigate) • This procedure that identifies promising locations for body parts to find likely poses

12. Evidence Propagation • Steps in this procedure: • Body Part Detection: identify possible body part locations from the current range image • Probabilistic Inverse Kinematics: update the body configuration X given possible correspondences between mesh vertices and part detections • Data Association and Inference: determine the best subset of such correspondences

13. Body Part Detection • Five body parts: head, left hand, right hand, left foot and right foot are found from the current range image. • Interest Point(AGEX) Detection • start on the geodesic centroid of the mesh: AGEX1(M) • recursively find thevertex AGEXk(M) which hasmax geodesic distance to AGEXk-1(M) • Identification of Parts • points are classified as body part by training these data using a marker-based motion capture system( LED mark)

14. Evidence Propagation

15. Experiments • Using a Swissranger SR4000 Time-of-Flight camera • Tracking results on real-world test sequences, sorted from most complex (left) to least complex (right).

16. Experiments • ATennis sequence Only use Model-Based search Our combined tracker

17. Conclusions • A novel algorithm for combining part detections with local hill-climbing for marker less tracking of human pose. • With the hybrid, GPU-accelerated filtering approach