1 / 17

Kinematic Jump Processes for Monocular 3D Human Tracking

Kinematic Jump Processes for Monocular 3D Human Tracking. Cristian Sminchisescu (University of Toronto) Bill Triggs (INRIA Rhone-Alpes). Goal: track human body motion in monocular video and estimate 3D joint motion. Why Monocular ? Movies, archival footage

ina-burks
Download Presentation

Kinematic Jump Processes for Monocular 3D Human Tracking

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Kinematic Jump Processes for Monocular 3D Human Tracking Cristian Sminchisescu (University of Toronto) Bill Triggs (INRIA Rhone-Alpes)

  2. Goal: track human body motion in monocular video and estimate 3D joint motion Why Monocular ? • Movies, archival footage • Resynthesis, e.g. change point of view or actor • Tracking / interpretation of actions & gestures (HCI) • How do humans do this so well?

  3. Overall Modeling Approach • Generative Human Model • Kinematics, geometry, photometry • Predicts images or descriptors • Priors and anatomical constraints • Model-image matching cost function • Robust, probabilistically motivated • Contour and intensity based • Tracking by search / optimization • Discovers well supported configurations of matching cost

  4. Why is 3D-from-monocular hard? Depth ambiguities Image matching ambiguities Violations of physical constraints

  5. How many local minima are there? Thousands ! – even without image matching ambiguities …

  6. Examples of Kinematic Ambiguities • Minima are separated by large distances in parameter space

  7. Monocular 3D Tracking Methods • CONDENSATION (discrete, motion models) • Deutscher et al.’00: annealing, walking • Sidenbladh et al.’00,02: importance sampling (walking + snippets) • CSS, ET/HS/Hyperdynamics (continuous, cost-sensitive) • Sminchisescu&Triggs’01,02 Covariance Scaled Sampling (CSS) Hypersurface Sweeping (HS) Hyperdynamics

  8. Search Globality and Adaption • Cost sensitive continuous search methods are • Efficient - avoid large wastage factors with random sampling • Generic - no assumptions on known motions • Focus on locating transition states and nearby minima • But • Still local (i.e. sometimes myopic) • Minima are typically far in parameter space • No knowledge of global long-range minimum structure • Want to search quasi-globally, yet preserve generality • Can we find other minima more efficiently by exploiting intrinsic problem structure?

  9. Kinematic Jump Sampling • For any given model configuration, we can explicitly build the interpretation tree of alternative kinematic solutions with identical joint projections • work outwards from root of kinematic tree, recursively evaluating forward/backward ‘flips’ for each body part • Alternatively, sample by generating flips randomly • … or, for tracking, sample shallowly and treat each limb quasi-independently

  10. Efficient Inverse Kinematics • The inverse kinematics is simple, efficient to solve • Constrained by many observations (3D articulation centers) • The quasi-spherical articulation of the body • Mostly in closed form • The iterative solution is also very competitive • Optimize over model-hypothesized 3D joint assignments • 1 local optimization work per new minimum found • An adaptive diffusion method (CSS) is necessary for correspondence ambiguities

  11. Candidate Sampling Chains The KJS Algorithm C=SelectSamplingChain(mi) C C1 CM s=CovarianceScaledSampling(mi) S=BuildInterpretationTree (s,C) E=InverseKinematics(S) Prune and locally optimize E

  12. Tracking Experiments • 4s agile dancing sequence, 25 frames per second • Cluttered background, self-occlusion, motion in depth • Automatically select kinematic jump samples (KJS) from short 3-link chains (rooted at hips, shoulders, neck) • 8 modes, CSS diffusion with scaling 4

  13. Jump Sampling in Action

  14. Quantitative Search Statistics • Initialize in one minimum, different sampling regimes • Improved minima localization by KJS • Local optimization often not necessary

  15. Summary • Kinematic Jump Sampling Algorithm • Construct interpretation trees of 3D joint positions corresponding to monocular kinematic ambiguities • Solve efficiently using closed-form inverse kinematics • Highly accurate hypothesis generator for long-range search • Local optimization polishing often un-necessary • Explicit kinematic jumps + cost-sensitive sampling • Address both depth and image matching ambiguities • Future work • Scene constraints (ground plane, equilibrium) • Jump strategies for image matching • Prior knowledge (Sminchisescu&Jepson03 upcoming)

  16. The End

  17. The End

More Related