1 / 32

Modeling the Shape of People from 3D Range Scans

Modeling the Shape of People from 3D Range Scans. Dragomir Anguelov AI Lab Stanford University Joint work with Praveen Srinivasan, Hoi-Cheung Pang, Daphne Koller, Sebastian Thrun, James Davis. The Dataset. 70 scans. Cyberware Scans 4 views, ~125k polygons ~65k points each

artan
Download Presentation

Modeling the Shape of People from 3D Range Scans

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Modeling the Shape of People from 3D Range Scans Dragomir Anguelov AI Lab Stanford University Joint work with Praveen Srinivasan, Hoi-Cheung Pang, Daphne Koller, Sebastian Thrun, James Davis

  2. The Dataset 70 scans • Cyberware Scans • 4 views, • ~125k polygons • ~65k points each • Problems • Missing surface • Drastic pose changes 48 scans

  3. Modeling Human Shape Pose variation Body-shape variation

  4. Space of Human Shapes Movie [scape movie]

  5. Talk outline • Data processing pipeline • Registration • Recovering an articulated skeleton • Modeling the space of human deformations • Pose deformations • Body shape deformations • Shape completion • Partial view completion • Animating motion capture sequences

  6. Talk outline • Data processing pipeline • Registration • Recovering an articulated skeleton • Modeling the space of human deformations • Pose deformations • Body shape deformations • Shape completion • Partial view completion • Animating motion capture sequences

  7. Data Processing Pipeline

  8. Registration - CC Algorithm [Anguelov et al. 2004] Z X Correlated Correspondence Algorithm • Computes an embedding of mesh Z into mesh X • 1. Defines a discrete Markov Net Mover correspondence variables of mesh Z • Markov Net potentials enforce • Minimal surface deformation • Similar local surface appearance • Preservation of geodesic distance • 2. Embedding of Z into X is computed by performing Loopy Belief Propagation on M Input: Pair of scans Output: Correspondences X Z Related work: [Huttenlocher et al 00][Coughlan 02]

  9. Results: Human poses dataset Cyberware scans Model Registrations • 4 markers were used on each scan to avoid the need for multiple initializations of Loopy-BP

  10. Recovering articulation Input: models, correspondences Output: rigid parts, skeleton

  11. Recovering articulation [Anguelov et al’04] 4 3 2 1 • Stages of the process • Register meshes using Correlated Correspondences algorithm • Initialize • break template surface into N arbitrary components • Cluster surface into rigid parts • Estimate joints Related work: [Cheung et al’03]

  12. a1 aN Part labels … Transformations Model xN T x1 Points … Transformed Model y1 yN … b1 bK Point corrs Instance z1 zK Points … Probabilistic Generative Model

  13. Contiguity Prior • Parts are preferably contiguous regions • Adjacent points on the surface should have similar labels • Enforce this with a Markov network: a1 a2 a3

  14. Clustering algorithm • Objective • Algorithm • Given transformations , perform min-cut* inference to get • Given labels , solve for rigid transformations Data Likelihood Contiguity prior * [Greig et al. 89], [Kolmogorov & Zabih 02]

  15. Results: Puppet articulation

  16. Results: Arm articulation

  17. Results: 50 scans of a human Tree-shaped skeleton found Rigid parts found

  18. Talk outline • Data processing pipeline • Registration • Recovering an articulated skeleton • Modeling the space of human deformations • Pose deformations • Body shape deformations • Shape completion • Partial view completion • Animating motion capture sequences

  19. Modeling Pose Deformation input Joint angles Deformations output Regression function

  20. Modeling Pose Deformation Predict independently for each triangle Reconstruct complete shape template Related work: [Allen et al ‘03][Sumner+Popovic 2004]

  21. Rigid articulated deformation Pose deformation Given estimates of R, Q, synthesizing the shape is straightforward : Representation of pose deformation

  22. Learning pose deformation • For each polygon, predict entries of from rotations of nearest 2 joints (represented as twists ). • Linear regression parameters : • Obtaining values of in the first place:

  23. Twists and exponential maps Twist From twist to rotation matrix Joint angles

  24. Pose deformation space

  25. Learning body-shape deformation • Include also change in shape due to different people: • Do PCA over body-shape matrices : • Getting estimates of :

  26. PCA over body shape

  27. Combining pose and body shape spaces

  28. Talk outline • Data processing pipeline • Registration • Recovering an articulated skeleton • Modeling the space of human deformations • Pose deformations • Body shape deformations • Shape completion • Partial view completion • Animating motion capture sequences

  29. Shape completion • Find surface Y from our space which matches a set of markers Z • Y[Z] : completed mesh • deforms out of space spanned by , R to match Z • Y’[Z]: predicted mesh • constrained to be in space spanned by , R • Target optimized by iteratively solving for , R orY while holding the others fixed

  30. Partial view completion • Process: • Add a few markers (~6-8) • Run CC algorithm to get > 100 markers • Optimize to find Y[Z]

  31. Shape completion from motion capture data

  32. Conclusions • Presented a data-driven method of modeling human deformations induced by • Pose • Body shape • Extending the model • Nonlinear prediction of pose deformation (ongoing) • Shape complete original scans using current model • Acquire and learn from the entire pose-bodyshape matrix • Prior on likely joint angles, e.g. [Popovic + et al ’04] • Enforce temporal consistency in tracking applications • Extending the possible applications • Markerless motion capture (shape completion in shape-from-silhouette data) • Modeling other beasts

More Related