330 likes | 500 Views
Modeling the Shape of People from 3D Range Scans. Dragomir Anguelov AI Lab Stanford University Joint work with Praveen Srinivasan, Hoi-Cheung Pang, Daphne Koller, Sebastian Thrun, James Davis. The Dataset. 70 scans. Cyberware Scans 4 views, ~125k polygons ~65k points each
E N D
Modeling the Shape of People from 3D Range Scans Dragomir Anguelov AI Lab Stanford University Joint work with Praveen Srinivasan, Hoi-Cheung Pang, Daphne Koller, Sebastian Thrun, James Davis
The Dataset 70 scans • Cyberware Scans • 4 views, • ~125k polygons • ~65k points each • Problems • Missing surface • Drastic pose changes 48 scans
Modeling Human Shape Pose variation Body-shape variation
Space of Human Shapes Movie [scape movie]
Talk outline • Data processing pipeline • Registration • Recovering an articulated skeleton • Modeling the space of human deformations • Pose deformations • Body shape deformations • Shape completion • Partial view completion • Animating motion capture sequences
Talk outline • Data processing pipeline • Registration • Recovering an articulated skeleton • Modeling the space of human deformations • Pose deformations • Body shape deformations • Shape completion • Partial view completion • Animating motion capture sequences
Registration - CC Algorithm [Anguelov et al. 2004] Z X Correlated Correspondence Algorithm • Computes an embedding of mesh Z into mesh X • 1. Defines a discrete Markov Net Mover correspondence variables of mesh Z • Markov Net potentials enforce • Minimal surface deformation • Similar local surface appearance • Preservation of geodesic distance • 2. Embedding of Z into X is computed by performing Loopy Belief Propagation on M Input: Pair of scans Output: Correspondences X Z Related work: [Huttenlocher et al 00][Coughlan 02]
Results: Human poses dataset Cyberware scans Model Registrations • 4 markers were used on each scan to avoid the need for multiple initializations of Loopy-BP
Recovering articulation Input: models, correspondences Output: rigid parts, skeleton
Recovering articulation [Anguelov et al’04] 4 3 2 1 • Stages of the process • Register meshes using Correlated Correspondences algorithm • Initialize • break template surface into N arbitrary components • Cluster surface into rigid parts • Estimate joints Related work: [Cheung et al’03]
a1 aN Part labels … Transformations Model xN T x1 Points … Transformed Model y1 yN … b1 bK Point corrs Instance z1 zK Points … Probabilistic Generative Model
Contiguity Prior • Parts are preferably contiguous regions • Adjacent points on the surface should have similar labels • Enforce this with a Markov network: a1 a2 a3
Clustering algorithm • Objective • Algorithm • Given transformations , perform min-cut* inference to get • Given labels , solve for rigid transformations Data Likelihood Contiguity prior * [Greig et al. 89], [Kolmogorov & Zabih 02]
Results: 50 scans of a human Tree-shaped skeleton found Rigid parts found
Talk outline • Data processing pipeline • Registration • Recovering an articulated skeleton • Modeling the space of human deformations • Pose deformations • Body shape deformations • Shape completion • Partial view completion • Animating motion capture sequences
Modeling Pose Deformation input Joint angles Deformations output Regression function
Modeling Pose Deformation Predict independently for each triangle Reconstruct complete shape template Related work: [Allen et al ‘03][Sumner+Popovic 2004]
Rigid articulated deformation Pose deformation Given estimates of R, Q, synthesizing the shape is straightforward : Representation of pose deformation
Learning pose deformation • For each polygon, predict entries of from rotations of nearest 2 joints (represented as twists ). • Linear regression parameters : • Obtaining values of in the first place:
Twists and exponential maps Twist From twist to rotation matrix Joint angles
Learning body-shape deformation • Include also change in shape due to different people: • Do PCA over body-shape matrices : • Getting estimates of :
Talk outline • Data processing pipeline • Registration • Recovering an articulated skeleton • Modeling the space of human deformations • Pose deformations • Body shape deformations • Shape completion • Partial view completion • Animating motion capture sequences
Shape completion • Find surface Y from our space which matches a set of markers Z • Y[Z] : completed mesh • deforms out of space spanned by , R to match Z • Y’[Z]: predicted mesh • constrained to be in space spanned by , R • Target optimized by iteratively solving for , R orY while holding the others fixed
Partial view completion • Process: • Add a few markers (~6-8) • Run CC algorithm to get > 100 markers • Optimize to find Y[Z]
Conclusions • Presented a data-driven method of modeling human deformations induced by • Pose • Body shape • Extending the model • Nonlinear prediction of pose deformation (ongoing) • Shape complete original scans using current model • Acquire and learn from the entire pose-bodyshape matrix • Prior on likely joint angles, e.g. [Popovic + et al ’04] • Enforce temporal consistency in tracking applications • Extending the possible applications • Markerless motion capture (shape completion in shape-from-silhouette data) • Modeling other beasts