400 likes | 561 Views
Modeling the Shape of People from 3D Range Scans. Dragomir Anguelov AI Lab Stanford University Joint work with Praveen Srinivasan, Hoi-Cheung Pang, Daphne Koller, Sebastian Thrun . The Dataset. Scans 4 views, ~125k polygons ~65k points each Problems Missing surface
E N D
Modeling the Shape of People from 3D Range Scans Dragomir Anguelov AI Lab Stanford University Joint work with Praveen Srinivasan, Hoi-Cheung Pang, Daphne Koller, Sebastian Thrun
The Dataset • Scans • 4 views, • ~125k polygons • ~65k points each • Problems • Missing surface • Drastic pose changes
Space of Human Shapes movie [scape movie]
Talk outline • Scan registration • Rigid • Nonrigid • Correlated correspondence • Decomposition into approximately rigid parts • Learning a deformable human model
Talk outline • 3D Scan registration • Rigid • Nonrigid • Correlated correspondence • Decomposition into approximately rigid parts • Learning a deformable human model
Rigid Scan Registration • Aligning point clouds X and Z • Match each point in Z to its closest point in X • Compute the best rigid transformation T which aligns the matching pairs • Transform the points of cloud X by T • Repeat • Iterative Closest Point (ICP) algorithm [Besl et al ’92]
ICP matching step • Objective • Setting obtains • Substituting the above for s yields a system linear in R • Choose a parameterization of R (quaternions q) and solve • Exact solution • Overall, ICP converges to a local minimum of the energy (using )
Deformable templatemodel X Set of deformable springs Nonrigid registration Deforms X to match a scan Z Minimizes the deformation of X Nonrigid Registration Example
Data Generation / Correspondences Deformation / Transformation C Q Model mesh X Transformed mesh X’ Data mesh Z Generative Model Goal: Given meshes X and Z, recover transformation Q and correspondences C
Data mesh generation Transformed Model Data Correspondence ck specifies which point in X’ generates point zk: Z X’
X Z Nonrigid ICP failure example • Nonrigid ICP attempts to maximize • Local Minima • Poor transformation and correspondence estimates reinforce each other
Nonrigid-ICP decorrelates the correspondences Correspondences are conditionally independent given Q If Q is unknown, correspondences are correlated E.g. nearby points in Z should be mapped to nearby points in X The search space is exponential for the general registration problem c1 c2 Correlated correspondences
Correlated Correspondence • Compute an embedding of mesh Z into model X • The domain of ciis the set of possible (discrete) matches for zion the surface of X • The solution is a consistent assignment to all correspondence variables C Z X
(c1) (c2) (c1, c2) c1 c2 (c1,c3) (c2,c3) c3 (c3) Correlated Correspondence Z • Edge is generated from some edge in X • Prefer values of c1, c2 according to the deformation of X they induce • Doing this for all edges in Z results in a Markov network over C X
{1, …, K} Pairwise Markov Networks ij • Inference is Markov Nets is generally intractable • Good approximate algorithms exist (belief propagation) i yi yj
(c1) (c2) (c1, c2) c1 c2 (c1,c3) (c2,c3) c3 (c3) CC Potentials • Deformation potentials • Spin image* potentials • Quantify the similarity of the local surface around two matching points • Geodesic Potentials *[Johnson+Hebert ’97]
Nearby points in Z must be nearby in X Constraint between each pair of adjacent points zk, zl Distant points in Z must be distant in X Constraint between each pair of distant points zk, zl (farther than 5r) Geodesic potentials Z X X Z r resolution of mesh X
Local surface signatures • Use spin-images[Johnson ’97] • 2D Histogram of distances from an oriented reference point • Rotationally-invariant / Robust under clutter and occlusion / Compressible (PCA) • Potentialf(ck = i) encodes how well the signature of point zk matches the signature of point xi in the model:
Results: Human poses dataset Cyberware scans Model Registrations • 4 markers were used on each scan to avoid the need for multiple initializations of Loopy-BP • ArtModel I found using 20 registered scans • ArtModel II found using 44 registered scans ArtModel II ArtModel I
CC movie [movie]
Talk Outline • Scan registration • Rigid • Nonrigid • Correlated correspondence • Recovering the articulated skeleton • Learning a deformable human model
Recovering the skeleton Input: models, correspondences Output: rigid parts, skeleton
… Part labels a1 aN Transformations Model … xN T Points x1 Transformed Model … y1 yN b1 bK Point labels Instance … z1 zK Points bk = ModelPoint (zk) Probabilistic Generative Model aj = Part (xj)
a1 a3 If i, j are connected in model mesh a2 1 > > 0.5 Part Properties • Parts are preferably contiguous regions • Adjacent points on the surface should have similar labels • Enforce this with a Markov network:
Iterative Optimization • Hard EM • Given transformations , perform min-cut inference to get • Given labels , solve for rigid transformations
Parts movie [parts.gif]
Talk Outline • Scan registration • Rigid • Nonrigid • Correlated correspondence • Recovering the articulated skeleton • Learning a deformable human model
Modeling Human Shape Pose variation Body-shape variation
Representation of pose deformation Rigid articulated deformation Pose deformation Given estimates of R, Q, synthesizing the shape is straightforward :
Learning non-rigid pose deformation • For each polygon, predict entries of from rotations of nearest 2 joints (represented as twists ). • Linear regression parameters :
Learning body-shape deformation • Include also change in shape due to different people: • Do PCA over body-shape matrices :
Shape completion • Process: • Add a few markers (~6-8) • Run CC algorithm to get >100 markers • Optimize for pose/body-shape parameters and non-rigid surface
Recap • Algorithms for registration between two surfaces • ICP algorithm iterates between finding corresponding points between two scans and computing the transformation between them • Rigid variant (6 unknown DOF) • Nonrigid variant (6 Number of polygons DOF) • CC explicitly encodes correlations between correspondence variables • Essential when we have poor initial transformation estimate • Exponential complexity, approximate algorithm • Modeling humans • Decouple effects of pose and different physique on shape deformation
Geodesic Potentials: closeclose • Nearby points in Zmust be nearby in X • Constraint between each pair of adjacent points zk, zl Z X
X Z Geodesic Potentials: farfar • Distant points in Zmust be distant in X • Constraint between each pair of distant points zk, zl