1 / 34

Applications

Applications. Shmuel Peleg and Joshua Herman, “ Panoramic Mosaics by Manifold Projection ”, Computer Vision and Pattern Recognition (CVPR), 1997 Wolfgang Heidrich and Hans-Peter Seidel , “ View-independent environment maps ”, SIGGRAPH / Eurographics Workshop on Graphics Hardware, 1998

sheila-bentley
Download Presentation

Applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Applications Shmuel Peleg and Joshua Herman, “Panoramic Mosaics by Manifold Projection”, Computer Vision and Pattern Recognition (CVPR), 1997 Wolfgang Heidrich and Hans-Peter Seidel, “View-independent environment maps”, SIGGRAPH / Eurographics Workshop on Graphics Hardware, 1998 Matthew Brand, “Charting a Manifold”, Mitsubishi tech report, 2003 K.Grochow, S. Martin, A. Hertzmann, and Z. Popovic Style-based Inverse Kinematics, Siggraph 2004

  2. Applications using manifolds • Many problems can be phrased in manifold terminology • Provides an alternative way of viewing the problem • Can also provide some formalism Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  3. Applications • Surface fitting • Consistent parameterization • Image-based rendering • Environment mapping • Animation Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  4. Surface fitting • CT, MRI data, noisy, densely sampled in widely-spaced contours • Boundary conditions less problematic Fitting Manifold Surfaces To 3D Point Clouds Cindy Grimm, David Laidlaw, and Joseph Crisco J. of Biomedical Engineering Feb. 2002 Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  5. Consistent parameterization • Build manifold once • Fit to multiple bone point data sets Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  6. Aligning parameterization • Identify similar points on meshes • Pin parameterization Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  7. Application: Panoramas • Problem statement: • Given images from a known camera movement • Rotation about camera axis • “Push-broom” pan (assumes negligible depth) • “Glue” images together into a single image Rover, nasa.gov Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  8. Camera rotation • Final image can be rendered on a cylinder • No parallax • Each image samples some number of pixels on cylinder (manifold) image Peleg and Herman Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  9. Push broom/vertical slit camera Peleg and Herman • Translation of camera • Image slit perpendicular to camera motion • Need not travel in straight line • Depth differences negligible • Parallax • Manifold is part of ground plane viewed by camera Direction of travel Ground plane Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  10. Practical problem • How to line up individual images to create one seamless image? • Manifold: Final image (3D function RGB on 2D manifold) • Charts: Individual images (2D charts) • Overlap regions/transition functions: Unknown • Assume translation • (Account for optical effects of camera) • Note: Only works for these two camera motions Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  11. General solution • Define a format for the transition function • E.g., translation in x,y • Define an error metric that measures how well two overlap regions agree • E.g., pixel difference • Optimize over free parameters in transition function • E.g., x,y shift between all pairs of overlapping images Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  12. Solving for overlaps, transitions • Find translation that minimizes pixel differences • Find y that minimizes || I0(s) – I1(y(s))|| • y(s) = s + Ds, where Ds is unknown 0 1 0 1 Ds Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  13. Final image • Transition functions align images (abstract manifold) • Final image colors? (RGB function on manifold) • Blend and embedding functions for each chart • Embedding function: Original image • Blending function: How much to use of each overlapping image • Usually favor very short blend regions Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  14. Application: Environment mapping • Place scene/model inside sphere • Light intensity/color found by intersecting normal with sphere • 1-1 mapping between normal direction and sphere • Every point on sphere assigned light intensity/color • Implementation • Store colors in one (or more) texture maps (2D) Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  15. Parameterization • Surface normal (point on sphere) to point in texture map • Atlas/local parameterization • Desirable properties • Even sampling of sphere • Adaptive • Partition • Overlap (mip mapping, continuity) • Simple to compute • Amenable to GPU implementation Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  16. Approach 1 • Single texture map • Not unique (poles) • Poor sampling • Simple to compute Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  17. Approach II • Cube mapping • Six charts • Discontinuities at edges • Sampling better at center of faces than edges • Simple (plane) computation • Which plane? Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  18. Approach III • Parabolic mapping • Chart functions use parabolic function • Better sampling • Slightly more computation • Less-noticeable seams Heidrich and Hans-Peter Seidel Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  19. Proposal • Use chart approach • Allows for adaptive sampling (more detail where needed) • Chart sizes uniform: Tile texture map • Include overlap • Minimal extra texture map • Mip-mapping/down sampling Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  20. Application: Animation • Human configuration space lies on a manifold of dimension n embedded in m dimensional space, where n << m • Articulated skeleton: over 40 degrees of freedom (shoulders, knees, hips, etc., each 1-3 degrees of rotation) • Individual motions (reaching, walking) certainly lie on lower dimension manifolds • End-point of reach plus time • Shape of manifold of all possible human motions? • Who knows? K.Grochow, S. Martin, A. Hertzmann, and Z. Popovic Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  21. Overview • Manifold learning • Data samples (e.g., motion capture, key frames) • Interpolation equals manifold construction • Editing equals manifold editing • High-level review of manifold learning techniques • What kind of manifolds can you learn/construct from data? • 2D animation example • Manifolds in animation Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  22. 2D illustration • Two joint angles • Circle X Circle manifold (torus) • Animation • Repetitive motion • Joint angle plot • Circle manifold • Animation is a 1D manifold embedded in 2D Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  23. 2D illustration • Two joint angles • Circle X Circle manifold (torus) • Animation • Repetitive motion • Joint angle plot • Circle manifold • Animation is a 1D manifold embedded in 2D Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  24. Manifold learning • Input: Sample points in Rm • E.g., Motion capture sequence, each pose is a data point, m is number of dof of joints • 2D example: q,f for each pose • Assume data lies on a manifold of dimension n • Constraints on manifold shape/geometry (e.g., linear, no self-intersections) • Goal: Parameterize manifold • Single chart Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  25. Manifold learning, linear techniques • Principal components analysis (PCA), Independent components analysis (ICA) • Assumes shape of manifold in space is linear/planar • Learn vector(s) that span the hyper-plane Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  26. Manifold learning, deformed linear • Support vector machines (SVM) • Deform space points are in first • Deformation is “nice” • Warps points to new places • “Guess” which deformation will work • Learn linear/planar function Linear parameterization Deformation Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  27. Manifold learning, local shape • Isomap, Local linear embedding (LLE), Semi-definite embedding (SDE) • Define a distance metric between points • Assumptions: • K closest neighbors are also closest K neighbors on the manifold • Disk around point in manifold would contain all K neighbors, but not others • Know dimension n of manifold • Possible to deform K neighbors into n dimensions while maintaining (relative) distances • Output: Single chart/parameterization of entire data set Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  28. Isomap, LLE, SDE cont. • Non-obvious failure modes • Circular/repetitive data sets • Self-intersections Modified : 1D embedding Raw result: 2D embedding Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  29. Manifold construction as learning • Use K neighbors to define chart domains (Uc) • Charts are “squished” Gaussians • Center, tangent vectors • Find transition functions (affine transformations) • Transformation takes tangent vectors into Rn • Aligns free vectors with neighbors Uc Matthew Brand, “Charting a Manifold”, Mitsubishi tech report, 2003 Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  30. Uses of animation manifold • General idea: • Construct (implicitly or explicitly) a manifold representing valid human poses • Create a new animation sequence • Foot must touch here, reach here, etc. • Not sufficient to constrain all degrees of freedom (dof) • Project on to manifold to fill in remaining dof K. Grochow, S. L. Martin, A. Hertzmann, Z. Popovic, Style-Based Inverse Kinematics, Siggraph 2004 Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  31. Re-sequencing as embedded manifold • Goal: Given existing sequence (samples), add more/change samples • Assumptions: • Samples come from some smooth manifold • Some form of interpolation gives new samples on manifold • Current approaches: Interpolation between neighboring samples in sequence for given new time Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  32. Re-phrasing problem • Manifold learning or sequence timing provides parameterization/abstract manifold • Embed manifold with smooth function • Parameterization • Use function fitting • Re-sequencing: Evaluate embedding function E(M) Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  33. Caveats • What makes animation data difficult? • “Distance” loses meaning in >> 10 dimensions • Every point equally far away • Can’t enumerate • Noise • Error in capture process • Skeleton only approximates human motion • Joint angle representation • Don’t explicitly deal with manifold, parameterization Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

  34. Summary • Manifolds provide a formalism for breaking a problem into manageable pieces • Charts provide local parameterization • Planar domains • Overlaps: Natural mechanism for moving between parameterizations • Blend functions instead of geometric constraints • No boundary condition problems • Explicitly encapsulating/representing manifold is beneficial • Cleaner algorithm specifications Siggraph 2005, 8/1/2005 www.cs.wustl.edu/~cmg/Siggraph2005/siggraph.html

More Related