1 / 25

IBR: View Interpolation Philipp Slusallek

IBR: View Interpolation Philipp Slusallek. Image-Based Rendering. Lightfield / Lumigraph. Sprite / Imposter. LDI. Geometry. Imposter / Sprite with Depth. View-dependent Texture Mapping. Panorama / Environment Map. Geometry-based. Image-based. View Interpolation.

don
Download Presentation

IBR: View Interpolation Philipp Slusallek

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. IBR: View Interpolation • Philipp Slusallek

  2. Image-Based Rendering Lightfield / Lumigraph Sprite / Imposter LDI Geometry Imposter / Sprite with Depth View-dependent Texture Mapping Panorama / Environment Map Geometry-based Image-based

  3. View Interpolation • 1. Interpolating between range images • - Chen and Williams, View Interpolation, 1993 • - Shade et al., Layered Depth Images, 1998 • 2. Correspondences and epipolar analysis • - Laveau and Faugeras, Collection of Images, 1994 • - McMillan and Bishop, Plenoptic Modeling, 1995 • 3. Combination of view synthesis and morphing • - Chen and Williams, 1993 • - Seitz and Dyer, 1995 •  Require depths or correspondences

  4. Drawing RGBZ Images • Various techniques: • 1. Priority sorted splatting (Gauss, small rectangles, or points) • - Chen and Williams • 2. Priority sorted height field • - McMillan and Bishop • 3. Micropolygon mesh • - Marks et al. • 4. Inverse ray-tracing of height field • - McMillan • 5. Decimate and draw as a polygon mesh

  5. View Interpolation • Issues: • Visibility/Occlusion • Holes • (Anti-)aliasing • Shading (diffuse)

  6. Directly from correspondences • Laveau and Faugeras Algorithm p p2 p1 Existing View2 Existing View1 p3 l13 l23 New View 3

  7. Merging RGBZ Images • Problems: • Overlaps • - Oversampling • - Occlusion • Gaps or holes • - Undersampling • - Expose unseen areas Chen and Williams 1993

  8. View Interpolation [Chen/Williams’93] • Preprocessing: • Correct correspondences for pixel (morph map) • Division into blocks (hierarchical) • Obtain priorities form Z filtering • Display: • Linear interpolation of morph map • Back-to-front rendering of blocks • Fill holes from adjacent pixels

  9. Sprites / Imposters • Purpose: • Cache complex rendered geometry • Decouple rendering updates from image updates • Drawing primitives • Issues: • Placement of polygon • Warping of texture (perspective, affine) • Validity of cache • Hierarchy • Continuity with environment

  10. Sprites / Imposters

  11. Imposters

  12. Sprites/Imposters with Depth • Better image warping: • Wider range of reuse • Backward mapping only with homograph • New mapping: • Stored depth map • Forward map depth map(approximate geometry) • Backward mapping of colorusing depth information d d’ d’

  13. I1 d1(I2 ) I2 Mapping with Depth • Forward Mapping: • Holes and aliasing

  14. I1(I2)d2 I2 Mapping with Depth • Backward Mapping: • What is d?

  15. I1(I2)d2 I2 Mapping with Depth • Solution: • Forward map depth • Reconstruct approximate geometry • Backward map color

  16. Layered Depth Images • Idea: • Handle disocclusion • Store invisible geometry in depth images • Data structure: • Per pixel list of depth samples • Per depth sample: • RGBA • Z • Encoded: Normal direction, distance • Pack into cache lines

  17. Layered Depth Images • Computation: • Incremental warping computation • Implicit ordering information • Process in up to four quadrant • Splat size computation • Table lookup • Fixed splat templates • Clipping of LDIs

  18. Layered Depth Images

  19. Micro-Polygon rendering • Goal: • Fill Holes by assuming smooth geometry • Approach: • Connect neighboring pixels with polygons • Delete polygons for very large steps • Render micro-polygons •  Heavy load on graphics system

  20. Some more Application: Stereo Example courtesy L. McMillan, MIT

  21. Some More Applications • Holography • Generating new views of models • Decoupling of rendering and image updates • Z is available • Avoid artifacts • Motion Capture • Matching of model to image data • Morphing for flexible models • Video Rewrite / Talking Head

  22. From Michael Cohen

  23. From Michael Cohen

  24. From Michael Cohen

  25. Image-Based Rendering • Limitations • Static scene geometry • Fixed lighting • Constrained look-from or look-at point • Possibilities: • Complete modeling and rendering system? • Integration of Computer Vision and Computer Graphics?

More Related