1 / 31

An Incremental Weighted Least Squares Approach To Surface Light Fields

An Incremental Weighted Least Squares Approach To Surface Light Fields. Greg Coombe Anselmo Lastra. Image-Based Rendering. Generate new views of a scene from existing views Sample appearance from physical world. Lightfields [Levoy96], Lumigraphs [Gortler96]. …. Surface Lightfields.

roscoe
Download Presentation

An Incremental Weighted Least Squares Approach To Surface Light Fields

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. An Incremental Weighted Least Squares Approach To Surface Light Fields Greg Coombe Anselmo Lastra

  2. Image-Based Rendering • Generate new views of a scene from existing views • Sample appearance from physical world Lightfields [Levoy96], Lumigraphs [Gortler96] …

  3. Surface Lightfields • A surface light field [Wood00] represents the appearance of a model with known geometry and static lighting (θ,Φ) surface position (u,v) viewing direction

  4. Surface Lightfields f1(Θ, Φ) f2(Θ, Φ) f3(Θ, Φ) f4(Θ, Φ)

  5. SLF - Batch Process Surface Lightfield Construction Renderer disk … [Chen02, Hillesland03]

  6. SLF - Online Process Incremental Surface Lightfield Construction Renderer 1024x768 @ 15fps [Coombe05]

  7. Scattered Data Approximation • [Coombe05] required resampling to grid • Samples are at arbitrary locations in domain due to geometry and camera ? ?

  8. Scattered Data Approximation • Scattered Data Approximation in lightfields • Unstructured Lightfields [Buehler01] • Tesselation of pure lightfield • Polynomial Texture Maps [Malzbender01] • Fit polynomials to set of images • Radial Basis Functions [Zickler05] • Interpolate sparse reflectance data

  9. Outline • A 2D scattered data approximation • Fast incremental construction • Realtime evaluation

  10. Our Approach • Represent the surface lightfield using Weighted Least Squares approximation • Modify WLS for the incremental framework • Adaptive and Hierarchical • CPU/GPU implementation • Process each image in about 1-2 seconds • Real-time rendering

  11. Bust model, 75 training images

  12. Least Squares Fitting • Find the “best” polynomial approximation to the input samples • “best” means minimizes sum of squared differences • the coefficients are determined by solving a linear system input samples reconstruction

  13. Least Squares reconstructed function input samples domain

  14. Weighted Least Squares • Problem: LS is a global approximation • Solution: Divide domain into multiple LS approximations, and combine to get global approximation • Use a set of low-degree polynomials • Non-linear blending (Partition of Unity) • Good discussion in Scattered Data Approximation, Holgar Wendland

  15. Weighted Least Squares reconstructed function polynomial approximations input samples domains centers

  16. Weighted Least Squares Θ x x x x Φ

  17. Outline • A 2D scattered data approximation • Fast incremental construction • Realtime evaluation

  18. Incremental WLS • Feedback is important in SLF construction • As each image is captured, it must be incorporated into the representation • How do we determine the size of each domain?

  19. Adaptive Construction • Start out with large domains • Adaptive shrink as more points arrive x x x x x x x x

  20. Hierarchical Construction • Start out with a single domain • Subdivide as more points arrive (quadtree) x x x x x x x x x x x

  21. Hierarchical Construction, First 10 images

  22. Hierarchical and Adaptive • Hierarchical • 1x1 Polynomial Texture Mapping • Fast at first, slows down as refines • # of domains is a power of 2 • Adaptive • Slow at first, speeds up as domains shrink • Can handle arbitrary # of domains

  23. Outline • A 2D scattered data approximation • Fast incremental construction • Realtime evaluation

  24. Implementation Pose Estimation Visibility / Reprojection Incremental Weighted Least Squares 1024x768 @ 15fps GPU Renderer

  25. Θ Θ Φ Φ Θ Φ Θ Φ GPU Implementation

  26. Results 4K patches 29K patches side by side, 14K patches

  27. Pitcher model, 65 images

  28. Performance • ~ 0.5 - 2 seconds per image for hierarchical construction • 0.5s for 4K bust model • 2s for 30K pitcher model • 95% is Least Squares Fitting • Adaptive is 2-3x more expensive • Rendering is 60fps or more

  29. Conclusion • Represent the surface lightfield using Weighted Least Squares approximation • Modify WLS for the incremental framework • Adaptive and Hierarchical • CPU/GPU implementation • Process each image in about 1-2 seconds • Real-time rendering

  30. Future Work • Since construction is dominated by LS fitting, implement on GPU • Extend to surface reflectance (4D) • Change basis functions • Order-of-magnitude more data

  31. Thanks • NVIDIA Graduate Student Fellowship • UNC Graphics Group • National Science Foundation • Questions?

More Related