1 / 57

Appearance modeling: textures and IBR Class 17

This course covers topics such as 3D photography, texture mapping, appearance modeling, image-based rendering, and surface lightfields. Explore papers and projects related to these subjects.

marthacox
Download Presentation

Appearance modeling: textures and IBR Class 17

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Appearance modeling: textures and IBR Class 17

  2. 3D photography course schedule

  3. Papers http://www.unc.edu/courses/2004fall/comp/290b/089/papers/

  4. Projects

  5. Volumetric 3D integration Multiple depth images Volumetric integration

  6. Appearance Modeling • Texturing • Single image • Multiple image • Image-based rendering • (Unstructured) lightfield rendering • Surface lightfields

  7. Texture mapping 3D model Need to estimate relative pose between camera and 3D model

  8. Texture Mapping • Conventional texture-mapping with texture coordinates • Projective texture-mapping

  9. Texture Map Synthesis I • Conventional Texture-Mapping with Texture Coordinates • Create a triangular texture patch for each triangle • The texture patch is a weighted average of the image patches from multiple photographs • Pixels that are close to image boundaries or viewed from a grazing angle obtain smaller weights Photograph 3D Triangle Texture Map

  10. Texture Map Synthesis II • Allocate space for texture patches from texture maps • Generalization of memory allocation to 2D • Quantize edge length to a power of 2 • Sort texture patches into decreasing order and use First-Fit strategy to allocate space First-Fit

  11. A Texture Map Packed with Triangular Texture Patches

  12. Appearance Modeling texture atlas

  13. Dealing with auto-exposure (Kim and Pollefeys, CVPR’04) Photometric alignment of textures (or HDR textures)

  14. Image as texture Depth image Texture image Triangle mesh Textured 3D Wireframe model Affine vs. projective texture mapping (see later)

  15. Lightfield literature • Plenoptic function • Lightfield (plane) and Lumigraph (some geometry) • Unstructered lightfield (some (view-dependent) geometry) • Surface lightfields (full geometry) • Plenoptic sampling (trade-off geometry vs. images) (Adelson&Bergen´91; McMillan&Bishop,Siggraph´95) (Levoy&Hanrahan,Siggraph´96 Gortler et al.,Siggraph´96) (Koch et al. ICCV´99; Heigl et al. DAGM´99; Buehler et al. Siggraph‘01) (Wood et al.,Siggraph´00, Chen et al., Siggraph‘02) (Chai et al.,Siggraph´00)

  16. viewpoint surface focal surface Lightfield rendering Approximate light rays by interpolating from closest light rays in lightfield • Transfer from images to virtual views over • focal surface determines which pixels to use • Projection of viewpoint surface in virtual camera • determines which views to get lightrays from

  17. Unstructured lightfield rendering (Koch et al.,ICCV´99; Heigl et al.,DAGM´99) For every pixel, combine best rays from closest views Novel view original viewpoints Focal surface demo

  18. Example: desk sequence 186 images recorded with hand-held camera

  19. Example: desk sequence structure and motion 7000points 190 images depth images

  20. Example: Desk Lightfield Planar focal surface (shadow artefacts)

  21. depth maps View-dependent geometry approximation object surface View-dependent surface approximation Novel view original viewpoints

  22. View-dependent geometry approximation Adaptation of geometry with the rendering viewpoint

  23. Geometry subdivision object surface View-dependent surface approximation Novel view original viewpoints depth maps Note: Only necessary when depth value significantly deviates from previous approximation

  24. Scalable geometric approximation Viewpoint-geometry without subdivision 2subdivisions 1 subdivision of viewpoint surface 4 subdivisions

  25. Example: Desk lightfield View-dependent geometry approximation (2 subdivisions) Planar focal surface

  26. Hardware accelerated rendering Use blending operation similar to Gouraud shading Use projective textures!

  27. Demo demo

  28. Extrapolation (Buehler et al., Siggraph´01) Rendered image Add mesh to cover whole image (compute non-binary blending weights) Blending field (courtesy Leonard McMillan)

  29. Surface Lightfields Surface light field (SLF) function Surface location Viewing direction Chen et al., Siggraph 2002, "Light Field Mapping: Efficient Representation and Hardware Rendering of Surface Light Fields" R. Grzeszczuk, Presentation on Light Field Mapping, SIGGRAPH 2002 Course Notes for Course “Image-based Modeling.” http://www.intel.com/research/mrl/research/lfm/

  30. Surface maps View maps Light field maps: stored as 2D texture maps Surface Lightfields • Surface light field (SLF) function • Partition SLF across surface primitives Pi • Approximate SLF for each Pi individually as

  31. Data Acquisition Partitioning Resampling Approximation Compression Rendering Light Field Mapping

  32. Data Acquisition Partitioning Resampling Approximation Compression Rendering Light Field Mapping

  33. Data Acquisition • 200-400 images captured by hand-held camera • Geometry scanned with structured lighting • Images registered to geometry

  34. Data Acquisition Partitioning Resampling Approximation Compression Rendering Light Field Mapping

  35. Triangle-centered: split the light field between individual triangles Partitioning • Partitioning the light field data across small surface primitives • Individual parts add up to original SLF • Ensure continuous approximations across neighbouring surface elements

  36. Partitioning • Partitioning the light field data across small surface primitives • Individual parts add up to original SLF • Ensure continuous approximations across neighbouring surface elements Triangle-centered: split the light field between individual triangles ->discontinuity 

  37. = Vertex-centered Partitioning • Partition surface light field data around every vertex Hat function

  38. Vertex-centered Partitioning

  39. Vertex-centered Partitioning • Define local reference frame of the vertex • Reparameterize each vertex light field to its local coordinate system Vertex light field

  40. Data Acquisition Partitioning Resampling Approximation Compression Rendering Light Field Mapping

  41. Resampling • Goal: Generate vertex light field function • Visibility computation determines unoccluded views for each triangle ring • 2 steps: • Normalization of texture size • Resampling of viewing directions

  42. Resampling Each column represents a different view 2nd view 1st view Ci-th view

  43. Resampling • 1. Normalization of texture size • Each texture patch has the same shape and size • Bilinear interpolation • 2. Resampling of viewing directions

  44. 1 2 3 4 ……. c ….. Ci 1 2 m ……. M Resampling 1. Normalization of texture size 2. Resampling of viewing directions Projection of original views

  45. 1 2 3 4 ……. c ….. Ci 1 2 m ……. Resampling 1. Normalization of texture size 2. Resampling of viewing directions Delaunay triangulation Uniform grid of views M

  46. Resampling 1. Normalization of texture size 2. Resampling of viewing directions 1 2 3 4 ……. n ….. N 1 2 m ……. M

  47. Data Acquisition Partitioning Resampling Approximation Compression Rendering Light Field Mapping

  48. N Surface maps View maps K K<<N Decomposition & Approximation • Rearrange 4-dimensional F into M*N matrix • Decompose F using matrix factorization • Truncate the sum after K terms

  49. Decomposition & Approximation • Split surface maps for triangle ring into surface maps for individual triangles • 3 surface maps for each approximation term of each triangle

  50. Decomposition & Approximation Each approximation =3 surface maps + 3 view maps 1st approximation 2nd approximation f …. Kth approximation

More Related