Symmetric Photography: Exploiting Data-sparseness in Reflectance Fields

1. Symmetric Photography:Exploiting Data-sparseness in Reflectance Fields Gaurav Garg Eino-Ville Hendrik P. A. Lensch Marc Levoy

2. Symmetric Photography:Dealing with 8D Reflectance Fields Relighting Example Capture Ground Truth

3. Overview • Full 8D Reflectance field! • Changing View (4D) * Changing Light (4D) • Eg.: For Each 4D, 3x3 images at 100x100 res results in 10^10 4D table • How do we deal with data explosion? • Exploit Symmetry between light/view • Helmholtz reciprocity • Exploit Data Sparseness

4. Symmetric Photography Transport equation: • T is symmetric (Helmholtz reciprocity) • T is not sparse • But sub-blocks of T are “data sparse”

5. Visualizing Symmetry andData-sparsity

6. Outline • Data Acquisition Setup • Exploiting Symmetry and Data Sparsity in the Transport Matrix • Results

7. Symmetric Setup

8. Acquisition

9. Outline • Data Acquisition Setup • Exploiting Symmetry and Data Sparsity in the Transport Matrix • Results

10. Hierarchical Tensors –Parallel Acquisition • If M=0, U1 and U2 are radiometrically isolated. • If M!=0, but is known, we can subtract it out to isolate U1 and U2 • This allows us to illuminate projector pixels in U1 and U2 in parallel.

11. Hierarchical Tensors –Rank-1 Approximation • “An image captured by the camera is the sum of the columns corresponding to the pixels lit by the projector. The image is also the sum of the corresponding rows” • Use two projector patterns (Pr and Pc) s.t. and • The rank-1 approximation of M is

12. Hierarchical Acquisition • Already have a rank-1 approximation • For root node, use flood lit image for first approximation • Divide node by 16 and move to next level • 4 projector blocks X 4 camera blocks • Use 4 projector patterns and capture 4 images (8 images total) • Evaluate previous level’s rank-1 approx against these images • If good enough, finish • If the size of the projector block is down to a pixel, finish • Else, use these images to create 16 rank-1 approximations, and goto 1.) for each of them Note – I have heavily glossed over the selection of projector patterns

13. Outline • Data Acquisition Setup • Exploiting Symmetry and Data Sparsity in the Transport Matrix • Results

14. Changing Light

15. Changing View

16. Symmetric vs. Dual Photography

17. Artifacts Due to Non-Symmetry

18. Hierarchy Levels

19. Table