Manifold Bootstrapping for SVBRDF Capture

1. Manifold Bootstrapping for SVBRDF Capture Yue Dong, Jiaping Wang, Xin Tong, John Snyder, Moshe Ben-Ezra, Yanxiang Lan, Baining Guo Tsinghua University Microsoft Research Asia Microsoft Research

2. High-Quality SVBRDF Acquisition high spatial variation high angular variation fast and simple too!

3. Related Work • brute force (6D) measurement (gonioreflectometer) [Dana et al. 1999, McAllister et al. 2002, Lawrence et al. 2006] • slow • expensive, specialized rig

4. Related Work • single-pass fitting methods [Lensch et al. 2003, Goldman et al. 2005, Zickler et al. 2005] • measures large dataset • fits limited models(parametric/isotropic)

5. Related Work • two-pass methods • linearly combine two representatives based on diffuse color[Debevec et al. 2003]

6. Related Work • two-pass methods • linearly combine two representatives based on diffuse color[Debevec et al. 2003]

7. Related Work • two-pass methods • linearly combine two representatives based on diffuse color[Debevec et al. 2003] • use existing BRDF database of representatives: non-specialized and isotropic[Matusik et al. 2003b; Weyrich 2006]

8. Observation • BRDF spatial variation is complex: • tangent/normal/local frame rotates • specularity/anisotropy varies • specular lobe’s falloff and cross-section changes • forms low-dimensional manifold over given target. • manifold isn’t globally linear[Matusik et al. 2003a] • manifold is locally linear.

9. SVBRDF Manifold locally linear globally non-linear

10. Local vs. Global Interpolation local interpolation

11. Local vs. Global Interpolation global interpolation

12. SVBRDF Manifold Bootstrapping SVBRDF Manifold Representative Space

13. SVBRDF Manifold Bootstrapping Representative Space Representative Measurements

14. SVBRDF Manifold Bootstrapping Representative Space Representative Measurements Material Sample

15. SVBRDF Manifold Bootstrapping Representative Space Representative Measurements Key Measurements Material Sample

16. SVBRDF Manifold Bootstrapping Representative Space Representative Measurements every pixel Key Space Key Measurements

17. SVBRDF Manifold Bootstrapping Representative Space Representative Measurements x Local EmbeddingIn Key Space Key Space Key Measurements

18. SVBRDF Manifold Bootstrapping Representative Space Representative Measurements x Local EmbeddingIn Key Space Key Measurements

19. SVBRDF Manifold Bootstrapping Representative Space Representative Measurements Local Embedding of x In Representative Space x Local EmbeddingIn Key Space Key Measurements

20. SVBRDF Manifold Bootstrapping Representative Space Representative Measurements Reconstructed BRDF of x Local Embedding of x In Representative Space x Local EmbeddingIn Key Space Key Measurements

21. Results Real Material Sample

22. Outline • Data Acquisition • SVBRDF Reconstruction • Validation

23. Representative BRDFs • portable BRDF scanner • 6 LED light directions, 320x240 view directions • data amplification by microfacet model • 0.1s per BRDF

24. Key Measurements • fixed camera • background environmental lighting + moving area source

25. Timing • representative BRDFs and key measurements • 10-15 minutes • data processing • less than 5 minutes

26. Outline • Data Acquisition • SVBRDF Reconstruction • Validation

27. SVBRDF Reconstruction Representative BRDFs

28. Representative Local Interpolation ? + w3 = w1 + w2 BRDF of x x Representative BRDFs Material Sample

29. Representative Local Interpolation • choose which representatives to interpolate from • solve for weights wi ? w3 = w1 w1 + w2 + w3 w2 BRDF of x x Representative BRDFs Material Sample

30. Key Measurement Environment Lighting Representative BRDFs Projected Keys of Representative BRDFs Key Measurements Material Sample

31. Key Measurement Projected Keys of Representative BRDFs Key Measurements

32. Key Local Interpolation Key of x nearest neighbor in key space x Projected Keys of Representative BRDFs Key Measurements

33. Key Local Interpolation • solve for weights: LLE[Roweis & Saul 2000] where Key of x x + w3 = w1 + w2 Key Measurements

34. BRDF Reconstruction Neighborhood Key of x + w3 = w1 + w2 Local Embedding in Key Space

35. BRDF Reconstruction + w3 + w2 = w1 Key of x weights BRDF of x + w3 = w1 + w2 Local Embedding in Key Space

36. Outline • Data Acquisition • SVBRDF Reconstruction • Validation

37. Key Space vs. Representative Space • Projection depend on the environmental lighting conditions • preserve distances ⇒ preserve BRDF manifold structure

38. Key Space vs. Representative Space • Projection depend on the environmental lighting conditions • preserve distances ⇒ preserve BRDF manifold structure • global distances ⇒ preserve neighborhoods • local distances ⇒ preserve weights

39. Distance Preservation • preservation evaluation

40. Distance Preservation • preservation evaluation • # of lighting conditions

41. Distance Preservation • preservation evaluation • # of lighting conditions • criterion:global:τg> 0.9local:τl> 0.85

42. Results Real Material Sample

43. Extension to local frame variations • Normal variations • Tangent rotations

44. Representative Enlargement … enlarged BRDFs over normal rotation … enlarged BRDFs over tangent rotation

45. Results Real Material Sample

46. Results Real Material Sample

47. Conclusion • Manifold bootstrapping captures high-resolution SVBRDF • assumes BRDF forms low-dimensional manifold • decomposes acquisition into two phases • makes sparse measurement in both • phase one (representatives) = sparse spatial, dense angular • phase two (keys) = sparse angular, dense spatial • simplifies and accelerates the capture process

48. Conclusion • Manifold bootstrapping captures high-resolution SVBRDF • assumes BRDF forms low-dimensional manifold • decomposes acquisition into two phases • makes sparse measurement in both • phase one (representatives) = sparse spatial, dense angular • phase two (keys) = sparse angular, dense spatial • simplifies and accelerates the capture process

49. Acknowledgements • Paul Debevec for HDR images • Steve Lin for video narration • Anonymous reviewers for helpful comments

50. Thanks