1 / 60

Light Fields

Light Fields. Properties and applications. Outline. What are light fields Acquisition of light fields from a 3D scene from a real world scene Image rendering from light fields Changing viewing angle Changing the focal plane Sampling and reconstruction Depth vs spectral support

eljah
Download Presentation

Light Fields

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Light Fields Properties and applications

  2. Outline • What are light fields • Acquisition of light fields • from a 3D scene • from a real world scene • Image rendering from light fields • Changing viewing angle • Changing the focal plane • Sampling and reconstruction • Depth vs spectral support • Optimal reconstruction • Analysis of light transport

  3. Outline • What are light fields • Acquisition of light fields • from a 3D scene • from a real world scene • Image rendering from light fields • Changing viewing angle • Changing the focal plane • Sampling and reconstruction • Depth vs spectral support • Optimal reconstruction • Analysis of light transport

  4. The Plenoptic Function • Plenus – Complete, full. • Optic - appearance, look. • The set of things one can ever see • Light intensity as a function of • Viewpoint – orientation and position • Time • Wavelength • 7D function!

  5. The 5D Plenoptic Function • Ignoring wavelength and time • We need a 5D function to describe light rays across occlusions • 2D orientation • 3D position

  6. The Light Field (4D • Assuming no occlusions • Light is constant across rays • Need only 4D to represent the space of Rays • Is this assumption reasonable? • In free space, i.e outside the convex hull of the scene occluders

  7. The Light Field • Parameterizations • Point on a Plane or curved Surface (2D) and Direction on a Hemisphere (2D) • Two Points on a Sphere • Two Points on two different Planes

  8. Two Plane Parameterization • Convenient parameterization for computational photography • Why? • Similar to camera geometry (i.e. film plane vs lens plane) • Linear parameterization - easy computations , no trigonometric functions, etc.

  9. 2D light field Used for visualization. Assume the world is flat (2D)

  10. Intuition The image a pinhole at (u,v) captures All views of a pixel (s,t) Light Field Rendering , LevoyHanrahan '96.

  11. Outline • What are light fields • Acquisition of light fields • from a 3D scene • from a real world scene • Image rendering from light fields • Changing viewing angle • Changing the focal plane • Sampling and reconstruction • Depth vs spectral support • Optimal reconstruction • Analysis of light transport

  12. Acquisition of Light Fields • Synthetic 3D Scene • Discretize s,t,u,v and capture all rays intersecting the objects using a standard Ray Tracer

  13. Acquisition of Light Fields Real world scenes Will be explained in more detail next week…

  14. Outline • What are light fields • Acquisition of light fields • from a 3D scene • from a real world scene • Image rendering from light fields • Changing viewing angle • Changing the focal plane • Sampling and reconstruction • Depth vs spectral support • Optimal reconstruction • Analysis of light transport

  15. Changing the View Point • Problem: Computer Graphics • Render a novel view point without expensive ray tracing • Solution: • Sample a Synthetic light field using Ray Tracing • Use the Light Field to generate any point of view, no need to Ray Trace Light Field Rendering , LevoyHanrahan '96.

  16. Changing the View Point • Conceptually: Use Ray Trace from all pixels in image plane • Actually: Use Homographic mapping from XY plane to the VU and TS, and lookup resulting ray radiance. pinhole

  17. Light Field Interpolation • Problem: Finite sampling of the Light Field – • may not be sampled • Solution: Proper interpolation / reconstruction is needed • Nearest neighbor, • Linear, • Custom Filter • Detailed Analysis later on… NN + Linear NN Linear

  18. Changing the focal plane Fourier Slice Photography , Ng, 05

  19. In-Camera Light Field Parameterization

  20. The camera operator • Can define a camera as an operator on the Light Field. • The conventional camera operator: y x [Stroebel et al. 1986]

  21. D’ D Reminder - Thin lens formula 1 1 const + = D’ D To focus closer - increase the sensor-to-lens distance.

  22. Refocusing - Reparameterization

  23. Reparametrization - 4D

  24. Refocusing - Reparameterization Reparameterization of the light field Refocus Change of distance between planes Shearing of the Light field

  25. Refocusing camera operator • Shear and Integrate the original light field *(cos term from conventional camera model is absorbed into L)

  26. Computation of Refocusing Operator • Naïve Approach • For every X,Y go over all U,V and calculate the sum after reparameterization => O(n^4) • Can we do better ???? y x

  27. Fourier Slice Theorem • F – Fourier Transform Operator • I – Integral Projection Operator • S – Slicing Operator

  28. Fourier Analysis of the Camera Operator • Recall that the Refocusing Camera Operator is: • And from the Last theorem we get The Fourier Slice Photography Theorem • Better Algorithm!

  29. Fourier Slice Photography , Ng, 05

  30. Fourier Slice Photography Thm – More corollaries • Two important results that are worth mentioning: • 1. Filtered Light Field Photography Thm • 2. The light field dimensionality gap *K=?

  31. Filtered Light Field Photography Thm • Theorem: Filtered Light Field Photography

  32. The light field dimensionality gap • The light field is 4D • In the frequency domain – The support of all the images with different focus depth is a 3D manifold • This observation was used in order to generate new views of the scene from a focal stack (Levin et al. 2010)

  33. Outline • What are light fields • Acquisition of light fields • from a 3D scene • from a real world scene • Image rendering from light fields • Changing viewing angle • Changing the focal plane • Sampling and reconstruction • Depth vs spectral support • Optimal reconstruction • Analysis of light transport

  34. Light Field Sampling • Light Field Acquisition – Discretization • Light Field Sampling is Limited Example – Camera Array: t,s u,v

  35. Sampling in frequency domain • Aliasing in the frequency domain • Need to analyze Light Field Spectrum ALIASING! = * No Aliasing!

  36. Scene Depthand Light Field • Light Field Spectrum is related to Scene Depth • From Lambertianproperty each point in the scene corresponds to a line in the Light Field • Line slope is a function of the depth (z) of the point. Plenoptic Sampling , Chai et al., 00.

  37. Spectral Support of Light Field • Constant Depth Scene Light Field LF Spectrum Plenoptic Sampling , Chai et al., 00.

  38. Spectral Support of Light Field • Varying Depth Scene LF Spectrum Plenoptic Sampling , Chai et al., 00.

  39. Spectral Support of Light Field Plenoptic Sampling , Chai et al., 00.

  40. Reconstruction Filters Optimal Slope for filter: Plenoptic Sampling , Chai et al., 00.

  41. Limitations • Assumptions • Lambertian surfaces • Free Space – No occlusions

  42. Frequency Analysis of Light Transport • Informally: Different features of lighting and scene causes different effects in the Frequency Content • Blurry Reflections • Shadow Boundries High frequency Low frequency A Frequency analysis of Light Transport , Durand et al. 05.

  43. Not Wave Optics!!!

  44. Frequency Analysis of Light Transport • Look at light transport as a signal processing system. • Light source is the input signal • Interaction are filters / transforms Source Transport Occlusion Transport Reflection (BRDF)

  45. Local Light Field • We study the local 4D Light Field around a central Ray during transport • In Spatial Domain • In Frequency Domain * Local light field offers us the ability to talk about the Spectrum In a local setting

  46. Local Light Field (2D) Parameterization • The analysis is in flatland, an extension to 4D light field is available x-v parameterization x-Θ parameterization A Frequency analysis of Light Transport , Durand et al. 05.

  47. Example Scenario Reflection A Frequency analysis of Light Transport , Durand et al. 05.

  48. Light Transport – Spatial Domain • Light Propagation  Shearof the local Light Field • No change in slope (v) • Linear change in displacement (X)

  49. Light Transport – Frequency Domain • Shear in spatial domain is also a shear in Frequency domain

  50. Occlusion • Spatial domain: • Occlusion  pointwise multiplication in the spatial domain • The incoming light field is multiplied by the binary occlusion function of the occluders. • Frequency domain • convolution in the frequency domain:

More Related