1 / 24

Plenoptic Sampling

Plenoptic Sampling. Jian Huang, CS 594, Spring 2002. Light Field Methods. For image-based rendering algorithms based on plenoptic function, some way of discrete sampling is always used What is the optimal sampling? Can Nyquist sampling theory be applied?

valora
Download Presentation

Plenoptic Sampling

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Plenoptic Sampling Jian Huang, CS 594, Spring 2002

  2. Light Field Methods • For image-based rendering algorithms based on plenoptic function, some way of discrete sampling is always used • What is the optimal sampling? Can Nyquist sampling theory be applied? • Plenoptic Sampling, Proc. SIGGRAPH’2000, Jin-Xiang Chai, Xin Tong, Shing-Chow Chan, Heung-Yeung Shum, Microsoft Research, China • Minimal sampling rate for light fields

  3. Background • Texture mapping, very accurate geometry and only a few images • Image-based methods with depths: • 3D warping, LDI (layered depth image), view morph, interpolation • A few images with depth information • Light field methods • Do not assume any depth information: light fields, lumigraph, concentric mosaics • Rely on over-sampling (large amount images) • Intensive data acquisition, storage and processing

  4. Two Plane Parametrization Object Focal plane (st) Camera plane (uv)

  5. Reconstruction

  6. Plenoptic Sampling • How many samples of the plenoptic function (e.g., from a 4D light field) and how much geometrical and textural information are needed to generate a continuous representation of the plenoptic function? • Two specific goals: • Minimum sampling rate for light field rendering • Minimum sampling curve in joint image and geometry space

  7. Formulation • A high-dimension signal processing problem • Assumption • Lambertian surface • Uniform sampling geometry or lattice • Study: • Spectral support of light field signals • Not closed-form spectral representation

  8. A Key Concept • The spectral support of a light field signal is bounded by only the minimum and maximum depths, irrespective of how complicated the spectral support might be because of depth variations in the scene. • Given the minimum and maximum depths, a reconstruction filter with an optimal and constant depth can be designed to achieve anti-aliased light field rendering.

  9. Contribution • The minimum sampling rate of light field rendering is obtained by compacting the replicas of the spectral support of the sampled light field within the smallest interval without any overlap. • Using more depth information, plenoptic sampling in the joint image and geometry space allows us to greatly reduce the number of images needed. • The relationship between the number of images and the geometrical information under a given rendering resolution can be described by a minimum sampling curve. • This minimal sampling curve serves as the design principles for IBR systems, bridging the gap between image-based rendering and traditional geometry-based rendering.

  10. Convolution • Sampling a continuous light field function, l, with sampling pattern, p, reconstruction filter (kernel), r, reproducing images i.

  11. Spectral Support • Let z(u,v,s,t) to be the depth function, i.e. geometry • The same point is views in camera 0 and t as point v and v’. • Assuming Lambertian surface, each epi-polar line is of uniform color

  12. Radiance • The radiance received at camera location (s,t) is: • With the Fourier transform being: (complicated to compute)

  13. Spectral Support • Using a rectangular sampling lattice, the sampled function: • L has to be bandlimited. • Need to use a sampling rate higher than Nyquist for alias-free sampling.

  14. Research Question • Is there an optimal reconstruction function (like sinc in conventional signal processing)? • The design of such a function is related to the depth function. How exact should depth be captured and recovered?

  15. Scene with constant depth • Choosing the first frame as reference, l(u,v,0,0).

  16. Spectral Support • Focusing on sampling in (v,t) sub-space

  17. Spatially varying depth model • Straightforward to observe that the spectral support of a scene of depth range [zmin, zmax] is bounded by two lines:

  18. Scene Images

  19. Reconstruction at constant depth • Assuming a constant depth in reconstruction • Optimal depth:

  20. Sample Renderings • At min depth, optimal depth, average depth and max depth:

  21. Minimum Sampling Rate • How tight can you pack:

  22. Joint Image and Geometry Space • With a precise geometry, can decompose a scene into multiple layers of depths

  23. Joint Image and Geometry Space • With depth uncertainty, for instance, a noisy depth image Obviously, the curve will be scene dependent!

  24. Results (more in the paper)

More Related