1 / 44

Course 1: Room 6DE Computational Photography

Dive into the world of film-like photography through a computational lens, understanding aspects like lighting, optics, and sensors in the 4D-ray space. Uncover the trade-offs of no-flash photography and the complexities of flash film-like photography, delving into ray models and lens systems. Explore the imaging intuition of film-like optics and the intricacies of lens measurements, focal lengths, and aperture for depth-of-focus. Gain insights into lens flaws, aberrations, and distortion, and learn about film-style sensors and dynamic range limits. Discover the challenges of mapping scenes to human vision and color sensing in film-like photography.

tvernon
Download Presentation

Course 1: Room 6DE Computational Photography

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Course 1: Room 6DE Computational Photography A.4: Understanding Film-like Photography Tumblin

  2. Computational Photography A3:UnderstandingFilm-Like Photography(30 minutes) Jack Tumblin Northwestern University

  3. ‘Film-Like’ Photography Film Camera designs still dominate: • ‘Instantaneous’ light measurement… • Of focal plane image behind a lens. • Reproduce those amounts of light; • Display ‘exactly matches’ the scene Let’s examine them using 4D-ray space terms:

  4. Film-Like Photography: Special Case • Lighting: Ray Sources (external) • Scene: Ray Modulator (external) • Optics: Ray Benders Thin Lens Approx. • Sensors: Ray Bundle Sensor Irradiance Measurement E(x,y) • Processing Ray Normalized E(x,y) • Display Recreate Rays Normalized E(x,y)

  5. No-flash Photography: Full of Tradeoffs... • Available light vs. exposure time vs. scene movement vs. field of view vs. focus depth vs. sensitivity vs. noise vs. color rendition vs. color gamut vs. contrast vs. visible detail vs. …. Flash

  6. ‘Film-Like Photography’: Ray Model Light + 3D Scene: Illumination, shape, movement, surface BRDF,… Image: Planar 2D map of light intensities Image Plane I(x,y) Position(x,y) Angle(,) ‘Center of Projection’ (P3 or P2 Origin)

  7. Ray BUNDLES approximate Rays • Lens Systems: approximate rays with bundles • Finite angle, not rays (lens aperture) • Finite area, not points (circle of confusion) http://www.nationmaster.com/encyclopedia/Lens-(optics)

  8. Film-like Optics: Imaging Intuition Well-Lit 3D Scene: Ray ‘Center of Projection’ Position (x,y) Angle(,) 2D Sensor: Pixel Grid or Film,… ‘Pinhole’ Model: Rays copy scene onto ‘film’

  9. Film-like Optics: Imaging Intuition Scene Sensor Ray Lens Angle(,) Position (x,y) ‘Center of Projection’ ‘Pinhole’ Model: Rays copy scene onto ‘film’

  10. Not One Ray, but aBundle of Rays . Scene Sensor Ray Lens Angle(,) Position (x,y) ‘Center of Projection’

  11. Not One Ray, but a Bundle of Rays Lens Scene Sensor Aperture • (BUT Ray model isn’t perfect: ignores diffraction) • Lens, aperture, and diffraction sets the point-spread-function (PSF) (How? See: Goodman,J.W. ‘An Introduction to Fourier Optics’ 1968)

  12. Review: Lens Measurements Lens Scene Sensor S1 S2 • How do we compute S1 and S2 for a lens? • What is the ‘Ray-Bending Strength’ for a lens?

  13. Review: Focal Length f • Lens focal lengthf : where parallel rays converge Lens S1 =  S2=f

  14. Review: Focal Length f • Lens focal lengthf : where parallel rays converge • smaller focal length: more ray-bending ability… Lens S1 =  f

  15. Review: Focal Length f • Lens focal lengthf : where parallel rays converge • greaterfocal length: less ray-bending ability… • For flat glass; for air : f =  Lens S1 =  f

  16. Review: Thin Lens Law Lens Scene Sensor f f S2 S1 • Thin Lens Law: in focus when: • Note that S1  f and S2 f http://webphysics.davidson.edu/Applets/Optics/intro.html Try it Live! Physlets…

  17. Aperture and Depth-Of-Focus: Lens Scene Sensor f f Focus Depth Blur S2 S1 • For samefocal length: • Smaller Aperture  Larger focus depth, but less light

  18. Aperture and Depth-Of-Focus: Lens Scene Sensor f f Focus Depth Blur S2 S1 • For samefocal length: • Larger Aperture  smaller focus depth, but more light

  19. Aperture and Depth of Focus Smaller aperturedeepens focus *BUT* Reduceslight Diffraction limits!

  20. Impossible Focusing Tasks? • Some very deep objects REQUIRE out-of-focus images; No lens + aperturehas enough depth-of-focus!

  21. Focal Length vs. Viewpoint vs. Focus Wide angle isn’t flattering; do you know why? Wideangle Standard Telephoto Large/Deep  Depth of Focus  Small/shallow

  22. Lens Flaws: Glare, Flare, Ghosts… • Unwanted interreflections, scatter,specularity

  23. Basic Ray Optics: Thin Lens Flaws Aberrations: Real lenses don’t converge rays perfectly • Spherical: edge rays  center rays • Coma: diagonal rays focus deeper at edge http://www.nationmaster.com/encyclopedia/Lens-(optics)

  24. Radial Distortion (e.g. ‘Barrel’ and ‘pin-cushion’) straight lines curve around the image center

  25. Vignette Effects • Internal shadowing: • Oblique sensor: cos  • Ray path length is longer off-axis. • Compensating Mask: • Darken image center… • (Computed or optical) http://homepage.ntlworld.com/j.houghton/vignette.htm

  26. Film-Style Sensors: Dynamic Range Limits • Under-Exposure • Highlight details: Captured • Shadow details: Lost • Over-Exposure • Highlight details: Lost • Shadow details: Captured

  27. Problem:Map Scene to Display Domain of Human Vision: from ~10-6 to ~10+8 cd/m2 starlight moonlight office light daylight flashbulb 10-6 10-2 1 10 100 10+4 10+8 ?? ?? 0 255 Range of Typical Displays: from ~1 to ~100 cd/m2

  28. Film-like Color Sensing • Visible Light:   400-700 nM wavelength • ‘Monochrome’ == single spectral channel • Equiluminant Curvedefines ‘luminance’ vs. wavelength http://www.yorku.ca/eye/photopik.htm

  29. Film-like Color Sensing • Human Color Vision: 3 types of cones (S,M,L) • ‘Full Color’ Cameras, Displays: 3 spectral bands (e.g. R,G,B) • RGB spectral curves Vaytek CCD camera with Bayer grid www.vaytek.com/specDVC.htm

  30. Color Sensing • 3-chip: separate R,G,B sensors, vs. • 1-chip: interleaved R,G,B: quality vs. cost http://www.cooldictionary.com/words/Bayer-filter.wikipedia

  31. 1-Chip Color Sensing: Bayer Grid • Estimate RGBat ‘G’ cels from neighboring values http://www.cooldictionary.com/words/Bayer-filter.wikipedia

  32. Lens Flaws: Chromatic Aberration • Dispersion: wavelength-dependent refractive index • (enables prism to spread white light beam into rainbow) • Modifies ray-bending and lens focal length: f() • color fringes near edges of image • Corrections: add ‘doublet’ lens of flint glass, etc. http://www.swgc.mun.ca/physics/physlets/opticalbench.html

  33. Chromatic Aberration • Lens Design Fix: Multi-element lenses Complex, expensive, many tradeoffs! • Computed Fix: Geometric warp for R,G,B Image Center Image Edge

  34. Computational Illumination

  35. Make this brighter Make this darker Remove this specular highlight Soften this shadow Move shadow back

  36. Processing: Polarization? Sunlit haze is often strongly polarized. Polarization filter yieldsmuch richer sky colors

  37. Processing: Very Long Exposure ? Michael Wesely: Open Shutter at The Museum of Modern Art • http://www.wesely.org/moma.php?show_page=1 • http://www.moma.org/exhibitions/2004/Michael_Wesely_11-20-04.html Postdamer Platz, Berlin 18 month long exposure 26 Month long exposure: Notice the sun tracks

  38. Many Limitations & Tradeoffs:(how can computing change them?) • Optics: Single focus distance, limited depth-of-field, limited field-of-view, internal reflections/flare/glare • Lighting: Camera has no knowledge of ray source strength, position, direction; little control (e.g. flash) • Sensor: Exposure setting, motion blur, noise, response time,… • Processing: • Quantization/color depth, camera shake, scene movement…

  39. Conclusions • Film-like photography methods limit digital photography to film-like results (or less!). • Broaden, unlock our views of photography: • 4-D, 8-D, even 10-D Ray Space holds the photographic signal. Look for new solutions by creating, gathering, processing RAYS, not focal-plane intensities. • Choose the best, most expressive sets of rays, THEN find the best way to measure them.

  40. Useful links: Interactive Thin Lens Demo (or search ‘physlet optical bench’) www.swgc.mun.ca/physics/physlets/opticalbench.html For more about color: • Prev. SIGGRAPH courses (Stone et al.) • Good: www.cs.rit.edu/~ncs/color/a_spectr.html • Good: www.colourware.co.uk/cpfaq.htm • Good: www.yorku.ca/eye/toc.htm

  41. RAYS and PROCESSING • ONE Ray carries doubly infinitesimal power: Ray bundles with finite, measurable power will: • Span a non-zero area • Fill a non-zero solid angle • Everything is Linear: (HUGE win!) Ray reflectance, transmission, absorption, scatter*… • Rays are REVERSIBLE. Helmholtz reciprocity Ray bundles? Not so much: falls quickly with angle,area growth…

More Related