Computational Photography

Computational Photography • Ramesh Raskar • Mitsubishi Electric Research Labs, Cambridge MA http://www.merl.com/people/raskar/photo

Traditional ‘film-like’ Photography Detector Lens Pixels Image

Computational Photography Novel Cameras GeneralizedSensor Generalized Optics Processing

Computational Photography Programmable Lighting Light Sources Modulators Novel Cameras Generalized Optics GeneralizedSensor Generalized Optics Processing Scene

Computational Photography Programmable Lighting Light Sources Novel Cameras a. Multi-flash Illum GeneralizedSensor Generalized Optics Processing c. Image Fusion b. SAMPd. Camera Array e. Gradient Sensing f. Flutter Shutter Scene

Depth Edge Camera

Depth Discontinuities Internal and externalShape boundaries, Occluding contour, Silhouettes

Shadows Clutter Many Colors Highlight Shape Edges Mark moving parts Basic colors

SAMP Camera(Single Axis Multiple Parameters) Parameters vary in focus, exposure, and aperture.

Multiparameter Camera

A Night Time Scene: Objects are Difficult to Understand due to Lack of Context Dark Bldgs Reflections on bldgs Unknown shapes

Enhanced Context :All features from night scene are preserved, but background in clear ‘Well-lit’ Bldgs Reflections in bldgs windows Tree, Street shapes

Night Image Background is captured from day-time scene using the same fixed camera Result: Enhanced Image Day Image

But, Simple Pixel Blending Creates Ugly Artifacts

Gradient CameraSensing Difference between Neighboring Pixels Short Exposure Goal: High Dynamic Range Long Exposure

High Dynamic Range ImagesSensing Pixel Difference with Locally Adaptive Gain Scene Intensity camera saturation map Gradient camera saturation map Intensity camera fails to capture rangeGradients saturate at very few isolated pixels

Figure 2 results Problem: Motion Deblurring Input Image

Blurred Taxi Image Deblurred by solving a linear system. No post-processing

Time = T Time = 0 Application: Aerial Imaging Sharpness versus Image Pixel Brightness Long Exposure: The moving camera creates smear Shutter Open Time Shutter Closed Short Explosure:Avoids blur. But the image is dark Shutter Open Time Shutter Closed Time Solution: Flutter Shutter Shutter Open Goal: Capture sharp image with sufficient brightness using a camera on a fast moving aircraft Time Shutter Closed

Fluttered Shutter Camera Raskar, Agrawal, Tumblin Siggraph2006 Ferroelectric shutter in front of the lens is turnedopaque or transparent in a rapid binary sequence

Coded Exposure Photography: Assisting Motion Deblurring using Fluttered Shutter Raskar, Agrawal, Tumblin (Siggraph2006) Short Exposure Traditional MURA Coded Shutter Captured Photos Deblurred Results Result has Banding Artifacts and some spatial frequencies are lost Decoded image is as good as image of a static scene Image is dark and noisy

Ramesh Raskar, MERL Computational Photography Novel Cameras GeneralizedSensor Programmable Lighting Generalized Optics Light Sources Processing a. Multi-flash Illum c. Image Fusion b. SAMPd. Camera Array e. Gradient Sensing f. Flutter Shutter

Computational Photography