MIT Media Lab | Camera Culture

1. MIT Media Lab | Camera Culture Shoot Now, Relight Later achoo@mit.edu Illumination Multiplexing with Lock-In Time of Flight Sensors www.media.mit.edu/~achoo/demux ICCP 2014 AchutaKadambi, Ayush Bhandari, Refael Whyte, Adrian Dorrington, Ramesh Raskar How Can We Relight a Photograph? Illumination Multiplexing Time of Flight Operating Principle Time of Flight 3D Sensors Hardware Prototype Illumination Multiplexing Demonstration: Relighting Post-Capture Demonstration: Multiplexing Wavelength Nanosecond vs Millisecond Coding Demonstration: Color ToF camera by multiplexing RGB lights. In this paper we propose a fusion of two popular contexts, time of flight range cameras and illumination multiplexing. Time of flight cameras are a low cost, consumer-oriented technology capable of acquiring range maps at 30 frames per second. Such cameras have a natural connection to conventional illumination multiplexing strategies as both paradigms rely on the capture of multiple shots and synchronized illumination. While previous work on illumination multiplexing has exploited coding at millisecond intervals, we repurpose sensors that are ordinarily used in time of flight imagstrategiesing to demultiplex via nanosecond coding. Demonstration: Scene Relighting. Operation of ToF range cameras. The time difference of arrival between the emitted and received code is encoded in the phase offset. Thus, phase encodes depth. We validate our technique with a hardware prototype. It consists of an FPGA, Lock-in Sensor, and Solid State Illumination sources. Time of Flight 3D cameras are an emerging camera technology. Such technology forms the basis for the new Kinect. Illumination multiplexing is not a new problem, but requires fast captures. Figure from Schechner 2007 Illumination multiplexing is not a new problem, but requires fast captures. Figure from Schechner 2007 Case 1: Conventional High Speed Multiplexing Case 2: Using a lock-in ToF sensor allows for nanosecond coding. Case 3: Optimizing the Nanosecond Codes is the goal.