1 / 44

A Theory of Multiplexed Illumination

A Theory of Multiplexed Illumination. Technion, Israel. Yoav Y. Schechner. Shree Nayar, Peter Belhumeur. Columbia University. ICCV Conference October 2003, Nice, France. Acknowledgments: Taub Foundation, BSF, NSF, Harish Peri. Georghiades, Belhumeur & Kriegman

nclancy
Download Presentation

A Theory of Multiplexed Illumination

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Theory of Multiplexed Illumination Technion, Israel Yoav Y. Schechner Shree Nayar, Peter Belhumeur Columbia University ICCV Conference October 2003, Nice, France Acknowledgments: Taub Foundation, BSF, NSF, Harish Peri

  2. Georghiades, Belhumeur & Kriegman Yale Face Database B Shape Recovery BRDF Material Recognition Human Vision Rendering Object/Face Recognition

  3. Synthesized illumination Raw images: individual sources Image-Based Rendering based on Schechner et. al. Multiplexed Illumination

  4. High quality New capabilities Very simple, yet flexible setup Our Method Schechner, Nayar, Belhumeur Multiplexed Illumination

  5. Image-Based Rendering / Recognition + + Schechner et. al. Multiplexed Illumination

  6. Low res. High res. Illumination Direction Resolution Schechner et. al. Multiplexed Illumination

  7. Illumination direction resolution vs. Image intensity Trade-Off Schechner et. al. Multiplexed Illumination

  8. Image intensity Illumination direction resolution vs. signal = SNR 1 N elements ~ noise N ~ 1 N N independent images Exposure time ~ N Trade-Off Schechner, Nayar, Belhumeur Multiplexed Illumination

  9. Image intensity Illumination direction resolution vs. signal = SNR 1 N elements ~ noise N ~ 1 N N independent images Exposure time ~ N Schechner, Nayar, Belhumeur Multiplexed Illumination

  10. Multiplexed Illumination Standard Illumination

  11. Multiplexed Illumination 2 2 2 3 3 3 1 1 1 i a1 1 0 1 1 i 2 a0 1 1 = i 2 3 1 2 3 Estimating i 1 i1 -1 1 a 1 2 i = 1 1 -1 a 2 i-1 1 1a 3 a1 0 1 3 i = intensity under sourcek a = acquired measurement k

  12. Standard Illumination i = intensity under source k i = estimate ofi a = acquired measurement 2 2 2 k 3 3 3 1 1 1 a1 0 0 i 1 1 1 a = 0 1 0 i + s - 2 2 2 a0 0 1 i 3 3 3 i1 0 0 a 1 +s i = 0 1 0 a 2 i0 0 1a 3

  13. Multiplexed Illumination i = intensity under source k i = estimate ofi a = acquired measurement 2 2 2 k 3 3 3 1 1 1 a1 1 0 i +s 1 1 1 a = 0 1 1 i 2 2 2 a1 0 1 i 3 3 3 +s 3 1 i1 -1 1 a 1 2 4 i = 1 1 -1 a 2 i-1 1 1a 3

  14. Demultiplexed Images Multiplexed Illumination

  15. Theory of Multiplexed Illumination +s -1 W W t -1 2 W Trace[] s i = a Variance( ) = N i Minimize i = intensities under individual sources a = acquired measurements For each pixel W a = i

  16. Optimal Multiplexing Codes Based on Hadamard Codes Spectroscopy, X-ray astronomy: Harwit & Sloane 1979 Minimize -1 W W t Trace[] Schechner, Nayar, Belhumeur Multiplexed Illumination

  17. Optimal Multiplexing Codes Based on Hadamard Codes Spectroscopy, X-ray astronomy: Harwit & Sloane 1979 Solution W source On/Off w =1 or 0 1 1 1 0 1 0 0 m,s 1 1 0 1 0 0 1 1 0 1 0 0 1 1 1’s N-1 N+1 0 1 0 0 1 1 1 2 2 1 0 0 1 1 1 0 ~ half the sources are On 0 0 1 1 1 0 1 0’s 0 1 1 1 0 1 0 Schechner, Nayar, Belhumeur Multiplexed Illumination

  18. SNR multiplex 1 1 1 -1 1 -1 -1 = SNR 1 1 -1 1 -1 -1 1 -1 single W = 1 -1 1 -1 -1 1 1 1 -1 1 -1 -1 1 1 1 4 1 -1 -1 1 1 1 -1 -1 -1 1 1 1 -1 1 -1 1 1 1 -1 1 -1 ~ +1/ ~ N N N 2 2 Easy to Invert Schechner, Nayar, Belhumeur Multiplexed Illumination

  19. standard demultiplexed = 2 N Fixed acquisition time T SNR SNR multiplex single Schechner, Nayar, Belhumeur Multiplexed Illumination

  20. standard demultiplexed 2 = 2 N N Fixed acquisition time T SNR SNR multiplex single Fixed high quality SNR multiplex demultiplexed standard T T = multiplex single standard Schechner, Nayar, Belhumeur Multiplexed Illumination

  21. standard demultiplexed 2 = 2 N N standard N N 0.6 = N 3 multiplex single single Fixed acquisition time T SNR SNR multiplex single Fixed high quality SNR multiplex demultiplexed standard T T = multiplex single Fixed acquisition time T single demultiplexed Fixed high quality SNR multiplex

  22. Setup Scalable, Simple, Flexible projector Schechner, Nayar, Belhumeur Multiplexed Illumination

  23. Illumination Patterns Multiplexed Illumination Single-Source Illumination Schechner, Nayar, Belhumeur Multiplexed Illumination

  24. Setup Schechner, Nayar, Belhumeur Multiplexed Illumination

  25. Anistropic illumination -4 -1 -2 -3 -4 -5 -4 2 2 2 2 2 2 2 1 Schechner, Nayar, Belhumeur Multiplexed Illumination

  26. Adaptive directional resolution Schechner, Nayar, Belhumeur Multiplexed Illumination

  27. bright dark dark bright dark dark bright bright bright bright bright dark dark The Quadtree algorithm Schechner, Nayar, Belhumeur Multiplexed Illumination

  28. bright bright bright bright bright bright bright The Quadtree algorithm Schechner, Nayar, Belhumeur Multiplexed Illumination

  29. Raw Images (Diffuse Objects) Multiplexed images Single-source images Schechner, Nayar, Belhumeur Multiplexed Illumination

  30. Single-Source Images Schechner, Nayar, Belhumeur Multiplexed Illumination

  31. Demultiplexed (Decoded) Images Schechner, Nayar, Belhumeur Multiplexed Illumination

  32. Specular Objects Schechner, Nayar, Belhumeur Multiplexed Illumination

  33. Specular Objects Schechner, Nayar, Belhumeur Multiplexed Illumination

  34. Schechner, Nayar, Belhumeur Multiplexed Illumination

  35. Quantitative verification Single-source Demultiplexed +1/ N N 2 Average ratio = 7.97 = Expected ratio = 8.02 noise std [gray-levels] measurement samples

  36. High Definition Specularities Image-Based Rendering Raw Images Schechner, Nayar, Belhumeur Multiplexed Illumination

  37. blue green red intensity intensity l l l l cyan magenta yellow l l Color Schechner, Nayar, Belhumeur Multiplexed Illumination

  38. Saturation Saturation Limit Benefit Highlights Diffuse Object Schechner, Nayar, Belhumeur Multiplexed Illumination

  39. Detector (pixel) Light energy E light Electric energy = E elect 50% efficiency E light 2 Camera Noise Model - Revisited Schechner, Nayar, Belhumeur Multiplexed Illumination

  40. e nothing { or Photon (shot) Noise Detector (pixel) Electrons Photon either 50% quantum efficiency Schechner, Nayar, Belhumeur Multiplexed Illumination

  41. { Photons e e e nothing Photon (shot) Noise Electrons either 50% quantum efficiency Schechner, Nayar, Belhumeur Multiplexed Illumination

  42. 2 s 1.5 1 readout noise variance shot noise variance 0.5 0 50 100 150 200 250 I Camera Noise Model - Revisited * Point-Grey Dragonfly, Gain 3.5 dB, 30fps Schechner, Nayar, Belhumeur Multiplexed Illumination

  43. SNR Gain= 8.02 N=255 9 8 Dalsa 1M75 SNR Gain 4 Pixelink Redlake MotionPro PtGrey Dragonfly 2 PCO Sensican 1 0 saturation limit 20% 40% 60% 80% Schechner, Nayar, Belhumeur Multiplexed Illumination

  44. Conclusions Multiplexing: Higher quality Very simple, yet flexible setup Yoav Schechner Shree Nayar Peter Belhumeur

More Related