1 / 71

Mitsubishi Electric Research Labs (MERL) Cambridge, MA

Active Imaging for 3D Photography. Mitsubishi Electric Research Labs (MERL) Cambridge, MA. Ramesh Raskar. Active Imaging for 3D Photography. Structured Light Scanning Fast Stripping Global-direct Separation Projector Calibration Quadric and Homography Transfer Low Cost Scanning

whoffman
Download Presentation

Mitsubishi Electric Research Labs (MERL) Cambridge, MA

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Active Imaging for 3D Photography Mitsubishi Electric Research Labs (MERL) Cambridge, MA Ramesh Raskar

  2. Active Imaging for 3D Photography • Structured Light Scanning • Fast Stripping • Global-direct Separation • Projector Calibration • Quadric and Homography Transfer • Low Cost Scanning • Multi-Flash Stereo • Motion • Coded Exposure

  3. Labeling Space Each location receives a unique signal based upon it’s physical location.

  4. High Speed Tracking

  5. Optical Capture • Goal • Find 2D or 3D location of tags • Approach • Each tag has a photosensor • Label space using zones created by special high speed projectors • Find zone in which the tag lies

  6. Inside of Projector Condensing Optics Light Source Focusing Optics Gray code Slide The Gray code pattern

  7. Inside of Tag

  8. Solution Overview • Use inexpensive solid state projectors to “label space”. • Solid State lighting (LED’s) can be modulated extremely quickly (MHz to GHz). • Each point in space receives a different code sequence. • Inexpensive and small tags receive the codes, and can detect their own location.

  9. Labeling Space Each tag receives a unique signal based upon it’s physical location.

  10. How Labeling Works Light source Screen Optics GrayCode Mask pos=0 pos=15 Light source blink one by one and each position on the screen has different light pattern. 4 light make 4 bit position resolution

  11. Expand to 2D/3D System 2D measurement 3D measurement X2 data X data X data Y data Y data

  12. E-Tech Demo

  13. Key Features • 500 Hz Tracking • Id for each Marker Tag • Capture in Natural Environment • Visually imperceptible tags • Photosensing Tag can be hidden under clothes • Ambient lighting is ok • Unlimited Number of Tags Allowed • Base station and tags only a few 10’s $

  14. i + fraction of activated source elements High Frequency Illumination Pattern surface source camera

  15. participating medium B D A C A : Direct B : Interrelection E C : Subsurface translucent surface D : Volumetric E : Diffusion Direct and Global Illumination surface source P camera

  16. Fast Separation of Direct and Global Images Using High Frequency Illumination Shree K. Nayar Gurunandan G. Krishnan Columbia University Michael D. Grossberg City College of New York Ramesh Raskar MERL

  17. i + fraction of activated source elements High Frequency Illumination Pattern surface source camera

  18. i - + High Frequency Illumination Pattern surface source camera fraction of activated source elements

  19. Separation from Two Images direct global

  20. Diffuse Interreflections Specular Interreflections Diffusion Volumetric Scattering Subsurface Scattering

  21. Scene

  22. Direct Global Scene

  23. Mesh direct global Shower Curtain: Diffuser Shadow

  24. Direct Global Shower Curtain: Diffuser

  25. Project Topics • Structured Light Scanning • Fast Stripping • Can you build a scanner using very low cost hardware? • Without full 2D cameras or video projectors? • Global-direct Separation • Can you scan difficult (global effect) using direct/global separation?

  26. Active Imaging for 3D Photography • Structured Light Scanning • Fast Stripping • Global-direct Separation • Projector Calibration • Quadric and Homography Transfer • Low Cost Scanning • Multi-Flash Stereo • Motion • Coded Exposure

  27. Planar projective transfer (homography) • Two images of 3D points on a plane are related by a 3x3 matrix M Screen Defined by 4 or more corresponding pixels i Camera 1 j Camera 2

  28. Planar display surfaceUse homography (A3x3) M j = Ai i j A

  29. Ruled quadrics: hyperboloids of one sheet Degenerate ruled quadrics: cone two planes Curved projective transferQuadric classification Projectively equivalent to sphere: sphere ellipsoid paraboloid hyperboloid of two sheets

  30. Quardic curved shape Displays Planetarium Sim/Viz Center Raskar, vanBaar, Willwacher, Rao‘Quadric Transfer for Immersive Curved Displays’, EuroGraphics 2004

  31. Quadrics For 3D points X on Quadric X Q : 4x4 symmetric matrix, Nine d.o.f Q In general 9 points in 3D define quadric

  32. Parametric Image Transfer X X i j i j Planar Homography Quadric Transfer

  33. Simplified Quadric Image Transfer X 17 param warp x’ x Planar homography: 4 corresponding pixels Quadric transfer: 9 corresponding pixels

  34. Overlap on Quadric Screens

  35. Vertex Shader for Quadric Transfer in Cg • vertout main( appin IN, uniform float4x4 modelViewProj, uniform float4 constColor, uniform float3x3 A, uniform float3x3 E, uniform float3 e) { • vertout OUT; • float4 m1 = float4(IN.position.x, IN.position.y, IN.position.z, 1.0f ); • float4 m, mi ; float3 m2,mp; float scale; • m = mul( modelViewProj, m1); • m2.x = m.x/m.w; m2.y = m.y/m.w; m2.z = 1; • scale = mul(m2, mul(E,m2)); • mp = mul(A,m2) + sqrt(scale)*e; • mi.x = m.w * (mp.x)/(mp.z); • mi.y = m.w * (mp.y)/(mp.z); • mi.zw = m.zw; • OUT.position = mi;  • OUT.color0 = IN.color0; // Use the original per-vertex color specified • return OUT; • } • (Code in Course Notes)

  36. Project Topics • Structured Light Scanning • Fast Stripping • Can you build a scanner using very low cost hardware?Without full 2D cameras or video projectors? • Global-direct Separation • Can you scan difficult (global effect) using direct/global separation? • Projector Calibration • Quadric and Homography Transfer • Can you calibrate with non-planar surfaces ?

  37. Active Imaging for 3D Photography • Structured Light Scanning • Fast Stripping • Global-direct Separation • Projector Calibration • Quadric and Homography Transfer • Low Cost Scanning • Multi-Flash Stereo • Motion • Coded Exposure

  38. Depth Edges with Multi-Flash Raskar, Tan, Feris, Yu, Turk – ACM SIGGRAPH 2004

  39. Shadow-Free Depth Edges Shadow-Free Bottom Flash Top Flash Left Flash Right Flash Ratio images and directions of epipolar traversal Depth Edges

  40. Result Photo Canny Intensity Edge Detection Our Method

  41. Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris1, Ramesh Raskar2, Longbin Chen1, Karhan Tan3 and Matthew Turk1 1University of California, Santa Barbara 2Mitsubishi Electric Research Labs 3Epson Palo Alto Lab

  42. Sign of Depth Edge Original Ratio Left Ratio Right Signed Edges - - + + (+) Foreground (-) Background

  43. Shadow Width Bottom Flash Image Ratio Image Plot Along Scanline

  44. Qualitative Depth Useful Prior Information for Stereo !

More Related