710 likes | 729 Views
Active Imaging for 3D Photography. Mitsubishi Electric Research Labs (MERL) Cambridge, MA. Ramesh Raskar. Active Imaging for 3D Photography. Structured Light Scanning Fast Stripping Global-direct Separation Projector Calibration Quadric and Homography Transfer Low Cost Scanning
E N D
Active Imaging for 3D Photography Mitsubishi Electric Research Labs (MERL) Cambridge, MA Ramesh Raskar
Active Imaging for 3D Photography • Structured Light Scanning • Fast Stripping • Global-direct Separation • Projector Calibration • Quadric and Homography Transfer • Low Cost Scanning • Multi-Flash Stereo • Motion • Coded Exposure
Labeling Space Each location receives a unique signal based upon it’s physical location.
Optical Capture • Goal • Find 2D or 3D location of tags • Approach • Each tag has a photosensor • Label space using zones created by special high speed projectors • Find zone in which the tag lies
Inside of Projector Condensing Optics Light Source Focusing Optics Gray code Slide The Gray code pattern
Solution Overview • Use inexpensive solid state projectors to “label space”. • Solid State lighting (LED’s) can be modulated extremely quickly (MHz to GHz). • Each point in space receives a different code sequence. • Inexpensive and small tags receive the codes, and can detect their own location.
Labeling Space Each tag receives a unique signal based upon it’s physical location.
How Labeling Works Light source Screen Optics GrayCode Mask pos=0 pos=15 Light source blink one by one and each position on the screen has different light pattern. 4 light make 4 bit position resolution
Expand to 2D/3D System 2D measurement 3D measurement X2 data X data X data Y data Y data
Key Features • 500 Hz Tracking • Id for each Marker Tag • Capture in Natural Environment • Visually imperceptible tags • Photosensing Tag can be hidden under clothes • Ambient lighting is ok • Unlimited Number of Tags Allowed • Base station and tags only a few 10’s $
i + fraction of activated source elements High Frequency Illumination Pattern surface source camera
participating medium B D A C A : Direct B : Interrelection E C : Subsurface translucent surface D : Volumetric E : Diffusion Direct and Global Illumination surface source P camera
Fast Separation of Direct and Global Images Using High Frequency Illumination Shree K. Nayar Gurunandan G. Krishnan Columbia University Michael D. Grossberg City College of New York Ramesh Raskar MERL
i + fraction of activated source elements High Frequency Illumination Pattern surface source camera
i - + High Frequency Illumination Pattern surface source camera fraction of activated source elements
Separation from Two Images direct global
Diffuse Interreflections Specular Interreflections Diffusion Volumetric Scattering Subsurface Scattering
Direct Global Scene
Mesh direct global Shower Curtain: Diffuser Shadow
Direct Global Shower Curtain: Diffuser
Project Topics • Structured Light Scanning • Fast Stripping • Can you build a scanner using very low cost hardware? • Without full 2D cameras or video projectors? • Global-direct Separation • Can you scan difficult (global effect) using direct/global separation?
Active Imaging for 3D Photography • Structured Light Scanning • Fast Stripping • Global-direct Separation • Projector Calibration • Quadric and Homography Transfer • Low Cost Scanning • Multi-Flash Stereo • Motion • Coded Exposure
Planar projective transfer (homography) • Two images of 3D points on a plane are related by a 3x3 matrix M Screen Defined by 4 or more corresponding pixels i Camera 1 j Camera 2
Planar display surfaceUse homography (A3x3) M j = Ai i j A
Ruled quadrics: hyperboloids of one sheet Degenerate ruled quadrics: cone two planes Curved projective transferQuadric classification Projectively equivalent to sphere: sphere ellipsoid paraboloid hyperboloid of two sheets
Quardic curved shape Displays Planetarium Sim/Viz Center Raskar, vanBaar, Willwacher, Rao‘Quadric Transfer for Immersive Curved Displays’, EuroGraphics 2004
Quadrics For 3D points X on Quadric X Q : 4x4 symmetric matrix, Nine d.o.f Q In general 9 points in 3D define quadric
Parametric Image Transfer X X i j i j Planar Homography Quadric Transfer
Simplified Quadric Image Transfer X 17 param warp x’ x Planar homography: 4 corresponding pixels Quadric transfer: 9 corresponding pixels
Vertex Shader for Quadric Transfer in Cg • vertout main( appin IN, uniform float4x4 modelViewProj, uniform float4 constColor, uniform float3x3 A, uniform float3x3 E, uniform float3 e) { • vertout OUT; • float4 m1 = float4(IN.position.x, IN.position.y, IN.position.z, 1.0f ); • float4 m, mi ; float3 m2,mp; float scale; • m = mul( modelViewProj, m1); • m2.x = m.x/m.w; m2.y = m.y/m.w; m2.z = 1; • scale = mul(m2, mul(E,m2)); • mp = mul(A,m2) + sqrt(scale)*e; • mi.x = m.w * (mp.x)/(mp.z); • mi.y = m.w * (mp.y)/(mp.z); • mi.zw = m.zw; • OUT.position = mi; • OUT.color0 = IN.color0; // Use the original per-vertex color specified • return OUT; • } • (Code in Course Notes)
Project Topics • Structured Light Scanning • Fast Stripping • Can you build a scanner using very low cost hardware?Without full 2D cameras or video projectors? • Global-direct Separation • Can you scan difficult (global effect) using direct/global separation? • Projector Calibration • Quadric and Homography Transfer • Can you calibrate with non-planar surfaces ?
Active Imaging for 3D Photography • Structured Light Scanning • Fast Stripping • Global-direct Separation • Projector Calibration • Quadric and Homography Transfer • Low Cost Scanning • Multi-Flash Stereo • Motion • Coded Exposure
Depth Edges with Multi-Flash Raskar, Tan, Feris, Yu, Turk – ACM SIGGRAPH 2004
Shadow-Free Depth Edges Shadow-Free Bottom Flash Top Flash Left Flash Right Flash Ratio images and directions of epipolar traversal Depth Edges
Result Photo Canny Intensity Edge Detection Our Method
Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris1, Ramesh Raskar2, Longbin Chen1, Karhan Tan3 and Matthew Turk1 1University of California, Santa Barbara 2Mitsubishi Electric Research Labs 3Epson Palo Alto Lab
Sign of Depth Edge Original Ratio Left Ratio Right Signed Edges - - + + (+) Foreground (-) Background
Shadow Width Bottom Flash Image Ratio Image Plot Along Scanline
Qualitative Depth Useful Prior Information for Stereo !