A Unifying Framework for Depth from Triangulation

1. pixelintensityat xi I1 xi time Optical I2 time time I2 I1 Passive I2 I1 I1 I2 Stereo Active … Spatial domain occlusion Temporal domain occlusion Shape from shading Structured light Laser scanning … xi Time Camera I1 Projector P2 Projectedlaser Spatialmatching vector Camera Spatial domain laser scanning Spatial Stereo Temporal Stereo Spatial Stereo Temporal Stereo Temporal domain laser scanning Spacetime Stereo: A Unifying Framework for Depth from Triangulation James Davis Ravi Ramamoorthi Szymon Rusinkiewicz Honda Research Institute Columbia University Princeton University Old Taxonomy Classifying systems as passive or active has lead to a separation in the literature of triangulation methods which are closely related. Traditional Spatial Stereo Traditional stereo searches for correspondence using a spatial matching window. This window reduces ambiguity when spatial texture is present. Moving Objects Moving objects cause errors in temporal matching that are analogous to errors caused by object discontinuities in spatial matching. Structured Light Structured light methods project a sequence of coded patterns onto objects to determine depth. Treating these patterns as temporal texture rather than as known codes, allows multiple structured light systems to be used together. Temporal Stereo A temporal matching window can be used to establish correspondence when unstructured light variation or other temporal texture is present. New Taxonomy Unifying triangulation methods within the spacetime framework allows new hybrid algorithms to be easily identified. Example Temporal Patterns The temporal texture at an individual pixel will depend on the nature of the lighting variation. Flashlights, shadows and lasers all produce unique texture. Laser Scanning Laser scanners use a single stripe of light to disambiguate correspondence. Treating this light as texture rather than as a Gaussian feature can relax calibration requirements and prevent interreflections from causing ambiguity. Improved Algorithm Temporal stereo is a new hybrid algorithm which allows improved reconstruction of static scenes with unstructured variable lighting. Spacetime Stereo The matching window should not necessarily lie entirely within the spatial or temporal domains. A spatial-temporal matching window may be desirable.