3D Laser Stripe Scanner or “A Really Poor Man’s DeltaSphere”

1. 3D Laser Stripe Scanneror “A Really Poor Man’s DeltaSphere” Chad Hantak December 6, 2004

2. Overview • Introduction • Acquisition Device • Calibration • Software Framework • Processing • Limitations • Future Work

3. Introductionor “What’s the project about?” • 3D Scanning System • Similar to DeltaSphere • Range Samples from fixed COP • Acquire sparse depth with color • Acquisition Device • Cheap • Simple to operate

4. DeltaSphere • Dense Depth • Time of Flight Acquisition • Collects Range Samples over 360 degrees of Azimuth and 135 degrees of elevation • Cons • Slow • Scanning • Processing • Expensive

5. Acquisition Device Components • Laser Level • Available at hardware stores • Used for leveling (pictures, shelves, …) • Emits a plane of laser light (Vertical or Horizontal) • Camcorder • Canon GL2 (3 CCD) • Little overkill for this

6. Acquisition Device Construction • Camcorder rotates about fixed point • Laser Level is fixed next to camcorder

7. Depth Range Acquisition“The Theory” • Calibrated Rig • Know laser stripe’s plane equation in camera’s coordinate system • During Rotation • Determine amount of rotation • Find laser’s stripe points in image • Determine where image rays intersect stripe plane • Yields depth samples • At same time acquire color

8. Calibration • Need to know laser stripe plane in camera coordinate system • Record calibration grid with laser stripe activated

9. Bouget’s Matlab Toolkit • Camera Intrinsics • Standard Bouget’s Toolkit • Extras from toolkit • Calibration plane in camera’s coordinate system for each Image

10. Laser Stripe’s Plane • For each image • Find stripe points (user clicks) • Determine world rays • Intersect with grid’s plane for world points • Take all the world points and fit them to a plane (least squares) • Result is laser stripe plane

11. Software Framework • Implemented in C++ • Libraries • OpenCV • Image Processing • http://www.intel.com/research/mrl/research/opencv/ • DirectShow • Interface to Camcorder • http://msdn.microsoft.com/library/default.asp?url=/library/en-us/directshow/htm/introductiontodirectshow.asp

12. Two Executables • depthPanorama • Hook into camcorder • Acquires / processes frames • Saves into shared memory • panoramaView • Reads from shared memory • Displays result to user

13. depthPanorama • 3 Systems • RotationTracker • Estimates the amount of rotation • ColorProcessor • Creates the color panorama • DepthProcessor • Creates the depth panorama

14. RotationTracker • Estimates the amount of rotation between two frames • Uses Lucas-Kanade Point Tracking • Estimate rotation if “enough” points moved • Uses RANSAC to estimate the amount of rotation

15. ColorProcessor • Updates the Color Panorama • Process • Only update after “enough” rotation • Extract and resize left half of frame into panorama’s memory • Right half of frame contains the laser stripe

16. Depth Processor • Updates the Depth Panorama • Process • Find the laser stripe points in the frame (samples every x scan lines) • Turn image points into world rays • Intersect rays with laser stripe plane • For valid intersections, place depth value in depth panorama • For “some” invalid intersections, attempt to interpolate

17. Finding Laser Stripe Image Pointsor “The Hard Part” • Know in the frame where points are (right half) • Extension to ModelCamera system • Process scan line looking for candidate peaks with symmetry • Exploit fact camcorder is just rotating • Determine homography from one frame to the next, warp previous frame, subtract from current frame • Homography is based on amount of rotation between frames • Since laser stripe moves differently it will stand out better in resultant image • There is still noise in the image • Need some image processing • Edge Detection, Erosion, Dilation

18. Turing Image Points into Depth Samples • Know the camcorder intrinsics • Allows image points to be turned into rays into world from camcorder’s COP • Intersect these rays with plane • Splat depth to neighboring depth samples • Associate a confidence level with each depth sample • Perfect; confidence = 1.0 • Interpolated; confidence = 0.75 • Splatted; confidence = 0.6 • When updating if incoming confidence higher, use it

19. No Intersection • Due to noise, image points of stripe may not be correct • World ray may not intersect stripe plane • If neighboring samples are valid, interpolate missing depth value • Lower confidence

20. panoramaView • Simply reads values sent by depthPanorama • Displays texture-mapped quads of depth values • Updates continually from shared memory

21. Results

22. Current Problems • Rotation tracking • Off by a few degrees at end of 360 degree pan (< 10 degrees) • Tweaking LK Tracking parameters • Smooth out the rotation (use LK & correlation to determine when rotation complete) • Stripe point determination • Adjusting image processing parameters • Depth discontinuities • Take them into account when rendering

23. Future Work • Fix current problems • Better laser light / Point detection • Stronger at distance • Want longer range scans • Combine different scans • Introduces new slew of problems

24. References • R. Laganiere, “Programming computer vision applications: A step-by-step guide to the use of the Intel OpenCV library and the Microsoft DirectShow technology”, http://www.site.uottawa.ca/~laganier/tutorial/opencv+directshow/. • V. Popescu, E. Sacks and G. Bahmutov, “Interactive Modeling from Dense Color and Sparse Depth” , 3DPTV, 2004. • S. Sinha and M. Pollefeys, “Towards Calibrating a Pan-Tilt-Zoom Camera Network”, 5th Workshop on Omnidirectional Vision, Camera Networks, and Non-classical Cameras (OMNIVIS), May 16, 2004.