1 / 45

Real-Time 3D Model Acquisition

Real-Time 3D Model Acquisition. Princeton University Stanford University. Szymon Rusinkiewicz Olaf Hall-Holt Marc Levoy. 3D Scanning. Possible Research Goals. Low noise Guaranteed high accuracy High speed Low cost Automatic operation No holes. 3D Model Acquisition Pipeline.

Download Presentation

Real-Time 3D Model Acquisition

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Real-Time 3D Model Acquisition Princeton University Stanford University Szymon Rusinkiewicz Olaf Hall-Holt Marc Levoy

  2. 3D Scanning

  3. Possible Research Goals • Low noise • Guaranteed high accuracy • High speed • Low cost • Automatic operation • No holes

  4. 3D Model Acquisition Pipeline 3D Scanner

  5. View Planning 3D Model Acquisition Pipeline 3D Scanner

  6. View Planning Alignment 3D Model Acquisition Pipeline 3D Scanner

  7. View Planning Alignment Merging 3D Model Acquisition Pipeline 3D Scanner

  8. View Planning Alignment Done? Merging 3D Model Acquisition Pipeline 3D Scanner

  9. View Planning Alignment Done? Merging Display 3D Model Acquisition Pipeline 3D Scanner

  10. 3D Model Acquisition Difficulties • Much (often most) time spent on “last 20%” • Pipeline not optimized for hole-filling • Not sufficient just to speed up scanner –must design pipeline for fast feedback

  11. Real-Time 3D Model Acquisition

  12. Real-Time 3D Model Acquisition Pipeline 3D Scanner View Planning Alignment Human Done? Merging Display

  13. Alignment Merging Real-Time 3D Model Acquisition Pipeline 3D Scanner View Planning Challenge: Real Time Done? Display

  14. Real-Time 3D Model Acquisition Pipeline 3D Scanner View Planning Alignment Part I: Structured-LightTriangulation Done? Merging Display

  15. Real-Time 3D Model Acquisition Pipeline 3D Scanner View Planning Alignment Part II: Fast ICP Done? Merging Display

  16. Real-Time 3D Model Acquisition Pipeline 3D Scanner View Planning Alignment Part III: Voxel Grid Done? Merging Display

  17. Laser Camera Triangulation • Project laser stripe onto object Object

  18. Camera Triangulation • Depth from ray-plane triangulation Object Laser (x,y)

  19. Triangulation • Faster acquisition: project multiple stripes • Correspondence problem: which stripeis which?

  20. Multi-stripe Multi-frame Single-stripe Single-frame Continuum of Triangulation Methods Slow, robust Fast, fragile

  21. Time-Coded Light Patterns • Assign each stripe a unique illumination codeover time [Posdamer 82] Time Space

  22. Illumination history = (WB),(BW),(WB) Code Codes for Moving Scenes • Assign time codesto stripe boundaries • Perform frame-to-frametracking of correspondingboundaries • Propagate illumination history [Hall-Holt & Rusinkiewicz, ICCV 2001]

  23. Designing a Code • Want many “features” to track:lots of black/white edges at each frame • Try to minimize ghosts – WW or BB “boundaries” that can’t be seen directly

  24. Designing a Code 0000 1101 1010 0111 1111 0010 0101 1000 1011 0110 0001 1100 0100 1001 1110 0011 [Hall-Holt & Rusinkiewicz, ICCV 2001]

  25. Implementation • Pipeline: • DLP projector illuminates scene @ 60 Hz. • Synchronized NTSC camera captures video • Pipeline returns range images @ 60 Hz. Project Code Capture Images Find Boundaries Match Boundaries Decode Compute Range

  26. Real-Time 3D Model Acquisition Pipeline 3D Scanner View Planning Alignment Part II: Fast ICP Done? Merging Display

  27. Aligning 3D Data • ICP (Iterative Closest Points): for each point on one scan, minimize distance to closest point on other scan…

  28. Aligning 3D Data • … and iterate to find alignment • Iterated Closest Points (ICP) [Besl & McKay 92]

  29. ICP in the Real-Time Pipeline • Potential problem with ICP: local minima • In this pipeline, scans close together • Very likely to converge to correct (global) minimum • Basic ICP algorithm too slow (~ seconds) • Point-to-plane minimization • Projection-based matching • With these tweaks, running time ~ milliseconds[Rusinkiewicz & Levoy, 3DIM 2001]

  30. Real-Time 3D Model Acquisition Pipeline 3D Scanner View Planning Alignment Part III: Voxel Grid Done? Merging Display

  31. Merging and Rendering • Goal: visualize the model well enoughto be able to see holes • Cannot display all the scanned data – accumulates linearly with time • Standard high-quality merging methods:processing time ~ 1 minute per scan

  32. Merging and Rendering

  33. Merging and Rendering

  34. Merging and Rendering

  35. + Merging and Rendering

  36. Merging and Rendering • Point rendering, using accumulated normals for lighting

  37. Example: Photograph 18 cm.

  38. Result

  39. Postprocessing • Real-time display • Quality/speed tradeoff • Goal: let user evaluate coverage, fill holes • Offline postprocessing for high-quality models • Global registration • High-quality merging (e.g., using VRIP [Curless 96])

  40. Postprocessed Model

  41. Recapturing Alignment

  42. Summary • 3D model acquisition pipeline optimized for obtaining complete, hole-free models • Use human’s time most efficiently • Pieces of pipeline selected for real-time use: • Structured-light scanner for moving objects • Fast ICP variant • Simple grid-based merging, point rendering

  43. Limitations • Prototype noisier than commercial systems • Could be made equivalent with careful engineering • Ultimate limitations on quality: focus, texture • Scan-to-scan ICP not perfect  alignment drift • Due to noise, miscalibration, degenerate geometry • Reduced, but not eliminated, by “anchor scans” • Possibly combine ICP with separate trackers

  44. Future Work • Faster scanning • Better stripe boundary tracking • Multiple cameras, projectors • High-speed cameras, projectors • Application in different contexts • Cart- or shoulder-mounted for digitizing rooms • Infrared for imperceptibility

  45. Acknowledgments • Collaborators: • Li-Wei He • James Davis • Lucas Pereira • Sean Anderson • Sponsors: • Sony • Intel • Interval

More Related