1 / 18

Automatic Registration of Color Images to 3D Geometry

Computer Graphics International 2009. Automatic Registration of Color Images to 3D Geometry. Yunzhen Li and Kok-Lim Low School of Computing National University of Singapore. * Presented by Binh-Son Hua. Problem Statement. Color images from untracked camera. Range images.

ross-woodard
Download Presentation

Automatic Registration of Color Images to 3D Geometry

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Computer Graphics International 2009 Automatic Registration of Color Images to 3D Geometry Yunzhen Li and Kok-Lim LowSchool of ComputingNational University of Singapore * Presented by Binh-Son Hua

  2. Problem Statement Color images from untracked camera Range images . . . Automatically register color images to 3D model 3D model Colored 3D model 2

  3. Motivations • Applications of active range sensing • Manufacturing, cultural heritage modeling, etc. • Photometric properties needed for visually-realistic models • Only some range scanners can capture color • Color may not have required resolution • E.g. for close-up or zoomed-in views of paintings • View-dependent reflection requires many color images from different directions • Therefore, better to capture color separately • However, impractical to manually register color images to 3D geometry 3

  4. Previous Work • Feature-based approaches • Match corresponding features in both color images and 3D model • Can be fully automated • Restricted to certain types of objects • [Stamos & Allen, ICCV 2001], [Liu & Stamos, CVPR 2005] • Statistics-based approaches • Used only if reflected intensities of range sensing light were recorded with range data • Sensing light often not in visible light spectrum • Compute statistical dependence between color images and sensing light intensities • Mutual information, chi-square, cross-correlation • Camera calibrated & tracked, or co-locate with scanner • [Williams et al, 2004], [Hantak & Lastra, 3DPVT 2006] 4

  5. Color images . . . Our Approach Sparse 3D model Multiview geometry reconstruction Registration Color mapping Colored 3D model Detailed scanned 3D model 5

  6. Steps • Data acquisition • Multiview geometry reconstruction • Approximate registration of sparse model to detailed model • Registration refinement • Color mapping 6

  7. 1. Data Acquisition • Range data • Laser range scanner • Color images • Uncalibrated and untracked digital camera • Project special light pattern on large textureless surfaces • Improve image feature detection and MVG reconstruction 7

  8. 2. MVG Reconstruction • Detect and match features in color images • Use SIFT • Compute MVG • Structure-from-motion • Incrementally add a new image and apply sparse bundle adjustment (SBA) • Result is a sparse 3D model • 3D point cloud • Camera parameters 8

  9. 2. MVG Reconstruction • Example sparse 3D model 9

  10. 3. Approximate Registration • To align sparse model with detailed model • Unknown relative scale and pose • Register one image in MVG to 3D model • User input 6 point correspondences • Estimated transformation propagated to other views and 3D points in MVG • Sparse model only approximately aligned to detailed model • Error in user inputs • Error in MVG • Geometric distortion in detailed model 10

  11. 4. Registration Refinement • Need non-rigid alignment of MVG with detailed model • To overcome geometric distortion in range images • Registration refinement • Automatically detect planes in detailed model • Identify 3D points in MVG near the planes • Refine MVG to minimize distance between 3D points and planes • Easily incorporated into sparse bundle adjustment • Better than using ICP algorithm • Two models are treated as rigid shapes • Cannot refine MVG 11

  12. 4. Registration Refinement • Example result Before registration refinement Afterregistration refinement 12

  13. 5. Color Mapping • Colors from different views can be used for view-dependent rendering • View-dependent texture mapping • Surface light field • We simply want to assign a single color to each surface point, but • Simple averaging blurs out details • Different exposures • Occlusions • Depth boundaries • Vignetting and view-dependent reflection 13

  14. 5. Color Mapping • Use weighted blending • Use lower weights near image and depth boundaries • Preserve fine details Without details preservation With details preservation 14

  15. 5. Color Mapping • Smooth color and intensity transitions Without weighted blending With weighted blending 15

  16. Result • Office scene • 30 color images (7 with projected pattern) 16

  17. Conclusion • Achieve accuracies within 3–5 pixels everywhere on each image • Not reliant on detection of any specific type of features in both color images and geometric model • Project light pattern to improve robustness of MVG • Better registration accuracy in face of geometric distortion • Effective color mapping method 17

  18. Acknowledgements • The Photo Tourism team • For sharing part of their code on MVG • Prashast Khandelwal • For contribution to preliminary work • Singapore Ministry of Education • For the funding 18

More Related