210 likes | 268 Views
Image-based walkthroughs from partial and incremental scene reconstructions. Sudipta N. Sinha Microsoft Research, Redmond http://research.microsoft.com. Kumar Srijan Syed Ahsan Ishtiaque C. V. Jawahar Center for Visual Information Technology, IIIT-Hyderabad http://cvit.iiit.ac.in.
E N D
Image-based walkthroughs from partial and incremental scene reconstructions Sudipta N. Sinha Microsoft Research, Redmond http://research.microsoft.com Kumar Srijan SyedAhsanIshtiaque C. V. Jawahar Center for Visual Information Technology, IIIT-Hyderabad http://cvit.iiit.ac.in
Problem • Efficiently organize and browse these huge image collections? • Keep Incorporating an incoming stream of images into an existing framework?
Related Work • World-Wide Media Exchange (WWMX) • PhotoCompas • Realityflythrough • Aspen Movie Map • Photowalker • Sea of Images • Google Streetview • Photo Tourism
Photo Tourism Input Images Computing correspondences For each pair, estimate an F-matrix and refine matches Detect Features in each image Match keypoints between each pair of images Chain pairwise matches into tracks Incremental SfM Select a good initial pair to seed reconstruction Add new images and triangulate new points Bundle adjust Full Scene Reconstruction Snavely et. al, Photo Tourism: Exploring image collections in 3D
Bottlenecks and Issues • Global scene reconstruction via incremental structure from motion (Sfm) • Sensitivity to the choice of the initial pair • Cascading of errors • O(N4) in the worst case Snavely et. al, Photo Tourism: Exploring image collections in 3D
Bottlenecks and Issues • Timing Breakdown Full Scene Reconstruction for Trafalgar Square dataset with 8000 images took > 50 days Snavely et. al, Photo Tourism: Exploring image collections in 3D
Our approach Independent Partial Scene Reconstructions instead of Global Scene Reconstruction • “ In a walkthrough, users primarily observe near by overlapping images.” • Advantages: • Robustness to errors in incremental SfM module • Worst case linear running time • Scalable • Incremental
Partial Reconstructions Compute partial Reconstructions Compute Matches Refine Matches Incorrect Match Correct Match Image Match Standard SfM
User interface and navigation Sample image Input images Verified neighbors Partial reconstruction Visualization Interface
Global vs. Partial • Global : Allows transition to any image • Partial : Allows transition to a limited number of overlapping images • A -> B implies B -> A A A B B
Incremental insertion Geometric Verification Match Compute Partial Scene Reconstruction New Image Improve Connectivity
Dataset Golconda Fort, Hyderabad Fort Dataset 5989 images
Results • Courtyard Dataset with 687 images • Initialized with 200 images • Added 487 image one by one • Largest CC of 674 images.
Conclusion • Image navigation system based on partial reconstructions can effectively be used to navigate through large collections of images. • Robustness to errors • Able incorporate more images as they become available.
Future Work • Complete automation • Download images directly from the internet • Add into the framework
Acknowledgements • “Photo tourism: Exploring photo collections in 3D“ • Noah Snavely, Cornell University • Steven M. Seitz, University of Washington • Richard Szeliski, Microsoft Research
Acknowledgements • “Visual Word based Location Recognition in 3D models using Distance Augmented Weighting” • Friedrich Fraundorfer, Marc Pollefeys ETH Zürich • Changchang Wu ,Jan-Michael Frahm ,Marc Pollefeys - UNC Chapel Hill
Thank You • Questions