1 / 21

Image-based walkthroughs from partial and incremental scene reconstructions

Image-based walkthroughs from partial and incremental scene reconstructions. Sudipta N. Sinha Microsoft Research, Redmond http://research.microsoft.com. Kumar Srijan Syed Ahsan Ishtiaque C. V. Jawahar Center for Visual Information Technology, IIIT-Hyderabad http://cvit.iiit.ac.in.

quincy-hooper
Download Presentation

Image-based walkthroughs from partial and incremental scene reconstructions

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Image-based walkthroughs from partial and incremental scene reconstructions Sudipta N. Sinha Microsoft Research, Redmond http://research.microsoft.com Kumar Srijan SyedAhsanIshtiaque C. V. Jawahar Center for Visual Information Technology, IIIT-Hyderabad http://cvit.iiit.ac.in

  2. Introduction

  3. Problem • Efficiently organize and browse these huge image collections? • Keep Incorporating an incoming stream of images into an existing framework?

  4. Related Work • World-Wide Media Exchange (WWMX) • PhotoCompas • Realityflythrough • Aspen Movie Map • Photowalker • Sea of Images • Google Streetview • Photo Tourism

  5. Photo Tourism Input Images Computing correspondences For each pair, estimate an F-matrix and refine matches Detect Features in each image Match keypoints between each pair of images Chain pairwise matches into tracks Incremental SfM Select a good initial pair to seed reconstruction Add new images and triangulate new points Bundle adjust Full Scene Reconstruction Snavely et. al, Photo Tourism: Exploring image collections in 3D

  6. Bottlenecks and Issues • Global scene reconstruction via incremental structure from motion (Sfm) • Sensitivity to the choice of the initial pair • Cascading of errors • O(N4) in the worst case Snavely et. al, Photo Tourism: Exploring image collections in 3D

  7. Bottlenecks and Issues • Timing Breakdown Full Scene Reconstruction for Trafalgar Square dataset with 8000 images took > 50 days Snavely et. al, Photo Tourism: Exploring image collections in 3D

  8. Our approach Independent Partial Scene Reconstructions instead of Global Scene Reconstruction • “ In a walkthrough, users primarily observe near by overlapping images.” • Advantages: • Robustness to errors in incremental SfM module • Worst case linear running time • Scalable • Incremental

  9. Partial Reconstructions Compute partial Reconstructions Compute Matches Refine Matches Incorrect Match Correct Match Image Match Standard SfM

  10. User interface and navigation Sample image Input images Verified neighbors Partial reconstruction Visualization Interface

  11. Global vs. Partial • Global : Allows transition to any image • Partial : Allows transition to a limited number of overlapping images • A -> B implies B -> A A A B B

  12. Incremental insertion Geometric Verification Match Compute Partial Scene Reconstruction New Image Improve Connectivity

  13. Dataset Golconda Fort, Hyderabad Fort Dataset 5989 images

  14. Results

  15. Results

  16. Results • Courtyard Dataset with 687 images • Initialized with 200 images • Added 487 image one by one • Largest CC of 674 images.

  17. Conclusion • Image navigation system based on partial reconstructions can effectively be used to navigate through large collections of images. • Robustness to errors • Able incorporate more images as they become available.

  18. Future Work • Complete automation • Download images directly from the internet • Add into the framework

  19. Acknowledgements • “Photo tourism: Exploring photo collections in 3D“ • Noah Snavely, Cornell University • Steven M. Seitz, University of Washington • Richard Szeliski, Microsoft Research

  20. Acknowledgements • “Visual Word based Location Recognition in 3D models using Distance Augmented Weighting” • Friedrich Fraundorfer, Marc Pollefeys ETH Zürich • Changchang Wu ,Jan-Michael Frahm ,Marc Pollefeys - UNC Chapel Hill

  21. Thank You • Questions

More Related