1 / 19

Geometry Directed Browser for Personal Photographs

Geometry Directed Browser for Personal Photographs. Center for Visual Information Technology IIIT Hyderabad. Aditya Deshpande , Siddharth Choudhary, P J Narayanan, Kaustav Kundu, Krishna Kumar Singh, Aditya Singh, Apurva Kumar. Digital Photography

Download Presentation

Geometry Directed Browser for Personal Photographs

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Geometry Directed Browser for Personal Photographs Center for Visual Information Technology IIIT Hyderabad Aditya Deshpande, Siddharth Choudhary, P J Narayanan, Kaustav Kundu, Krishna Kumar Singh, Aditya Singh, Apurva Kumar

  2. Digital Photography • - No hard copy - Capture photographs and relive later on display device • Photo-Browsers are tools to view digital photographs. E.g. Windows Photo Viewer, iPhoto, FSpot, KSquirrel etc. • Photo Browsing model has notevolved much. Photo-Browsing We use SfM and other 3D computer vision techniques to provide intuitive Geometry Directed Photo Browsing.

  3. Face Detection & Tagging on Social Networking Sites. • [Zhang et al. MM’03], Automatic annotation of family albums. • [Davis et al. MM’05], Additional contextual data viz. time of capture, geo-tag, indoor/outdoor scene, co-occurring faces. Related Work Above techniques only improve photo-browsing experience of social engagements.

  4. Apart from social engagements, a large chunk of user’s personal photographs consist of tourist places & monuments. • [Snavely et al. IJCV’08, SIGGRAPH’06] (Photosynth)- CPC Storage, local reconstruction to add new camera’s • Choudhary et al., Li et al., Sattler et al., Irschara et al. etc. - Localize new query images w/o exhaustive search. Our Goal We combine SfM-Reconstruction + Localization to provide intuitive browsing of user photos in 3D space of the monument.

  5. Our target platform is an off-the-shelf laptop or a desktop. • User is expected to click around 5-50 photographs for a particular monument. • The system should localize these user photographs in areasonable time. • The system should provide a smooth visualization / transitions of all user photos and ~105 points of the monument. Assumptions

  6. (1) Heavy SfM Reconstruction done offline in the cloud (3) User uploads personal photo’s through a camera / phone System Design (4) System registers user’sphotos to the point cloud and provides 3D visualization. (2) GDBPackage : reconstruction + addnl. information downloaded to local disk

  7. GDBPackage System Block Diagram 1 2 RegistrationModule VisualizationModule Estimated Camera’s User Photos System is divided in two parts : 1. Registration / Localization Module2. Visualization Module

  8. Trivial if photograph is taken from GPS enabled deviceand is geo-tagged! • What if no geo-tag information? • Two Localization Approaches :Image based search in a geo-tagged Image Dataset[Panda et al.] Geo-locate digital heritage site photos.Using structure information in SfM Dataset[Irschara et al. CVPR’09], match to nearby similar images.[Li et al. ECCV’10], visibility prioritized 3D-2D matches. [Sattler et al. ICCV’11, ECCV’12], visual words to find 2D-3D matches. Localizing User Photo’s

  9. [Choudhary et al. ECCV’12]- Triangulate a seed point in the user photograph.- Further 3D-2D search is guided by visibility probabilities.- Find ~20 independent matches.- Use RANSAC to estimate camera parameters. Localization - Choudhary et al. Up Vector View Direction 3D Position Probability Guided 3D-2D correspondence

  10. Data for Localization is stored in GDBPackage : (1) Cover Set (2) Visibility Matrix (3) Bi-Partite Visibility Graph • CPC images need not be stored, data requirements are minimal. • The method is fast and localizes images at the rate of 1sec/photo. Advantages of Localization Method

  11. In some cases the images lack sufficient monument geometryfor localization to work :- Occluded by people.- Noisy images of nearby scenery/smaller monuments.- Zoomed in images of smaller monument structures etc. Non-Localizable Photographs Zoomed In View of Small Structure(Pantheon Dataset) Completely Occludedby People(Colosseum Dataset)

  12. Photographs have time of capture stored in their EXIF-tags. • A non-localized image is placed at a position that is weightedaverage of its immediate known predecessor and immediateknown successor in time. • Similarly, linear interpolation is also done for the view-direction vector to get the complete camera pose. • The above method will not give the exact location, but placing it in temporal neighborhood suffices for display purposes. Non-Localizable Photographs

  13. 3D Viewer • Mouse Navigation • Button Navigation • Add Screenshot • Delete Path • Generate Photo-Tour • 2D Viewer Visualization Module

  14. Initial Mode : 3D Model and small preview (thumbnails) of • user photographs. • Select Mode : Animate to clicked photo and detailed view. • Linear quaternion interpolation of Rotation Matrix for smoothtransitions between images. • Smooth transitions give a feel of the geometric space of the monument. 3D Photo Browser : Geometry Directed Photo-Browsing

  15. User can save the current viewpoint (“Add Screenshots”) • Once a set of viewpoints are saved, he can smoothly animateover viewpoints. (“Generate Photo-Tour / Animate Path”) • User can delete the viewpoints and generate a new photo-tour. • Photo-Tours are a good way to creatively view personalphoto’s taken at a tourist place. 3D Photo Browser : Generating Custom Photo Tours

  16. Results (a) Localization Module (b) Visualization Module

  17. Minimal System Requirements. • Intuitive 3D Visualization of User Photographs. • Pipeline for 3D personal photo-viewing from SfM reconstruction. • Port our system to a mobile phone and have a touch/gesture interface. Conclusion and Future Work 3D Photo-Viewing & Localization App

  18. Thank you. Questions?More Results (a) Hampi Dataset (Stone Chariot) (b) Pantheon Dataset

  19. Platform Details

More Related