1 / 32

Multiple View Reconstruction Class 24

Multiple View Reconstruction Class 24. Multiple View Geometry Comp 290-089 Marc Pollefeys. Content. Background : Projective geometry (2D, 3D), Parameter estimation, Algorithm evaluation. Single View : Camera model, Calibration, Single View Geometry.

rendor
Download Presentation

Multiple View Reconstruction Class 24

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Multiple View ReconstructionClass 24 Multiple View Geometry Comp 290-089 Marc Pollefeys

  2. Content • Background: Projective geometry (2D, 3D), Parameter estimation, Algorithm evaluation. • Single View: Camera model, Calibration, Single View Geometry. • Two Views: Epipolar Geometry, 3D reconstruction, Computing F, Computing structure, Plane and homographies. • Three Views: Trifocal Tensor, Computing T. • More Views: N-Linearities, Self-Calibration,Multi View Reconstruction, Bundle adjustment, Dynamic SfM, Cheirality, Duality

  3. Multi-view computation

  4. practical structure and motion recovery from images • Obtain reliable matches using matching or tracking and 2/3-view relations • Compute initial structure and motion • sequential structure and motion recovery • hierarchical structure and motion recovery • Refine structure and motion • bundle adjustment • Auto-calibrate • Refine metric structure and motion

  5. Sequential structure and motion recovery Initialize structure and motion from 2 views For each additional view • Determine pose • Refine and extend structure

  6. Initial structure and motion Epipolar geometry  Projective calibration compatible with F Yields correct projective camera setup (Faugeras´92,Hartley´92) Obtain structure through triangulation Use reprojection error for minimization Avoid measurements in projective space

  7. Determine pose towards existing structure M 2D-3D 2D-3D mi+1 mi new view 2D-2D Compute Pi+1using robust approach (6-point RANSAC) Extend and refine reconstruction

  8. Non-sequential image collections Problem: Features are lost and reinitialized as new features 3792 points Solution: Match with other close views 4.8im/pt 64 images

  9. Relating to more views • For every view i • Extract features • Compute two view geometry i-1/i and matches • Compute pose using robust algorithm • Refine existing structure • Initialize new structure For every view i Extract features Compute two view geometry i-1/i and matches Compute pose using robust algorithm For all close views k Compute two view geometry k/i and matches Infer new 2D-3D matches and add to list Refine pose using all 2D-3D matches Refine existing structure Initialize new structure Problem: find close views in projective frame

  10. Refining and extending structure • Refining structure • Extending structure Triangulation (Iterative linear) (Hartley&Sturm,CVIU´97) • Initialize motion • Initialize structure • For each additional view • Determine pose of camera • Refine and extend structure • Refine structure and motion

  11. Structure and motion: example Input sequence 7000points 190 images Viewpoint surface mesh calibration demo

  12. ULM demo

  13. Hierarchical structure and motion recovery • Compute 2-view • Compute 3-view • Stitch 3-view reconstructions • Merge and refine reconstruction F T H PM

  14. Stitching 3-view reconstructions Different possibilities 1. Align (P2,P3) with (P’1,P’2) 2. Align X,X’ (and C’C’) 3. Minimize reproj. error 4. MLE (merge)

  15. Refining structure and motion • Minimize reprojection error • Maximum Likelyhood Estimation (if error zero-mean Gaussian noise) • Huge problem but can be solved efficiently (Bundle adjustment)

  16. Non-linear least-squares • Newton iteration • Levenberg-Marquardt • Sparse Levenberg-Marquardt

  17. Newton iteration Jacobian Taylor approximation normal eq.

  18. Levenberg-Marquardt Normal equations Augmented normal equations accept solve again l small ~ Newton (quadratic convergence) l large ~ descent (guaranteed decrease)

  19. Levenberg-Marquardt Requirements for minimization • Function to compute f • Start value P0 • Optionally, function to compute J (but numerical derivation ok too)

  20. Sparse Levenberg-Marquardt • complexity for solving • prohibitive for large problems (100 views 10,000 points ~30,000 unknowns) • Partition parameters • partition A • partition B (only dependent on A and itself) typically A contains camera parameters, and B contains 3D points

  21. Sparse bundle adjustment residuals: normal equations: with

  22. Sparse bundle adjustment normal equations: modified normal equations: solve in two parts:

  23. P1 P2 P3 M U1 U2 W U3 WT V 3xn (in general much larger) 12xm Sparse bundle adjustment Jacobian of has sparse block structure im.pts. view 1 Needed for non-linear minimization

  24. U-WV-1WT WT V 3xn 11xm Sparse bundle adjustment Eliminate dependence of camera/motion parameters on structure parameters Note in general 3n >> 11m Allows much more efficient computations e.g. 100 views,10000 points, solve 1000x1000, not 30000x30000 Often still band diagonal use sparse linear algebra algorithms

  25. Sparse bundle adjustment normal equations: modified normal equations: solve in two parts:

  26. Sparse bundle adjustment • Covariance estimation

  27. Degenerate configurations (H&Z Ch.21) • Camera resectioning • Two views • More views

  28. Camera resectioning • Cameras as points • 2D case – Chasles’ theorem

  29. Ambiguity for 3D cameras Twisted cubic (or less) meeting lin. subspace(s) (degree+dimension<3)

  30. Ambiguous two-view reconstructions Ruled quadric containing both scene points and camera centers  alternative reconstructions exist for which the reconstruction of points located off the quadric are not projectively equivalent • hyperboloid 1s • cone • pair of planes • single plane + 2 points • single line + 2 points

  31. Multiple view reconstructions • Single plane is still a problem • Hartley and others looked at 3 and more view critical configurations, but those are rather exotic and are not a problem in practice.

  32. Next class: Dynamic structure from motion

More Related