1 / 43

Multiple View Geometry in Computer Vision

Multiple View Geometry in Computer Vision. Marc Pollefeys Comp 290-089. Multiple View Geometry. A. a. a. c. c. b. b. f(a,b,c) =0. (a,b)  A. (reconstruction). (a,b,c)  (a,b,c). (calibration). (a,b)  c. (transfer). Course objectives.

hhilda
Download Presentation

Multiple View Geometry in Computer Vision

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Multiple View Geometryin Computer Vision Marc Pollefeys Comp 290-089

  2. Multiple View Geometry A a a c c b b f(a,b,c)=0 (a,b) A (reconstruction) (a,b,c) (a,b,c) (calibration) (a,b) c (transfer)

  3. Course objectives • To understand the geometric relations between multiple views of scenes. • To understand the general principles of parameter estimation. • To be able to compute scene and camera properties from real world images using state-of-the-art algorithms.

  4. Relation to other vision/image courses • Focuses on geometric aspects • No image processing • Comp 254: Image Processing an Analysis Mostly orthogonal to this course, complementary • Comp 256: Computer Vision (fall 2003) Will be much broader, based on new book: “Computer Vision: a modern approach” David Forsyth and Jean Ponce

  5. Material Textbook: Multiple View Geometry in Computer Vision by Richard Hartley and Andrew Zisserman Cambridge University Press Alternative book: The Geometry from Multiple Images by Olivier Faugeras and Quan-Tuan Luong MIT Press On-line tutorial: http://www.cs.unc.edu/~marc/tutorial.pdf http://www.cs.unc.edu/~marc/tutorial/

  6. Learning approach • read the relevant chapters of the books and/or reading assignements before the course.   • In the course the material will then be covered in detail and motivated with real world examples and applications.  • Small hands-on assignements will be provided to give students a "feel" of the practical aspects. • Students will also read and present some seminal papers to provide a complementary view on some of the covered topics. • Finally, there will also be a project where students will implement an algorithm or approach using concepts covered by the course.    Grade distribution • Class participation: 20% • Hands-on assignments: 10% • Paper presentation: 10% • Implementation assignment/project: 40% • Final: 20%

  7. Applications • MatchMoving Compute camera motion from video (to register real an virtual object motion)

  8. Applications • 3D modeling

  9. Content • Background: Projective geometry (2D, 3D), Parameter estimation, Algorithm evaluation. • Single View: Camera model, Calibration, Single View Geometry. • Two Views: Epipolar Geometry, 3D reconstruction, Computing F, Computing structure, Plane and homographies. • Three Views: Trifocal Tensor, Computing T. • More Views: N-Linearities, Multiple view reconstruction, Bundle adjustment, auto-calibration, Dynamic SfM, Cheirality, Duality

  10. Multiple View Geometry course schedule(tentative)

  11. Fast Forward! • Quick overview of what is coming…

  12. Background La reproduction interdite (Reproduction Prohibited), 1937, René Magritte.

  13. Projective 2D Geometry • Points, lines & conics • Transformations • Cross-ratio and invariants

  14. Projective 3D Geometry • Points, lines, planes and quadrics • Transformations • П∞, ω∞and Ω ∞

  15. Estimation How to compute a geometric relation from correspondences, e.g. 2D trafo • Linear (normalized), non-linear and Maximum Likelihood Estimation • Robust (RANSAC)

  16. Evaluation and error analysis How good are the results we get • Bounds on performance • Covariance propagation & Monte-Carlo estimation error residual

  17. Single-View Geometry The Cyclops, c. 1914, Odilon Redon

  18. Camera Models Mostly pinhole camera model but also affine cameras, pushbroom camera, …

  19. Camera Calibration • Compute P given (m,M) (normalized) linear, MLE,… • Radial distortion

  20. More Single-View Geometry • Projective cameras and planes, lines, conics and quadrics. • Camera center and camera rotation • Camera calibration and vanishing points, calibrating conic and the IAC

  21. Single View Metrology Antonio Criminisi

  22. Two-View Geometry The Birth of Venus (detail), c. 1485, Sandro Botticelli

  23. Epipolar Geometry Fundamental matrix Essential matrix

  24. Two-View Reconstruction

  25. Epipolar Geometry Computation (normalized) linear: minimal: MLE: RANSAC … and automated two view matching

  26. Rectification Warp images to simplify epipolar geometry

  27. Structure Computation • Points: Linear, optimal, direct optimal • Also lines and vanishing points

  28. Planes and Homographies Relation between plane and H given P and P’ Relation between H and F, H from F, F from H The infinity homography H∞

  29. Three-View Geometry The Birth of Venus (detail), c. 1485, Sandro Botticelli

  30. Trifocal Tensor

  31. Three View Reconstruction • (normalized) linear • minimal (6 points) • MLE (Gold Standard)

  32. Multiple-View Geometry The Birth of Venus (detail), c. 1485, Sandro Botticelli

  33. Multiple View Geometry Quadrifocal tensor 81 parameters, but only 29 DOF!

  34. Multiple View Reconstruction • Affine factorization • Projective factorization

  35. Multiple View Reconstruction • Sequential reconstruction

  36. P1 P2 P3 M U1 U2 W U3 WT V 3xn (in general much larger) 12xm Bundle Adjustment Maximum Likelyhood Estimation for complete structure and motion

  37. U-WV-1WT WT V 11xm 3xn Bundle Adjustment Maximum Likelyhood Estimation for complete structure and motion

  38. Bundle adjustment No bundle adjustment Bundle adjustment needed to avoid drift of virtual object throughout sequence Bundle adjustment (including radial distortion)

  39. * * projection constraints Auto-calibration

  40. Dynamic Structure from Motion

  41. Cheirality Oriented projective geometry • Allows to use fact that points are in front of camera • to recover quasi-affine reconstruction • to determine order for image warping • to determine orientation for rectification • with epipoles in images • etc.

  42. Duality Gives possibility to interchange role of P and X in algorithms

  43. Contact information Marc Pollefeys, Room 205 marc@cs.unc.edu Tel. 962 1845

More Related