1 / 29

Structure from Motion: Unknown Correspondence Solutions

Learn how to combine EM, MCMC, and nonlinear LS techniques to recover structure and motion with unknown correspondence in Structure from Motion problems. Explore the challenges and innovative solutions for handling this complex scenario.

wind
Download Presentation

Structure from Motion: Unknown Correspondence Solutions

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Structure from Motionwith Unknown Correspondence(aka: How to combine EM, MCMC, Nonlinear LS!)Frank Dellaert, Steve Seitz, Chuck Thorpe, and Sebastian Thrun slides courtesy of Frank Dellaert, adapted by Steve Seitz

  2. Traditionally: 2 Problems • Correspondence • Compute 3D positions, camera motion

  3. The Correspondence Problem

  4. The Structure from Motion Problem • Find the most likely structure and motion 

  5. Non-linear Least-Squares (Rick Szeliski’s lecture) = mi’ mi jik xj Structure From Motion Image i’ Image i uik

  6. Question How to recover structure and motion with unknown correspondence? Aside: related subproblems • known correspondence, unknown structure, motion • structure from motion • known motion, unknown correspondence, structure • stereo • known structure, unknown correspondence, motion • 3D registration, ICP

  7. Without Correspondence:Try Tracking Features Lucas-Kanade Tracker (figures from Tommasini et. al. 98)

  8. New Idea: try all correspondences Likelihood: P(; U, J) structure + motion image features correspondence Marginalize over J: P(; U) = J P(; U, J)

  9. Combinatorial Explosion • 3 images, 4 features: 4!3=13,824 • 5 images, 30 features: 30!5=1.3131e+162 • (number of stars:1e+20, atoms: 1e+79) • Computing P( ; U, J)is intractable !

  10. EM for Correspondence

  11. Expectation Maximization

  12. E-Step: Soft Correspondences P(J | U,t) View 1 View 2 View 3

  13. M-Step: SFM • Compute expected SFM solution • Problem: this requires solving an exponential number of SFM problems • If we assume correspondences jik are independent, we can simplify...

  14. vA=p1Au1+p2Au2+p3Au3+p4Au4 B A C D Virtual Measurements 1 2 B A C 4 D 3 View 2

  15. mi’ mi xj M-Step: SFM one SFM problem on virtual measurements Image i’ Image i vij jik

  16. Structure from Motion without Correspondence via EM: In each image, calculate the n2 “soft correspondences”

  17. Key Issue: Calculating Weights 1 2 B A C 4 D 3

  18. Weights = Marginal Probabilities Calculating Weights

  19. Mutual Exclusion

  20. Monte Carlo EM

  21. Monte Carlo Approximation

  22. P( ) = -------------- P( ) Sampling Assignments using the Metropolis Algorithm (1953)

  23. Swap Proposals (inefficient) a b c d e f

  24. ‘Chain Flipping’ Proposals

  25. Efficient Sampling (Example)

  26. MCMC to deal with mutex a b c d e f

  27. Input Images • Manual measurements • Random order

  28. Results: Cube

  29. Soft Correspondences ui scene feature xj EM iterations

More Related