3D reconstruction from uncalibrated images

1. 3D reconstruction from uncalibrated images 2003. 12. 2 Young Ki Baik Visual Information Processing Lab. School of Radio Science & Communication Engineering. Hong-Ik University

2. Contents • Introduction • Overview • Feature Extraction and Matching • Relating Image • Camera model / Projection matrix • Fundamental matrix • Initial reconstruction • Initial projection matrix

3. Contents • Auto-calibration • Motivation • The Absolute Dual Quadric method • Result • Future works • Critical motion sequence • Dense matching • 3D model building

4. 3D reconstruction from uncalibrated images • Overview Image Sequence Feature Extraction/ Matching Relating Image Projective Reconstruction Auto-Calibration Dense Matching 3D Model Building

5. Feature Extraction / Matching • Feature Extraction • Harris corner detector • OpenCV C.Harris and M.Stephens, “A combined corner and edge detector”, Fourth Alvey Vision conference, 1988

6. Feature Extraction / Matching • Matching • Optical flow • Initial matching • OpenCV • Template matching ( SAD ) • Near point matching

7. Relating images • Camera model

8. Relating images • Camera model

9. Relating images • Camera model

10. Relating images • Camera model • Non-square pixels and skew If mx and my pixels per unit distance in x and y directions and skew, Pixel

11. Relating images • Fundamental matrix F

12. Relating images • Fundamental matrix F • RANSAC • Normalized 8-Point algorithm (MVG p.265)

13. Initial reconstruction • Initial reconstruction • Triangulation method • Multiple View Geometry in computer vision • Hartly and Zisserman (p.297) World center

14. X x F Initial reconstruction • Initial projection matrix • Projection matrix • Get P matrix from initial world point X • Bundle adjustment • SparseLM

15. 3D reconstruction • Affine reconstruction(Calibration)

16. 3D reconstruction • Affine reconstruction(Calibration)

17. 3D reconstruction • Metric reconstruction Cholesky decomposition

18. 3D reconstruction • Metric reconstruction

19. Auto-calibration • Motivation • Avoid explicit calibration procedure • Complex procedure • Need for calibration object • Need to maintain calibration • Allow flexible acquisition • No prior calibration necessary • Possibility to vary intrinsic parameters

20. Auto-calibration • Projection matrix

21. Auto-calibration • Projection matrix

22. Auto-calibration • Constraints on intrinsic parameters • Constant parameter • Fixed camera K1 = K2 = … • Known parameter • Rectangular pixel : s = 0 • Square pixel : s = 0, fx = fy • Principle point known : ( ux , uy ) = image center

23. Auto-calibration • Constraints on intrinsic parameters • Constant parameter • Faugeras et al. ECCV’92, Hartley’93, Triggs’97 • Kruppa equations, A stratified solution, Quadric Method • Pollefeys et al. PAMI’98 … • Absolute dual quadric method • Known parameter • Heyden&Astrom CVPR’97 • Pollefeys et al. ICCV’98 …

24. Auto-calibration • The Absolute Dual Quadric method   *

25. Auto-calibration • The Absolute Dual Quadric method • Constant parameter

26. Auto-calibration • The Absolute Dual Quadric method • Constant parameter

27. Auto-calibration • Result • Affine reconstruction

28. Auto-calibration • Result • Metric reconstruction

29. Auto-calibration • Result ( Internal parameter K ) • Rig • Calibration (Vanishing point) • Quadric method

30. Future works • Critical Motion Sequence(CMS) • Critical Motion Sequences have more than one potential absolute conic satisfying all constraints • Pure translation / Pure rotation • Planar motion / Obital motion • Possible to derive classification of CMS • Sturm(CVPR´97), Kahl(ICCV´99)

31. Future works • Dense matching (Stereo vision) • Dense matching through the epipolar line • Conventional stereo • Rectangulr subregioning / Sensus stereo • Cooperative stereo algorithm • Graph cut /PDE-based dense matching

32. Future works • 3D model building • Volumetric model • Super resolution • Plenoptic modeling and rendering