From Digital Photos to Textured 3D Models on Internet - PowerPoint PPT Presentation

jana
from digital photos to textured 3d models on internet l.
Skip this Video
Loading SlideShow in 5 Seconds..
From Digital Photos to Textured 3D Models on Internet PowerPoint Presentation
Download Presentation
From Digital Photos to Textured 3D Models on Internet

play fullscreen
1 / 46
Download Presentation
From Digital Photos to Textured 3D Models on Internet
312 Views
Download Presentation

From Digital Photos to Textured 3D Models on Internet

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. From Digital Photos to Textured 3D Models on Internet Gang Xu Ritsumeikan University & 3D Media Co., Ltd.

  2. On-site Demo3DM Modeler3DM Stitcher3DM Calibrator3DM Viewer

  3. Applications • Cultural heritage modeling • Car & ship modeling • 3D Survey • Plant Modeling • Game • Virtual Reality • Web design

  4. Outline of This Tutorial • How can 3D be recovered from 2D? • Mathematical Preparations: Solving Linear Equations and Non-linear Optimization • Camera Model and Projection • Epipolar Geometry Between Images • Linear Algorithm for 2-View SFM • Bundle Adjustment (non-linear optimization)

  5. General Strategy • Step 1: Find a linear or closed-form solution • Step 2: Non-linear optimization • A linear or closed-form solution via many intermediate representations usually includes large errors. But this solution can serve as the initial guess in the non-linear optimization.

  6. Part I Mathematical Preparation

  7. Solving Linear Equations (1)

  8. Solving Linear Equations (2) • When n=2 • When n>2, the inverse of A does not exist.

  9. Solving Linear Equations (3) Best compromise Define Minimize A+ : pseudo-inverse matrix of A

  10. Solving Linear Homogeneous Equations • When c=0, we cannot use x Define Minimize Under ||x||=1 Solution: Eigen vector associated with the smallest eigen value of

  11. An Example (1) • Fitting a line to a 2D point set • Plane equation • Minimize

  12. An Example (2) x0 e1 a e2

  13. Non-Linear Optimization (1) • Minimize non-linear function

  14. Non-Linear Optimization (2) • High-dimensional space • No closed-form solution • Multiple local minima • Iterative procedure, starting from an initial guess, assuming that f is differentiable everywhere.

  15. An Example: Bundle Adjustment Bundle Adjustment: minimizing difference between observations and back-projections

  16. 3 Common Used Algorithms • Gradient Method (1st-order derivative) • Newton Method (2nd-orde derivative) • Levenberg-Marquartd Method = Gradient+Newton, used in 3DM Modeler, 3DM Stitcher and 3DM Calibrator

  17. Part II Camera and Projection

  18. Pinhole Camera • Pinhole Camera lens center Image

  19. Projection Formula y Y y (X,Y,Z) Focal length Z Principal point focus Mirror image Real image

  20. Projection Matrix Homogeneous Coordinates

  21. u Principal point x (u0,v0) v y Image Coordinate Systems Intrinsic parameters Principal point approximately at the image center A: Intrinsic matrix

  22. 3D Coordinate Transform Z’ X’ Y’ Object coordinate system Rotation R Translation t Z X Y Camera coordinate system

  23. From Object Coordinates to Pixel Coordinates Z’ X’ u Y’ v F

  24. What Is Unknown? • Intrinsic Parameters: focal length f • Extrinsic Parameters: Rotation matrix R, translation vector t

  25. Rotation Representations r/||r|| ||r|| • 3×3 Rotation MatrixR • Satisfying • There are only 3 degrees of freedom. • A rotation can be represented by a 3-vector r, with ||r|| representing the rotation amount, and r/||r|| representing the rotation axis. • Others: Euler Angles, Roll-Pitch-Yaw, Quarternion

  26. Rodrigue’s Formula

  27. Part III Epipolar Geometry, Essential Matrix, Fundamental Matrix and Homography Matrix

  28. Epipolar Geometry Between Two Images Epipolar lines Epipolar lines Epipolar plane epipoles F F’

  29. Epipolar Equation F,F’,x,x’ are coplanar. x x’ F F’ t

  30. Essential Matrix E: Essential Matrix

  31. Fundamental Matrix F: Fundamental matrix

  32. Degenerate Cases Lead to Homography • Case 1: t=0 (no translation, only rotation. Panorama- 3DM Stitcher) • Case 2: planar scene (3DM Calibrator) • In both cases: the image can be transformed into another by a homography.

  33. Determining E, F and H Matricesfrom Point Matches • E, F and H can only be determined up to scale. • F (and E) can be linearly determined from 8 or more point matches. • H can be linearly determined from 4 or more point matches.

  34. Part VI A Linear Algorithm for 2-View SFM

  35. Determining Translation from Essential Matrix • Assuming that the intrinsic matrix A is available • First compute Essential matrix • Determining t (up to a scale) from • There are two solutions t and -t

  36. Determining Rotation R • We can consider the problem as determining R from 3 pairs of vectors before and after the rotation

  37. Determining Rotation from n Pairs of Vectors R • Given (pi,pi’), i=1,…,n (n>=2), • Define and minimize • There is a linear solution using SVD. … …

  38. F F’ R,t Determining 3D coordinates • Given R, t, x and x’, determine unknown scales s and s’ such that the two lines “meet” in the 3D space.

  39. F F’ R,t In Front of Both Cameras • s>0, s’>0 in front of both cameras • s>0, s’<0 • s<0, s’>0 • s<0, s’<0

  40. Part V Bundle Adjustment

  41. Coordinate System • All space points and camera positions & camera poses are defined in a common coordinate system.

  42. Match Matrix • Match Matrix w(i,j)=1, if i-th point appears in j-th image; otherwise w(i,j)=0 j 1 2 3 … i 0 1 1 1 1 2 . . 1 1 0

  43. Cost Function

  44. Parameters to Determine • The parameters to determine are • (Xi,Yi,Zi), i=1,…,n. Totally 3n • (tj,rj,fj), j=1,…,m. Totally 7m • Altogether 3n+7m

  45. Initial Guess • LM method needs good initial guesses. • They can be obtained using the previous linear algorithm. • Needs to transform all data into the common coordinate system.

  46. References • Olivier Faugeras: 3D Computer Vision: A Geometry Viewpoint, MIT Press, 1993 • Gang Xu and Zhengyou Zhang: Epipolar Geometry in Stereo, Motion and Object Recognition, Kluwer, 1996 • Richard Hartley and Andrew Zisserman: Multiple View Geometry, Cambridge Univ Press, 2000