From Digital Photos to Textured 3D Models on Internet

312 Views

Download Presentation
## From Digital Photos to Textured 3D Models on Internet

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**From Digital Photos to Textured 3D Models on Internet**Gang Xu Ritsumeikan University & 3D Media Co., Ltd.**Applications**• Cultural heritage modeling • Car & ship modeling • 3D Survey • Plant Modeling • Game • Virtual Reality • Web design**Outline of This Tutorial**• How can 3D be recovered from 2D? • Mathematical Preparations: Solving Linear Equations and Non-linear Optimization • Camera Model and Projection • Epipolar Geometry Between Images • Linear Algorithm for 2-View SFM • Bundle Adjustment (non-linear optimization)**General Strategy**• Step 1: Find a linear or closed-form solution • Step 2: Non-linear optimization • A linear or closed-form solution via many intermediate representations usually includes large errors. But this solution can serve as the initial guess in the non-linear optimization.**Part I**Mathematical Preparation**Solving Linear Equations (2)**• When n=2 • When n>2, the inverse of A does not exist.**Solving Linear Equations (3)**Best compromise Define Minimize A+ : pseudo-inverse matrix of A**Solving Linear Homogeneous Equations**• When c=0, we cannot use x Define Minimize Under ||x||=1 Solution: Eigen vector associated with the smallest eigen value of**An Example (1)**• Fitting a line to a 2D point set • Plane equation • Minimize**An Example (2)**x0 e1 a e2**Non-Linear Optimization (1)**• Minimize non-linear function**Non-Linear Optimization (2)**• High-dimensional space • No closed-form solution • Multiple local minima • Iterative procedure, starting from an initial guess, assuming that f is differentiable everywhere.**An Example: Bundle Adjustment**Bundle Adjustment: minimizing difference between observations and back-projections**3 Common Used Algorithms**• Gradient Method (1st-order derivative) • Newton Method (2nd-orde derivative) • Levenberg-Marquartd Method = Gradient+Newton, used in 3DM Modeler, 3DM Stitcher and 3DM Calibrator**Part II**Camera and Projection**Pinhole Camera**• Pinhole Camera lens center Image**Projection Formula**y Y y (X,Y,Z) Focal length Z Principal point focus Mirror image Real image**Projection Matrix**Homogeneous Coordinates**u**Principal point x (u0,v0) v y Image Coordinate Systems Intrinsic parameters Principal point approximately at the image center A: Intrinsic matrix**3D Coordinate Transform**Z’ X’ Y’ Object coordinate system Rotation R Translation t Z X Y Camera coordinate system**From Object Coordinates to Pixel Coordinates**Z’ X’ u Y’ v F**What Is Unknown?**• Intrinsic Parameters: focal length f • Extrinsic Parameters: Rotation matrix R, translation vector t**Rotation Representations**r/||r|| ||r|| • 3×3 Rotation MatrixR • Satisfying • There are only 3 degrees of freedom. • A rotation can be represented by a 3-vector r, with ||r|| representing the rotation amount, and r/||r|| representing the rotation axis. • Others: Euler Angles, Roll-Pitch-Yaw, Quarternion**Part III**Epipolar Geometry, Essential Matrix, Fundamental Matrix and Homography Matrix**Epipolar Geometry Between Two Images**Epipolar lines Epipolar lines Epipolar plane epipoles F F’**Epipolar Equation**F,F’,x,x’ are coplanar. x x’ F F’ t**Essential Matrix**E: Essential Matrix**Fundamental Matrix**F: Fundamental matrix**Degenerate Cases Lead to Homography**• Case 1: t=0 (no translation, only rotation. Panorama- 3DM Stitcher) • Case 2: planar scene (3DM Calibrator) • In both cases: the image can be transformed into another by a homography.**Determining E, F and H Matricesfrom Point Matches**• E, F and H can only be determined up to scale. • F (and E) can be linearly determined from 8 or more point matches. • H can be linearly determined from 4 or more point matches.**Part VI**A Linear Algorithm for 2-View SFM**Determining Translation from Essential Matrix**• Assuming that the intrinsic matrix A is available • First compute Essential matrix • Determining t (up to a scale) from • There are two solutions t and -t**Determining Rotation**R • We can consider the problem as determining R from 3 pairs of vectors before and after the rotation**Determining Rotation from n Pairs of Vectors**R • Given (pi,pi’), i=1,…,n (n>=2), • Define and minimize • There is a linear solution using SVD. … …**F**F’ R,t Determining 3D coordinates • Given R, t, x and x’, determine unknown scales s and s’ such that the two lines “meet” in the 3D space.**F**F’ R,t In Front of Both Cameras • s>0, s’>0 in front of both cameras • s>0, s’<0 • s<0, s’>0 • s<0, s’<0**Part V**Bundle Adjustment**Coordinate System**• All space points and camera positions & camera poses are defined in a common coordinate system.**Match Matrix**• Match Matrix w(i,j)=1, if i-th point appears in j-th image; otherwise w(i,j)=0 j 1 2 3 … i 0 1 1 1 1 2 . . 1 1 0**Parameters to Determine**• The parameters to determine are • (Xi,Yi,Zi), i=1,…,n. Totally 3n • (tj,rj,fj), j=1,…,m. Totally 7m • Altogether 3n+7m**Initial Guess**• LM method needs good initial guesses. • They can be obtained using the previous linear algorithm. • Needs to transform all data into the common coordinate system.**References**• Olivier Faugeras: 3D Computer Vision: A Geometry Viewpoint, MIT Press, 1993 • Gang Xu and Zhengyou Zhang: Epipolar Geometry in Stereo, Motion and Object Recognition, Kluwer, 1996 • Richard Hartley and Andrew Zisserman: Multiple View Geometry, Cambridge Univ Press, 2000