# From Digital Photos to Textured 3D Models on Internet - PowerPoint PPT Presentation

1 / 46

From Digital Photos to Textured 3D Models on Internet Gang Xu Ritsumeikan University & 3D Media Co., Ltd. On-site Demo 3DM Modeler 3DM Stitcher 3DM Calibrator 3DM Viewer Applications Cultural heritage modeling Car & ship modeling 3D Survey Plant Modeling Game Virtual Reality Web design

## Related searches for From Digital Photos to Textured 3D Models on Internet

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

From Digital Photos to Textured 3D Models on Internet

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

## From Digital Photos to Textured 3D Models on Internet

Gang Xu

Ritsumeikan University &

3D Media Co., Ltd.

## On-site Demo3DM Modeler3DM Stitcher3DM Calibrator3DM Viewer

### Applications

• Cultural heritage modeling

• Car & ship modeling

• 3D Survey

• Plant Modeling

• Game

• Virtual Reality

• Web design

### Outline of This Tutorial

• How can 3D be recovered from 2D?

• Mathematical Preparations: Solving Linear Equations and Non-linear Optimization

• Camera Model and Projection

• Epipolar Geometry Between Images

• Linear Algorithm for 2-View SFM

### General Strategy

• Step 1: Find a linear or closed-form solution

• Step 2: Non-linear optimization

• A linear or closed-form solution via many intermediate representations usually includes large errors. But this solution can serve as the initial guess in the non-linear optimization.

## Part I

Mathematical Preparation

### Solving Linear Equations (2)

• When n=2

• When n>2, the inverse of A does not exist.

### Solving Linear Equations (3)

Best

compromise

Define

Minimize

A+ : pseudo-inverse matrix of A

### Solving Linear Homogeneous Equations

• When c=0, we cannot use

x

Define

Minimize

Under ||x||=1

Solution: Eigen vector associated with the smallest eigen value

of

### An Example (1)

• Fitting a line to a 2D point set

• Plane equation

• Minimize

x0

e1

a

e2

### Non-Linear Optimization (1)

• Minimize non-linear function

### Non-Linear Optimization (2)

• High-dimensional space

• No closed-form solution

• Multiple local minima

• Iterative procedure, starting from an initial guess, assuming that f is differentiable everywhere.

Bundle Adjustment: minimizing difference between observations and back-projections

### 3 Common Used Algorithms

• Newton Method (2nd-orde derivative)

• Levenberg-Marquartd Method = Gradient+Newton, used in 3DM Modeler, 3DM Stitcher and 3DM Calibrator

## Part II

Camera and Projection

### Pinhole Camera

• Pinhole Camera

lens center

Image

y

Y

y

(X,Y,Z)

Focal length

Z

Principal

point

focus

Mirror image

Real image

### Projection Matrix

Homogeneous Coordinates

u

Principal point

x

(u0,v0)

v

y

### Image Coordinate Systems

Intrinsic parameters

Principal point approximately at

the image center

A: Intrinsic matrix

### 3D Coordinate Transform

Z’

X’

Y’

Object coordinate

system

Rotation R

Translation t

Z

X

Y

Camera coordinate system

Z’

X’

u

Y’

v

F

### What Is Unknown?

• Intrinsic Parameters: focal length f

• Extrinsic Parameters: Rotation matrix R, translation vector t

### Rotation Representations

r/||r||

||r||

• 3×3 Rotation MatrixR

• Satisfying

• There are only 3 degrees of freedom.

• A rotation can be represented by a 3-vector r, with ||r|| representing the rotation amount, and r/||r|| representing the rotation axis.

• Others: Euler Angles, Roll-Pitch-Yaw, Quarternion

## Part III

Epipolar Geometry, Essential Matrix, Fundamental Matrix

and Homography Matrix

Epipolar lines

Epipolar lines

Epipolar plane

epipoles

F

F’

### Epipolar Equation

F,F’,x,x’ are coplanar.

x

x’

F

F’

t

### Essential Matrix

E: Essential Matrix

### Fundamental Matrix

F: Fundamental matrix

### Degenerate Cases Lead to Homography

• Case 1: t=0 (no translation, only rotation. Panorama- 3DM Stitcher)

• Case 2: planar scene (3DM Calibrator)

• In both cases: the image can be transformed into another by a homography.

### Determining E, F and H Matricesfrom Point Matches

• E, F and H can only be determined up to scale.

• F (and E) can be linearly determined from 8 or more point matches.

• H can be linearly determined from 4 or more point matches.

## Part VI

A Linear Algorithm

for 2-View SFM

### Determining Translation from Essential Matrix

• Assuming that the intrinsic matrix A is available

• First compute Essential matrix

• Determining t (up to a scale) from

• There are two solutions t and -t

### Determining Rotation

R

• We can consider the problem as determining R from 3 pairs of vectors before and after the rotation

### Determining Rotation from n Pairs of Vectors

R

• Given (pi,pi’), i=1,…,n (n>=2),

• Define and minimize

• There is a linear solution using SVD.

F

F’

R,t

### Determining 3D coordinates

• Given R, t, x and x’, determine unknown scales s and s’ such that the two lines “meet” in the 3D space.

F

F’

R,t

### In Front of Both Cameras

• s>0, s’>0 in front of both cameras

• s>0, s’<0

• s<0, s’>0

• s<0, s’<0

## Part V

### Coordinate System

• All space points and camera positions & camera poses are defined in a common coordinate system.

### Match Matrix

• Match Matrix w(i,j)=1, if i-th point appears in j-th image; otherwise w(i,j)=0

j

1 2 3 …

i

0

1

1

1

1

2

.

.

1

1

0

### Parameters to Determine

• The parameters to determine are

• (Xi,Yi,Zi), i=1,…,n. Totally 3n

• (tj,rj,fj), j=1,…,m. Totally 7m

• Altogether 3n+7m

### Initial Guess

• LM method needs good initial guesses.

• They can be obtained using the previous linear algorithm.

• Needs to transform all data into the common coordinate system.

### References

• Olivier Faugeras: 3D Computer Vision: A Geometry Viewpoint, MIT Press, 1993

• Gang Xu and Zhengyou Zhang: Epipolar Geometry in Stereo, Motion and Object Recognition, Kluwer, 1996

• Richard Hartley and Andrew Zisserman: Multiple View Geometry, Cambridge Univ Press, 2000