From digital photos to textured 3d models on internet
Download
1 / 46

PPT - PowerPoint PPT Presentation


  • 282 Views
  • Updated On :

From Digital Photos to Textured 3D Models on Internet Gang Xu Ritsumeikan University & 3D Media Co., Ltd. On-site Demo 3DM Modeler 3DM Stitcher 3DM Calibrator 3DM Viewer Applications Cultural heritage modeling Car & ship modeling 3D Survey Plant Modeling Game Virtual Reality Web design

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'PPT' - jana


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
From digital photos to textured 3d models on internet l.jpg

From Digital Photos to Textured 3D Models on Internet

Gang Xu

Ritsumeikan University &

3D Media Co., Ltd.


On site demo 3dm modeler 3dm stitcher 3dm calibrator 3dm viewer l.jpg

On-site Demo3DM Modeler3DM Stitcher3DM Calibrator3DM Viewer


Applications l.jpg
Applications

  • Cultural heritage modeling

  • Car & ship modeling

  • 3D Survey

  • Plant Modeling

  • Game

  • Virtual Reality

  • Web design


Outline of this tutorial l.jpg
Outline of This Tutorial

  • How can 3D be recovered from 2D?

  • Mathematical Preparations: Solving Linear Equations and Non-linear Optimization

  • Camera Model and Projection

  • Epipolar Geometry Between Images

  • Linear Algorithm for 2-View SFM

  • Bundle Adjustment (non-linear optimization)


General strategy l.jpg
General Strategy

  • Step 1: Find a linear or closed-form solution

  • Step 2: Non-linear optimization

  • A linear or closed-form solution via many intermediate representations usually includes large errors. But this solution can serve as the initial guess in the non-linear optimization.


Part i l.jpg

Part I

Mathematical Preparation



Solving linear equations 2 l.jpg
Solving Linear Equations (2)

  • When n=2

  • When n>2, the inverse of A does not exist.


Solving linear equations 3 l.jpg
Solving Linear Equations (3)

Best

compromise

Define

Minimize

A+ : pseudo-inverse matrix of A


Solving linear homogeneous equations l.jpg
Solving Linear Homogeneous Equations

  • When c=0, we cannot use

x

Define

Minimize

Under ||x||=1

Solution: Eigen vector associated with the smallest eigen value

of


An example 1 l.jpg
An Example (1)

  • Fitting a line to a 2D point set

  • Plane equation

  • Minimize


An example 2 l.jpg
An Example (2)

x0

e1

a

e2


Non linear optimization 1 l.jpg
Non-Linear Optimization (1)

  • Minimize non-linear function


Non linear optimization 2 l.jpg
Non-Linear Optimization (2)

  • High-dimensional space

  • No closed-form solution

  • Multiple local minima

  • Iterative procedure, starting from an initial guess, assuming that f is differentiable everywhere.


An example bundle adjustment l.jpg
An Example: Bundle Adjustment

Bundle Adjustment: minimizing difference between observations and back-projections


3 common used algorithms l.jpg
3 Common Used Algorithms

  • Gradient Method (1st-order derivative)

  • Newton Method (2nd-orde derivative)

  • Levenberg-Marquartd Method = Gradient+Newton, used in 3DM Modeler, 3DM Stitcher and 3DM Calibrator


Part ii l.jpg

Part II

Camera and Projection


Pinhole camera l.jpg
Pinhole Camera

  • Pinhole Camera

lens center

Image


Projection formula l.jpg
Projection Formula

y

Y

y

(X,Y,Z)

Focal length

Z

Principal

point

focus

Mirror image

Real image


Projection matrix l.jpg
Projection Matrix

Homogeneous Coordinates


Image coordinate systems l.jpg

u

Principal point

x

(u0,v0)

v

y

Image Coordinate Systems

Intrinsic parameters

Principal point approximately at

the image center

A: Intrinsic matrix


3d coordinate transform l.jpg
3D Coordinate Transform

Z’

X’

Y’

Object coordinate

system

Rotation R

Translation t

Z

X

Y

Camera coordinate system


From object coordinates to pixel coordinates l.jpg
From Object Coordinates to Pixel Coordinates

Z’

X’

u

Y’

v

F


What is unknown l.jpg
What Is Unknown?

  • Intrinsic Parameters: focal length f

  • Extrinsic Parameters: Rotation matrix R, translation vector t


Rotation representations l.jpg
Rotation Representations

r/||r||

||r||

  • 3×3 Rotation MatrixR

  • Satisfying

  • There are only 3 degrees of freedom.

  • A rotation can be represented by a 3-vector r, with ||r|| representing the rotation amount, and r/||r|| representing the rotation axis.

  • Others: Euler Angles, Roll-Pitch-Yaw, Quarternion



Part iii l.jpg

Part III

Epipolar Geometry, Essential Matrix, Fundamental Matrix

and Homography Matrix


Epipolar geometry between two images l.jpg
Epipolar Geometry Between Two Images

Epipolar lines

Epipolar lines

Epipolar plane

epipoles

F

F’


Epipolar equation l.jpg
Epipolar Equation

F,F’,x,x’ are coplanar.

x

x’

F

F’

t


Essential matrix l.jpg
Essential Matrix

E: Essential Matrix


Fundamental matrix l.jpg
Fundamental Matrix

F: Fundamental matrix


Degenerate cases lead to homography l.jpg
Degenerate Cases Lead to Homography

  • Case 1: t=0 (no translation, only rotation. Panorama- 3DM Stitcher)

  • Case 2: planar scene (3DM Calibrator)

  • In both cases: the image can be transformed into another by a homography.


Determining e f and h matrices from point matches l.jpg
Determining E, F and H Matricesfrom Point Matches

  • E, F and H can only be determined up to scale.

  • F (and E) can be linearly determined from 8 or more point matches.

  • H can be linearly determined from 4 or more point matches.


Part vi l.jpg

Part VI

A Linear Algorithm

for 2-View SFM


Determining translation from essential matrix l.jpg
Determining Translation from Essential Matrix

  • Assuming that the intrinsic matrix A is available

  • First compute Essential matrix

  • Determining t (up to a scale) from

  • There are two solutions t and -t


Determining rotation l.jpg
Determining Rotation

R

  • We can consider the problem as determining R from 3 pairs of vectors before and after the rotation


Determining rotation from n pairs of vectors l.jpg
Determining Rotation from n Pairs of Vectors

R

  • Given (pi,pi’), i=1,…,n (n>=2),

  • Define and minimize

  • There is a linear solution using SVD.


Determining 3d coordinates l.jpg

F

F’

R,t

Determining 3D coordinates

  • Given R, t, x and x’, determine unknown scales s and s’ such that the two lines “meet” in the 3D space.


In front of both cameras l.jpg

F

F’

R,t

In Front of Both Cameras

  • s>0, s’>0 in front of both cameras

  • s>0, s’<0

  • s<0, s’>0

  • s<0, s’<0


Part v l.jpg

Part V

Bundle Adjustment


Coordinate system l.jpg
Coordinate System

  • All space points and camera positions & camera poses are defined in a common coordinate system.


Match matrix l.jpg
Match Matrix

  • Match Matrix w(i,j)=1, if i-th point appears in j-th image; otherwise w(i,j)=0

j

1 2 3 …

i

0

1

1

1

1

2

.

.

1

1

0



Parameters to determine l.jpg
Parameters to Determine

  • The parameters to determine are

  • (Xi,Yi,Zi), i=1,…,n. Totally 3n

  • (tj,rj,fj), j=1,…,m. Totally 7m

  • Altogether 3n+7m


Initial guess l.jpg
Initial Guess

  • LM method needs good initial guesses.

  • They can be obtained using the previous linear algorithm.

  • Needs to transform all data into the common coordinate system.


References l.jpg
References

  • Olivier Faugeras: 3D Computer Vision: A Geometry Viewpoint, MIT Press, 1993

  • Gang Xu and Zhengyou Zhang: Epipolar Geometry in Stereo, Motion and Object Recognition, Kluwer, 1996

  • Richard Hartley and Andrew Zisserman: Multiple View Geometry, Cambridge Univ Press, 2000


ad