from digital photos to textured 3d models on internet
Download
Skip this Video
Download Presentation
From Digital Photos to Textured 3D Models on Internet

Loading in 2 Seconds...

play fullscreen
1 / 46

PPT - PowerPoint PPT Presentation


  • 291 Views
  • Uploaded on

From Digital Photos to Textured 3D Models on Internet Gang Xu Ritsumeikan University & 3D Media Co., Ltd. On-site Demo 3DM Modeler 3DM Stitcher 3DM Calibrator 3DM Viewer Applications Cultural heritage modeling Car & ship modeling 3D Survey Plant Modeling Game Virtual Reality Web design

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'PPT' - jana


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
from digital photos to textured 3d models on internet

From Digital Photos to Textured 3D Models on Internet

Gang Xu

Ritsumeikan University &

3D Media Co., Ltd.

applications
Applications
  • Cultural heritage modeling
  • Car & ship modeling
  • 3D Survey
  • Plant Modeling
  • Game
  • Virtual Reality
  • Web design
outline of this tutorial
Outline of This Tutorial
  • How can 3D be recovered from 2D?
  • Mathematical Preparations: Solving Linear Equations and Non-linear Optimization
  • Camera Model and Projection
  • Epipolar Geometry Between Images
  • Linear Algorithm for 2-View SFM
  • Bundle Adjustment (non-linear optimization)
general strategy
General Strategy
  • Step 1: Find a linear or closed-form solution
  • Step 2: Non-linear optimization
  • A linear or closed-form solution via many intermediate representations usually includes large errors. But this solution can serve as the initial guess in the non-linear optimization.
part i

Part I

Mathematical Preparation

solving linear equations 2
Solving Linear Equations (2)
  • When n=2
  • When n>2, the inverse of A does not exist.
solving linear equations 3
Solving Linear Equations (3)

Best

compromise

Define

Minimize

A+ : pseudo-inverse matrix of A

solving linear homogeneous equations
Solving Linear Homogeneous Equations
  • When c=0, we cannot use

x

Define

Minimize

Under ||x||=1

Solution: Eigen vector associated with the smallest eigen value

of

an example 1
An Example (1)
  • Fitting a line to a 2D point set
  • Plane equation
  • Minimize
non linear optimization 1
Non-Linear Optimization (1)
  • Minimize non-linear function
non linear optimization 2
Non-Linear Optimization (2)
  • High-dimensional space
  • No closed-form solution
  • Multiple local minima
  • Iterative procedure, starting from an initial guess, assuming that f is differentiable everywhere.
an example bundle adjustment
An Example: Bundle Adjustment

Bundle Adjustment: minimizing difference between observations and back-projections

3 common used algorithms
3 Common Used Algorithms
  • Gradient Method (1st-order derivative)
  • Newton Method (2nd-orde derivative)
  • Levenberg-Marquartd Method = Gradient+Newton, used in 3DM Modeler, 3DM Stitcher and 3DM Calibrator
part ii

Part II

Camera and Projection

pinhole camera
Pinhole Camera
  • Pinhole Camera

lens center

Image

projection formula
Projection Formula

y

Y

y

(X,Y,Z)

Focal length

Z

Principal

point

focus

Mirror image

Real image

projection matrix
Projection Matrix

Homogeneous Coordinates

image coordinate systems

u

Principal point

x

(u0,v0)

v

y

Image Coordinate Systems

Intrinsic parameters

Principal point approximately at

the image center

A: Intrinsic matrix

3d coordinate transform
3D Coordinate Transform

Z’

X’

Y’

Object coordinate

system

Rotation R

Translation t

Z

X

Y

Camera coordinate system

what is unknown
What Is Unknown?
  • Intrinsic Parameters: focal length f
  • Extrinsic Parameters: Rotation matrix R, translation vector t
rotation representations
Rotation Representations

r/||r||

||r||

  • 3×3 Rotation MatrixR
  • Satisfying
  • There are only 3 degrees of freedom.
  • A rotation can be represented by a 3-vector r, with ||r|| representing the rotation amount, and r/||r|| representing the rotation axis.
  • Others: Euler Angles, Roll-Pitch-Yaw, Quarternion
part iii

Part III

Epipolar Geometry, Essential Matrix, Fundamental Matrix

and Homography Matrix

epipolar geometry between two images
Epipolar Geometry Between Two Images

Epipolar lines

Epipolar lines

Epipolar plane

epipoles

F

F’

epipolar equation
Epipolar Equation

F,F’,x,x’ are coplanar.

x

x’

F

F’

t

essential matrix
Essential Matrix

E: Essential Matrix

fundamental matrix
Fundamental Matrix

F: Fundamental matrix

degenerate cases lead to homography
Degenerate Cases Lead to Homography
  • Case 1: t=0 (no translation, only rotation. Panorama- 3DM Stitcher)
  • Case 2: planar scene (3DM Calibrator)
  • In both cases: the image can be transformed into another by a homography.
determining e f and h matrices from point matches
Determining E, F and H Matricesfrom Point Matches
  • E, F and H can only be determined up to scale.
  • F (and E) can be linearly determined from 8 or more point matches.
  • H can be linearly determined from 4 or more point matches.
part vi

Part VI

A Linear Algorithm

for 2-View SFM

determining translation from essential matrix
Determining Translation from Essential Matrix
  • Assuming that the intrinsic matrix A is available
  • First compute Essential matrix
  • Determining t (up to a scale) from
  • There are two solutions t and -t
determining rotation
Determining Rotation

R

  • We can consider the problem as determining R from 3 pairs of vectors before and after the rotation
determining rotation from n pairs of vectors
Determining Rotation from n Pairs of Vectors

R

  • Given (pi,pi’), i=1,…,n (n>=2),
  • Define and minimize
  • There is a linear solution using SVD.

determining 3d coordinates

F

F’

R,t

Determining 3D coordinates
  • Given R, t, x and x’, determine unknown scales s and s’ such that the two lines “meet” in the 3D space.
in front of both cameras

F

F’

R,t

In Front of Both Cameras
  • s>0, s’>0 in front of both cameras
  • s>0, s’<0
  • s<0, s’>0
  • s<0, s’<0
part v

Part V

Bundle Adjustment

coordinate system
Coordinate System
  • All space points and camera positions & camera poses are defined in a common coordinate system.
match matrix
Match Matrix
  • Match Matrix w(i,j)=1, if i-th point appears in j-th image; otherwise w(i,j)=0

j

1 2 3 …

i

0

1

1

1

1

2

.

.

1

1

0

parameters to determine
Parameters to Determine
  • The parameters to determine are
  • (Xi,Yi,Zi), i=1,…,n. Totally 3n
  • (tj,rj,fj), j=1,…,m. Totally 7m
  • Altogether 3n+7m
initial guess
Initial Guess
  • LM method needs good initial guesses.
  • They can be obtained using the previous linear algorithm.
  • Needs to transform all data into the common coordinate system.
references
References
  • Olivier Faugeras: 3D Computer Vision: A Geometry Viewpoint, MIT Press, 1993
  • Gang Xu and Zhengyou Zhang: Epipolar Geometry in Stereo, Motion and Object Recognition, Kluwer, 1996
  • Richard Hartley and Andrew Zisserman: Multiple View Geometry, Cambridge Univ Press, 2000
ad