CV: 3D to 2D mathematics

1 / 33

# CV: 3D to 2D mathematics - PowerPoint PPT Presentation

CV: 3D to 2D mathematics. Perspective transformation; camera calibration; stereo computation; and more. Roadmap of topics. Review perspective transformation Camera calibration Stereo methods Structured light methods Depth-from-focus Shape-from-shading. Review coordinate systems.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about 'CV: 3D to 2D mathematics' - jarrett

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### CV: 3D to 2D mathematics

Perspective transformation;

camera calibration;

stereo computation;

and more

MSU CSE 240 Fall 2003 Stockman

• Review perspective transformation
• Camera calibration
• Stereo methods
• Structured light methods
• Depth-from-focus

MSU CSE 240 Fall 2003 Stockman

Review coordinate systems

Camera or sensor D

Camera or sensor C

Object or model M

World or global W

MSU CSE 240 Fall 2003 Stockman

Convenient notation for points and transformations

This point P has 2 real coordinates in the image

This point P has 3 real world coordinates in coordinate system W

This transformation maps each point in the real world W to a point in the image I

MSU CSE 240 Fall 2003 Stockman

### Current goal

Develop the theory in terms of modules (components) so that concepts are understood and can be put into practical application

MSU CSE 240 Fall 2003 Stockman

Perspective transformation

Camera origin is center of projection, not lens

X and Y are scaled by the ratio of focal length to depth Z

MSU CSE 240 Fall 2003 Stockman

In next homework & project
• fit camera model to image with jig
• jig has known precise 3D coordinates
• examine accuracy of camera model
• use camera model to do graphics
• use two camera models to compute depth from stereo

MSU CSE 240 Fall 2003 Stockman

Notes on perspective trans.
• 3D world scaled according to ratio of depth to focal length
• scaling formulas are in terms of real numbers with the same units

e.g. mm in the 3D world and

mm in the image plane

• real image coordinates must be further scaled to pixel row and column
• entire 3D ray images to the same 2D point

MSU CSE 240 Fall 2003 Stockman

Camera matrix C transforms 3D real world point into image row and column using 11 parameters

MSU CSE 240 Fall 2003 Stockman

The 11 parameters Cij model
• internal camera parameters:

focal length f

ratio of pixel height and width

any shear due to sensor chip alignment

• external orientation parameters:

rotation of camera frame relative to world frame

translation of camera frame relative to world

The 11 parameters of this model are NOT independent.

Radial distortion is not linear and is not modeled.

MSU CSE 240 Fall 2003 Stockman

Camera matrix via least squares

Minimize the residuals in the image plane. Get 2 equations for each pair ((r, c), (x, y, z))

MSU CSE 240 Fall 2003 Stockman

2 equations for each pair

Known 3D points

Here, (u, v) is the point in the image where 3D point (x,y,z) is projected. The 11 unknowns d jk form the camera matrix.

Known image points

Camera parameters

MSU CSE 240 Fall 2003 Stockman

2n linear equations from n pairs ((u,v) (x,y,z))

Standard linear algebra problem; easily solved in Matlab or by using a linear algebra package.

Often, package replaces b’s with the residuals.

MSU CSE 240 Fall 2003 Stockman

Use a jig for calibration
• Jig has known set of points
• Measure points in world system W or use the jig to define W
• Take image with camera and determine 2D points

Get pairings ((r, c) (x, y, z))

MSU CSE 240 Fall 2003 Stockman

Example calibration data

# # IMAGE: g1view1.ras# # INPUT DATA | OUTPUT DATA# |Point Image 2-D (U,V) 3-D Coordinates (X,Y,Z) | 2-D Fit Data Residuals X Y | A 95.00 336.00 0.00 0.00 0.00 | 94.53 337.89 | 0.47 -1.89 B 0.00 6.00 0.00 | | C 11.00 6.00 0.00 | | D 592.00 368.00 11.00 0.00 0.00 | 592.21 368.36 | -0.21 -0.36 N 501.00 363.00 9.00 0.00 0.00 | 501.16 362.78 | -0.16 0.22 O 467.00 279.00 8.25 0.00 -1.81 | 468.35 281.09 | -1.35 -2.09 P 224.00 266.00 2.75 0.00 -1.81 | 224.06 266.43 | -0.06 -0.43# CALIBRATION MATRIX 44.84 29.80 -5.504 94.53 2.518 42.24 40.79 337.9 -0.0006832 0.06489 -0.01027 1.000

MSU CSE 240 Fall 2003 Stockman

3D points on jig

Dimensions in inches

MSU CSE 240 Fall 2003 Stockman

Jig set in workspace

Mapping is established between 3D points (x, y, z) and image points (u, v)

MSU CSE 240 Fall 2003 Stockman

Other jigs used at MSU
• frame with wires and beads placed in car instead of the driver seat (to do stereo measurements of car driver)
• frame with wires and beads as big as a harp to calibrate space for people walking (up to 6 cameras, persons wear tight body suit with reflecting disks, cameras compute 3D motion trajectory)

MSU CSE 240 Fall 2003 Stockman

Least squares set up

A

X = B

11 x 1

=

2n x 1

2n x 11

MSU CSE 240 Fall 2003 Stockman

Least squares abstraction

MSU CSE 240 Fall 2003 Stockman

Justify the form of camera matrix
• Another sequence of slides
• Rotation, scaling, shear in 3D real world as a 3x3 (or 4x4) matrix
• Projection to real 2D image as 4x4 matrix
• Scaling real image coordinates to [r, c] coordinates as 4x4 matrix
• Combine them all into one 4x4 matrix

MSU CSE 240 Fall 2003 Stockman

### Other mathematical models

Two camera stereo

MSU CSE 240 Fall 2003 Stockman

Baseline stereo: carefully aligned cameras

MSU CSE 240 Fall 2003 Stockman

MSU CSE 240 Fall 2003 Stockman

2 calibrated cameras view the same 3D point at (r1,c1)(r2,c2)

MSU CSE 240 Fall 2003 Stockman

Compute closest approach of the two rays: use center point V

Shortest line segment between rays

MSU CSE 240 Fall 2003 Stockman

Solve for the endpoints of the connector

Scaler mult. Fix book

MSU CSE 240 Fall 2003 Stockman

Correspondence problem: more difficult aspect

MSU CSE 240 Fall 2003 Stockman

Correspondence problem is difficult
• Can use interest points and cross correlation
• Can limit search to epipolar line
• Can use symbolic matching (Ch 11) to determine corresponding points (called structural stereopsis)
• apparently humans don’t need it

MSU CSE 240 Fall 2003 Stockman

Epipolar constraint

With aligned cameras, search for corresponding point is 1D along corresponding row of other camera.

MSU CSE 240 Fall 2003 Stockman

Epipolar constraint for non baseline stereo computation

Need to know relative orientation of cameras C1 and C2

If cameras are not aligned, a 1D search can still be determined for the corresponding point. P1, C1, C2 determine a plane that cuts image I2 in a line: P2 will be on that line.

MSU CSE 240 Fall 2003 Stockman

Measuring driver body position

4 cameras were used to measure driver position and posture while driving: 2mm accuracy achieved

MSU CSE 240 Fall 2003 Stockman