Phase Correlation

1 / 49

# Phase Correlation - PowerPoint PPT Presentation

Phase Correlation. Phase Correlation. Take cross correlation. Take inverse Fourier transform.  Location of the impulse function gives the translation amount between the images. Phase Correlation. Computer Vision . Stereo Vision. Coordinate Systems.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about 'Phase Correlation' - bat

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Phase Correlation

Take cross correlation

Take inverse Fourier transform

 Location of the impulse function gives the translation amount between the images

### Computer Vision

Stereo Vision

Coordinate Systems
• Let O be the origin of a 3D coordinate system spanned by the unit vectors i, j, and k orthogonal to each other.

i

P

O

k

j

Coordinate vector

Homogeneous Coordinates

n

H

P

O

Homogeneous coordinates

Coordinate System Changes
• Rotation

where

Exercise: Write the rotation matrix for a 2D coordinate system.

Coordinate System Changes
• Rotation + Translation
Perspective Projection
• Perspective projection equations

3D World Points

• Camera Centers
• Image Points
Multi-View Geometry

Relates

• Camera Orientations
• Camera Parameters
Stereo

scene point

p

p’

image plane

optical center

Three Questions
• Correspondence geometry: Given an image point p in the first view, how does this constrain the position of the corresponding point p’ in the second?
• Camera geometry (motion): Given a set of corresponding image points {pi ↔ p’i}, i=1,…,n, what are the cameras C and C’ for the two views? Or what is the geometric transformation between the views?
• Scene geometry (structure): Given corresponding image points pi ↔ p’i and cameras C, C’, what is the position of the point X in space?

Epipolar Line

p’

Y2

X2

Z2

O2

Epipole

Stereo Constraints

M

Image plane

Y1

p

O1

Z1

X1

Focal plane

P

p

p’

O’

O

From Geometry to Algebra

All vectors shown lie on the same plane.

Matrix form of cross product

a=axi+ayj+azk

a×b=|a||b|sin(η)u

b=bxi+byj+bzk

The Essential Matrix

Essential matrix

Stereo Vision
• Two cameras.
• Known camera positions.
• Recover depth.
Recovering Depth Information

P

Q

P’1

P’2=Q’2

Q’1

O2

O1

Depth can be recovered with two images and triangulation.

disparity

Depth Z

Elevation Zw

A Simple Stereo System

LEFT CAMERA

RIGHT CAMERA

baseline

Right image:

target

Left image:

reference

Zw=0

Stereo View

Right View

Left View

Disparity

Stereo Disparity
• The separation between two matching objects is called the stereo disparity.
Parallel Cameras

P

Z

xl

xr

f

pl

pr

Ol

Or

Disparity:

T

T is the stereo baseline

(xl, yl)

Correlation Approach

LEFT IMAGE

• For Each point (xl, yl) in the left image, define a window centered at the point
Correlation Approach

RIGHT IMAGE

(xl, yl)

• … search its corresponding point within a search region in the right image
Correlation Approach

RIGHT IMAGE

(xr, yr)

dx

(xl, yl)

• … the disparity (dx, dy) is the displacement when the correlation is maximum

epipolar line

epipolar line

epipolar plane

Stereo correspondence
• Epipolar Constraint
• Reduces correspondence problem to 1D search along epipolar lines

For each epipolar line

For each pixel in the left image

Of course, matching single pixels won’t work; so, we match regions around pixels.

Stereo correspondence
• Compare with every pixel on same epipolar line in right image
• Pick pixel with the minimum matching error

?

=

g

f

Most

popular

Comparing Windows

For each window, match to closest window on epipolar line in other image.

Comparing Windows

Minimize

Sum of Squared

Differences

Maximize

Cross correlation

Feature-based correspondence
• Features most commonly used:
• Corners
• Similarity measured in terms of:
• surrounding gray values (SSD, Cross-correlation)
• location
• Edges, Lines
• Similarity measured in terms of:
• orientation
• contrast
• coordinates of edge or line’s midpoint
• length of line

line

corner

structure

Feature-based Approach

LEFT IMAGE

• For each feature in the left image…

line

corner

structure

Feature-based Approach

RIGHT IMAGE

• Search in the right image… the disparity (dx, dy) is the displacement when the similarity measure is maximum
Correspondence Difficulties
• Why is the correspondence problem difficult?
• Some points in each image will have no corresponding points in the other image.

(1) the cameras might have different fields of view.

(2) due to occlusion.

• A stereo system must be able to determine the image parts that should not be matched.
Structured Light
• Structured lighting
• Feature-based methods are not applicable when the objects have smooth surfaces (i.e., sparse disparity maps make surface reconstruction difficult).
• Patterns of light are projected onto the surface of objects, creating interesting points even in regions which would be otherwise smooth.
• Finding and matching such points is simplified by knowing the geometry of the projected patterns.
Stereo results
• Data from University of Tsukuba

Scene

Ground truth

(Seitz)

Results with window correlation

Estimated depth of field

(a fixed-size window)

Ground truth

(Seitz)

Results with better method
• A state of the art method
• Boykov et al., Fast Approximate Energy Minimization via Graph Cuts,
• International Conference on Computer Vision, September 1999.

Ground truth

(Seitz)

W = 3

W = 20

Window size
• Effect of window size
• Better results with adaptive window
• T. Kanade and M. Okutomi,A Stereo Matching Algorithm with an Adaptive Window: Theory and Experiment,, Proc. International Conference on Robotics and Automation, 1991.
• D. Scharstein and R. Szeliski. Stereo matching with nonlinear diffusion. International Journal of Computer Vision, 28(2):155-174, July 1998

(Seitz)

Other constraints
• It is possible to put some constraints.
• For example: smoothness. (Disparity usually doesn’t change too quickly.)

P

Pl

Pr

Yr

p

p

r

l

Yl

Xl

Zl

Zr

fl

fr

Ol

Or

R, T

Xr

Parameters of a Stereo System
• Intrinsic Parameters
• Characterize the transformation from camera to pixel coordinate systems of each camera
• Focal length, image center, aspect ratio
• Extrinsic parameters
• Describe the relative position and orientation of the two cameras
• Rotation matrix R and translation vector T
Applications

First-down line

courtesy of Sportvision

Applications