phase correlation l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Phase Correlation PowerPoint Presentation
Download Presentation
Phase Correlation

Loading in 2 Seconds...

play fullscreen
1 / 49

Phase Correlation - PowerPoint PPT Presentation


  • 170 Views
  • Uploaded on

Phase Correlation. Phase Correlation. Take cross correlation. Take inverse Fourier transform.  Location of the impulse function gives the translation amount between the images. Phase Correlation. Computer Vision . Stereo Vision. Coordinate Systems.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Phase Correlation' - bat


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
phase correlation2
Phase Correlation

Take cross correlation

Take inverse Fourier transform

 Location of the impulse function gives the translation amount between the images

computer vision

Computer Vision

Stereo Vision

coordinate systems
Coordinate Systems
  • Let O be the origin of a 3D coordinate system spanned by the unit vectors i, j, and k orthogonal to each other.

i

P

O

k

j

Coordinate vector

homogeneous coordinates
Homogeneous Coordinates

n

H

P

O

Homogeneous coordinates

coordinate system changes8
Coordinate System Changes
  • Rotation

where

Exercise: Write the rotation matrix for a 2D coordinate system.

coordinate system changes9
Coordinate System Changes
  • Rotation + Translation
perspective projection
Perspective Projection
  • Perspective projection equations
multi view geometry

3D World Points

  • Camera Centers
  • Image Points
Multi-View Geometry

Relates

  • Camera Orientations
  • Camera Parameters
stereo
Stereo

scene point

p

p’

image plane

optical center

three questions
Three Questions
  • Correspondence geometry: Given an image point p in the first view, how does this constrain the position of the corresponding point p’ in the second?
  • Camera geometry (motion): Given a set of corresponding image points {pi ↔ p’i}, i=1,…,n, what are the cameras C and C’ for the two views? Or what is the geometric transformation between the views?
  • Scene geometry (structure): Given corresponding image points pi ↔ p’i and cameras C, C’, what is the position of the point X in space?
stereo constraints

Epipolar Line

p’

Y2

X2

Z2

O2

Epipole

Stereo Constraints

M

Image plane

Y1

p

O1

Z1

X1

Focal plane

from geometry to algebra

P

p

p’

O’

O

From Geometry to Algebra

All vectors shown lie on the same plane.

matrix form of cross product
Matrix form of cross product

a=axi+ayj+azk

a×b=|a||b|sin(η)u

b=bxi+byj+bzk

the essential matrix
The Essential Matrix

Essential matrix

stereo vision
Stereo Vision
  • Two cameras.
  • Known camera positions.
  • Recover depth.
recovering depth information
Recovering Depth Information

P

Q

P’1

P’2=Q’2

Q’1

O2

O1

Depth can be recovered with two images and triangulation.

a simple stereo system

disparity

Depth Z

Elevation Zw

A Simple Stereo System

LEFT CAMERA

RIGHT CAMERA

baseline

Right image:

target

Left image:

reference

Zw=0

stereo view
Stereo View

Right View

Left View

Disparity

stereo disparity
Stereo Disparity
  • The separation between two matching objects is called the stereo disparity.
parallel cameras
Parallel Cameras

P

Z

xl

xr

f

pl

pr

Ol

Or

Disparity:

T

T is the stereo baseline

correlation approach

(xl, yl)

Correlation Approach

LEFT IMAGE

  • For Each point (xl, yl) in the left image, define a window centered at the point
correlation approach31
Correlation Approach

RIGHT IMAGE

(xl, yl)

  • … search its corresponding point within a search region in the right image
correlation approach32
Correlation Approach

RIGHT IMAGE

(xr, yr)

dx

(xl, yl)

  • … the disparity (dx, dy) is the displacement when the correlation is maximum
stereo correspondence

epipolar line

epipolar line

epipolar plane

Stereo correspondence
  • Epipolar Constraint
    • Reduces correspondence problem to 1D search along epipolar lines
stereo correspondence34

For each epipolar line

For each pixel in the left image

Of course, matching single pixels won’t work; so, we match regions around pixels.

Stereo correspondence
  • Compare with every pixel on same epipolar line in right image
  • Pick pixel with the minimum matching error
slide35

?

=

g

f

Most

popular

Comparing Windows

For each window, match to closest window on epipolar line in other image.

slide36

Comparing Windows

Minimize

Sum of Squared

Differences

Maximize

Cross correlation

feature based correspondence
Feature-based correspondence
  • Features most commonly used:
    • Corners
      • Similarity measured in terms of:
        • surrounding gray values (SSD, Cross-correlation)
        • location
    • Edges, Lines
      • Similarity measured in terms of:
        • orientation
        • contrast
        • coordinates of edge or line’s midpoint
        • length of line
feature based approach

line

corner

structure

Feature-based Approach

LEFT IMAGE

  • For each feature in the left image…
feature based approach39

line

corner

structure

Feature-based Approach

RIGHT IMAGE

  • Search in the right image… the disparity (dx, dy) is the displacement when the similarity measure is maximum
correspondence difficulties
Correspondence Difficulties
  • Why is the correspondence problem difficult?
    • Some points in each image will have no corresponding points in the other image.

(1) the cameras might have different fields of view.

(2) due to occlusion.

  • A stereo system must be able to determine the image parts that should not be matched.
structured light
Structured Light
  • Structured lighting
    • Feature-based methods are not applicable when the objects have smooth surfaces (i.e., sparse disparity maps make surface reconstruction difficult).
    • Patterns of light are projected onto the surface of objects, creating interesting points even in regions which would be otherwise smooth.
  • Finding and matching such points is simplified by knowing the geometry of the projected patterns.
stereo results
Stereo results
  • Data from University of Tsukuba

Scene

Ground truth

(Seitz)

results with window correlation
Results with window correlation

Estimated depth of field

(a fixed-size window)

Ground truth

(Seitz)

results with better method
Results with better method
  • A state of the art method
    • Boykov et al., Fast Approximate Energy Minimization via Graph Cuts,
    • International Conference on Computer Vision, September 1999.

Ground truth

(Seitz)

window size

W = 3

W = 20

Window size
  • Effect of window size
  • Better results with adaptive window
    • T. Kanade and M. Okutomi,A Stereo Matching Algorithm with an Adaptive Window: Theory and Experiment,, Proc. International Conference on Robotics and Automation, 1991.
    • D. Scharstein and R. Szeliski. Stereo matching with nonlinear diffusion. International Journal of Computer Vision, 28(2):155-174, July 1998

(Seitz)

other constraints
Other constraints
  • It is possible to put some constraints.
  • For example: smoothness. (Disparity usually doesn’t change too quickly.)
parameters of a stereo system

P

Pl

Pr

Yr

p

p

r

l

Yl

Xl

Zl

Zr

fl

fr

Ol

Or

R, T

Xr

Parameters of a Stereo System
  • Intrinsic Parameters
    • Characterize the transformation from camera to pixel coordinate systems of each camera
    • Focal length, image center, aspect ratio
  • Extrinsic parameters
    • Describe the relative position and orientation of the two cameras
    • Rotation matrix R and translation vector T
applications
Applications

First-down line

courtesy of Sportvision

applications49
Applications

Virtual advertising

courtesy of Princeton Video Image