motion computing in image analysis l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Motion Computing in Image Analysis PowerPoint Presentation
Download Presentation
Motion Computing in Image Analysis

Loading in 2 Seconds...

play fullscreen
1 / 36

Motion Computing in Image Analysis - PowerPoint PPT Presentation


  • 119 Views
  • Uploaded on

Motion Computing in Image Analysis. - Mani V Thomas CISC 489/689. Roadmap. Optic Flow Constraint Optic Flow Computation Gradient Based Approach Feature Based Approach Estimation Criterion Block Matching algorithms Conclusion.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Motion Computing in Image Analysis' - juliette


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
motion computing in image analysis

Motion Computing in Image Analysis

- Mani V Thomas

CISC 489/689

roadmap
Roadmap
  • Optic Flow Constraint
  • Optic Flow Computation
    • Gradient Based Approach
    • Feature Based Approach
  • Estimation Criterion
  • Block Matching algorithms
  • Conclusion

Some slides and illustrations are from M. Pollefeys and M. Shah

importance of visual motion
Importance of Visual Motion
  • Apparent motion of objects on the image plane is a strong cue to understand structure and 3D motion
  • Biological visual systems infer properties of the 3D world via motion
  • Two sub-problems of motion
    • Problem of correspondence estimation
      • Which elements of a frame correspond to which elements of the next frame
    • Problem of reconstruction
      • Given the correspondence and the camera’s intrinsic parameters can we infer 3D motion and/or structure

Courtesy: E. Trucco and A. Verri, “Introductory techniques for 3D Computer Vision”

apparent motion
Apparent Motion
  • Apparent motionof objects on the image plane
  • Caution required!!
    • Consider a perfectly uniform sphere that is rotating but no change in the light direction
      • Optic flow is zero
    • Perfectly uniform sphere that is stationary but the light is changing
      • Optic flow exists
  • Hope – apparent motion is very close to the actual motion

Courtesy: E. Trucco and A. Verri, “Introductory techniques for 3D Computer Vision”

optic flow computation
Optic Flow Computation
  • Two strategies for computing motion
    • Differential Methods
      • Spatio temporal derivatives for estimation of flow at every position
      • Multi-scale analysis required if motion not constrained within a small range
        • Dense flow measurements
    • Matching Methods
      • Feature extraction(Image edges, corners)
      • Feature/Block Matching and error minimization
        • Sparse flow measurements

Courtesy: E. Trucco and A. Verri, “Introductory techniques for 3D Computer Vision”

optic flow computation6
Optic Flow Computation
  • Image Brightness Constancy assumption
    • Let E be the image intensity as captured by the camera
    • Using Taylor series to expand E
    • Apparent brightness of moving objects remains constant
optic flow computation7
Optic Flow Computation
  • Image Brightness Constancy assumption
    • Apparent brightness of moving objects remains constant
    • The are the image gradient while the are the components of the motion field

Courtesy: E. Trucco and A. Verri, “Introductory techniques for 3D Computer Vision”

aperture problem
Aperture Problem
  • We can measure
    • Terms that can be measured
    • Terms to be computed
    • Number of equations - 1
  • The component of the motion field that is orthogonal to the spatial image gradient is not constrained by the image brightness constancy assumption
  • Intuitively
    • The component of the flow in the gradient direction is determined
    • The component of the flow parallel to an edge is unknown

Courtesy: E. Trucco and A. Verri, “Introductory techniques for 3D Computer Vision”

roadmap11
Roadmap
  • Optic Flow Constraint
  • Optic Flow Computation
    • Gradient Based Approach
    • Feature Based Approach
  • Estimation Criterion
  • Block Matching algorithms
  • Conclusion

Some slides and illustrations are from M. Pollefeys and M. Shah

optic flow constraint
Optic Flow Constraint
  • How to get more equations for a pixel?
    • Basic idea: impose additional constraints
      • Most common is to assume that the flow field is smooth locally
    • One method: pretend the pixel’s neighbors have the same (u,v)
    • If we use a 5x5 window, that gives us 25 equations per pixel!
lucas kanade optic flow
Lucas-Kanade Optic Flow
  • We now have more equations than unknowns
  • Solve the least squares problem
    • Minimum least squares solution (in d) is given by
    • First proposed by Lucas-Kanade in 1981
    • Summation performed over all the pixels in the window
lucas kanade optic flow14
Lucas-Kanade Optic Flow
  • Lucas-Kanade Optic flow
  • When is the Lucas-Kanade equations solvable
    • ATA should be invertible
    • ATA should not be too small (effects of noise)
      • Eigenvalues of ATA, 1 and 2 should not be small
    • ATA should be well conditioned
      • 1/2 should not be large (1 = larger eigenvalue)
slide15
Edge
  • Gradient is large in magnitude
  • Large 1 but small 2
low texture region
Low texture region
  • Gradients has small magnitude
  • Small 1 and small 2
high texture region
High texture region
  • Gradients are different with large magnitudes
  • Large 1 and large 2
improving the lucas kanade method
Improving the Lucas-Kanade method
  • When our assumptions are violated
    • Brightness constancy is not satisfied
    • The motion is not small
    • A point does not move like its neighbors
  • Iterative Lucas-Kanade Algorithm
    • Estimate velocity at each pixel by solving Lucas-Kanade equations
    • Warp H towards I using the estimated flow field
      • use image warping techniques
    • Repeat until convergence
iterative lucas kanade method

u=1.25 pixels

u=2.5 pixels

u=5 pixels

u=10 pixels

image H

image H

image I

image I

Gaussian pyramid of image H

Gaussian pyramid of image I

Iterative Lucas-Kanade method
iterative lucas kanade method20

warp & upsample

run iterative L-K

.

.

.

image J

image H

image I

image I

Gaussian pyramid of image H

Gaussian pyramid of image I

Iterative Lucas-Kanade method

run iterative L-K

roadmap21
Roadmap
  • Optic Flow Constraint
  • Optic Flow Computation
    • Gradient Based Approach
    • Feature Based Approach
  • Estimation Criterion
  • Block Matching algorithms
  • Conclusion

Some slides and illustrations are from M. Pollefeys and M. Shah

feature based method
Feature Based Method
  • Feature Extraction
    • Maxima in first derivative of the Image
      • Local peak in the first derivative
    • Numerical Approximation
  • Compute the motion parameters from the best bipartite graph
    • Correspondence between the feature points in one image with those in the other

For more information: Ramesh Jain, Rangachar Kasturi, Brian Schunck: Machine Vision 1995 (140 - 159)

roadmap23
Roadmap
  • Optic Flow Constraint
  • Optic Flow Computation
    • Gradient Based Approach
    • Feature Based Approach
  • Estimation Criterion
  • Block Matching algorithms
  • Conclusion

Some slides and illustrations are from M. Pollefeys and M. Shah

estimation criterion
Estimation Criterion
  • Pixel domain Criterion
    • MAE/MSE
    • Lorentzian
    • Correlation
  • Frequency Domain Criterion
    • Cross Correlation
    • Phase Correlation
estimation criterion contd
Estimation Criterion(contd.)
  • Pixel Domain Criterion
    • Estimation criterion aim at minimizing
    • prediction error is sensitive to noise if number of pixels is not large or if region is poorly textured
    • Common choice of estimation criterion
      • Quadratic function is not good since a single large error can bias the estimate of the field
      • Absolute value function is better than the quadratic since cost grows linearly with error
        • Does not require multiplications and is better suited for real-time video encoders
estimation criterion contd26
Estimation Criterion(contd.)
  • A more robust criterion is based on the Lorentzian function
    • Grows slower than |x| for larger errors
  • Similarity measure using Correlation
    • Computationally complex because of the multiplications
    • This criterion requires maximization
    • Usually the normalized Cross correlation is computed

For more details: M. Black, “Robust Incremental Optical Flow”

estimation criterion contd28
Estimation Criterion(contd.)
  • Frequency Domain Criterion
    • Amplitudes of both the FT are independent of z
    • Argument difference depends linearly on translation
      • Global motion is recovered by evaluating the phase difference over a number of frequencies and solving the resulting system of equations
      • In practice, this method will work only for a single object moving across a uniform background
estimation criterion contd29
Estimation Criterion(contd.)
  • Phase Correlation
    • In the case of a single global translation, the correlation surface becomes a Kronecker delta function
    • In practice, there are numerous peaks which correspond to the dominant displacements between the two images
    • The locations are relatively independent to illumination changes
roadmap30
Roadmap
  • Optic Flow Constraint
  • Optic Flow Computation
    • Gradient Based Approach
    • Feature Based Approach
  • Estimation Criterion
  • Block Matching algorithms
  • Conclusion

Some slides and illustrations are from M. Pollefeys and M. Shah

block matching algorithms
Block Matching Algorithms
  • Sparse motion measurements
    • Motion is spatially constant and temporally linear over a rectangular region of support
    • The minimization problem is
      • is an M x N block of pixels with the top-left corner co-ordinate at
block matching algorithms contd

-p

N

-p

N

M

M

p

p

Current Picture

Reference Frame

v

-p

N

(x,y)

(x+u,y+v)

M

-p

p

u

p

Block Matching Algorithms(contd.)
block matching algorithms contd33
Block Matching Algorithms(contd.)
  • Principle of Locality of Reference
  • Block Matching algorithms
    • Exhaustive Search
      • Always finds the “deepest” minimum
      • Computationally very expensive
    • If I x J is the picture resolution and rate is F fps the overall operations in comparing MxN blocks would be
      • This corresponds to 29.89 GOPS for p=15 at 30fps for a 720x480 image (3 operations per pixel of one subtraction, one absolute value and one addition)
block matching algorithms contd34
Block Matching Algorithms(contd.)
  • Logarithmic Search
    • Sub-optimal and may get trapped in a local minima
    • Computationally feasible for real-time video encoders
    • Search Method
      • Divide the search space at [-p/2, -p/2]
      • Search at (0,0) and at 8 major points at the perimeter of the rectangle at [-p/2, -p/2]
      • Using best match position as starting point, search in the eight perimeter points at the half distance window
    • If I x J is the picture resolution and rate is F fps the overall operations in comparing MxN blocks would be
      • This corresponds to 1.03 GOPS for p=15 at 30fps for a 720x480 image

For more information refer the work by Dr. Lai-Man Po and C. K. Cheung (http://www.ee.cityu.edu.hk/~lmpo/publications/index.html)

block matching algorithms contd35
Block Matching Algorithms(contd.)
  • Hierarchical Search
    • Sub-optimal for regions containing detail and increased storage requirements
    • Computationally feasible for real-time video encoders
    • Search method
      • Form several low resolution images by low pass filtering
      • At the lowest resolution perform a sub-optimal search like log search
      • Propagate search vectors to higher resolution images and perform search
    • If I x J is the picture resolution and rate is F fps the overall operations in comparing MxN blocks would be
      • This corresponds to 507.38 MOPS for p=15 at 30fps for a 720x480 image
conclusion
Conclusion
  • Motion estimation
  • Aperture problem
  • Different algorithms to perform motion analysis
    • Lucas-Kanade algorithm
  • Estimation criterion for motion field computation
  • Block Matching Algorithms
  • Computational complexity of motion analysis