computing motion from images n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Computing Motion from Images PowerPoint Presentation
Download Presentation
Computing Motion from Images

Loading in 2 Seconds...

play fullscreen
1 / 60

Computing Motion from Images - PowerPoint PPT Presentation


  • 117 Views
  • Uploaded on

Computing Motion from Images. Chapter 9 of S&S plus otherwork. General topics. Low level change detection Region tracking or matching over time Interpretation of motion MPEG compression Interpretation of scene changes in video Understanding human activites.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Computing Motion from Images' - lilian


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
computing motion from images

Computing Motion from Images

Chapter 9 of S&S plus otherwork.

Stockman MSU Fall 2008

general topics
General topics
  • Low level change detection
  • Region tracking or matching over time
  • Interpretation of motion
  • MPEG compression
  • Interpretation of scene changes in video
  • Understanding human activites

Stockman MSU Fall 2008

motion important to human vision
Motion important to human vision

Stockman MSU Fall 2008

what s moving different cases
What’s moving: different cases

Stockman MSU Fall 2008

image subtraction

Image subtraction

Simple method to remove unchanging background from moving regions.

Stockman MSU Fall 2008

change detection for surveillance
Change detection for surveillance

Stockman MSU Fall 2008

what to do with regions of change
What to do with regions of change?
  • Discard small regions
  • Discard regions of non interesting features
  • Keep track of regions with interesting features
  • Track in future frames from motion plus component features

Stockman MSU Fall 2008

motion field
Motion field

Stockman MSU Fall 2008

foe and foc
FOE and FOC

Will return to use the FOE or FOC or detection of panning to determine what the camera is doing in video tapes.

Stockman MSU Fall 2008

decathlete game
Decathlete game

Cheap camera replaces usual mouse for input

Running speed and jumping of the avatar is controlled by detected motion of the player’s hands.

Stockman MSU Fall 2008

motion detection input device
Motion detection input device

Jumping (hands)

Running (hands)

Stockman MSU Fall 2008

motion analysis controls hurdling event console
Motion analysis controls hurdling event (console)
  • Top left shows video frame of player
  • Middle left shows motion vectors from multiple frames
  • Center shows jumping patterns

Stockman MSU Fall 2008

related work
Related work
  • Motion sensed by crude cameras
  • Person dances/gestures in space
  • System maps movement into music
  • Creative environment?
  • Good exercise room?

Stockman MSU Fall 2008

computing motion vectors from corresponding points

Computing motion vectors from corresponding “points”

High energy neighborhoods are used to define points for matching

Stockman MSU Fall 2008

match points between frames
Match points between frames

Such large motions are unusual. Most systems track small motions.

Stockman MSU Fall 2008

requirements for interest points
Requirements for interest points

Match small neighborhood to small neighborhood. The previous “scene” contains several highly textured neighborhoods.

Stockman MSU Fall 2008

interest minimum directional variance
Interest = minimum directional variance

Used by Hans Moravec in his robot stereo vision system.

Interest points were used for stereo matching.

Stockman MSU Fall 2008

detecting interest points in i1
Detecting interest points in I1

Stockman MSU Fall 2008

match points from i1 in i2
Match points from I1 in I2

Stockman MSU Fall 2008

search for best match of point p1 in nearby window of i2
Search for best match of point P1 in nearby window of I2

For both motion and stereo, we have some constraints on where to search for a matching interest point.

Stockman MSU Fall 2008

motion vectors clustered to show 3 coherent regions
Motion vectors clustered to show 3 coherent regions

All motion vectors are clustered into 3 groups of similar vectors showing motion of 3 independent objects. (Dina Eldin)

Motion coherence: points of same object tend to move in the same way

Stockman MSU Fall 2008

two frames of aerial imagery
Two frames of aerial imagery

Video frame N and N+1 shows slight movement: most pixels are same, just in different locations.

Stockman MSU Fall 2008

can code frame n d with displacments relative to frame n
Can code frame N+d with displacments relative to frame N
  • for each 16 x 16 block in the 2nd image
  • find a closely matching block in the 1st image
  • replace the 16x16 intensities by the location in the 1st image (dX, dY)
  • 256 bytes replaced by 2 bytes!
  • (If blocks differ too much, encode the differences to be added.)

Stockman MSU Fall 2008

frame approximation
Frame approximation

Left is original video frame N+1. Right is set of best image blocks taken from frame N. (Work of Dina Eldin)

Stockman MSU Fall 2008

best matching blocks between video frames n 1 to n motion vectors
Best matching blocks between video frames N+1 to N (motion vectors)

The bulk of the vectors show the true motion of the airplane taking the pictures. The long vectors are incorrect motion vectors, but they do work well for compression of image I2!

Best matches from 2nd to first image shown as vectors overlaid on the 2nd image. (Work by Dina Eldin.)

Stockman MSU Fall 2008

motion coherence provides redundancy for compression

Motion coherence provides redundancy for compression

MPEG “motion compensation” represents motion of 16x16 pixels blocks, NOT objects

Stockman MSU Fall 2008

computing image flow

Computing Image Flow

Stockman MSU Fall 2008

assumptions
Assumptions

Stockman MSU Fall 2008

slide35

IMAGE

FLOW

EQUATION

1 of 2

Stockman MSU Fall 2008

image flow equation 2 of 2
Image flow equation 2 of 2

Stockman MSU Fall 2008

aperture problem
Aperture problem

Stockman MSU Fall 2008

info at corner constrains the flow along both edges
Info at corner constrains the flow along both edges

Solve constraints using contraint propagation or differential equation with boundary conditions.

Stockman MSU Fall 2008

tracking several objects

Tracking several objects

Use assumptions of physics to compute multiple smooth paths.

(work of Sethi and R. Jain)

Stockman MSU Fall 2008

tracking in images over time
Tracking in images over time

Stockman MSU Fall 2008

general constraints from physics
General constraints from physics

Stockman MSU Fall 2008

other possible constraints
Other possible constraints
  • Background statistics stable
  • Object color/texture/shape might change slowly over frames
  • Might have knowledge of objects under survielance
  • Objects appear/disappear at boundary of the frame

Stockman MSU Fall 2008

sethi jain algorithm
Sethi-Jain algorithm

Stockman MSU Fall 2008

total smoothness of m paths
Total smoothness of m paths

Stockman MSU Fall 2008

greedy exchange algorithm
Greedy exchange algorithm

Stockman MSU Fall 2008

example data structure
Example data structure

Total smoothness for trajectories of Figure 9.14

Stockman MSU Fall 2008

example of domain specific tracking vera bakic
Example of domain specific tracking (Vera Bakic)

Tracking eyes and nose of PC user. System presents menu (top). User moves face to position cursor to a particular box (choice). System tracks face movement and moves cursor accordingly: user gets into feedback-control loop.

Stockman MSU Fall 2008

segmentation of videos movies

Segmentation of videos/movies

Segment into scenes, shots, specific actions, etc.

Stockman MSU Fall 2008

types of changes in videos
Types of changes in videos

Stockman MSU Fall 2008

slide54

How do we compute the scene change?

Anchor person scene at left

Scene break

Street scene for news story

From Zhang et al 1993

Stockman MSU Fall 2008

histograms of frames across the scene change
Histograms of frames across the scene change

Histograms at left are from anchor person frames, while histogram at bottom right is from the street frame.

Stockman MSU Fall 2008

heuristics for ignoring zooms
Heuristics for ignoring zooms

Stockman MSU Fall 2008

american sign language example

American sign language example

Stockman MSU Fall 2008

example from yang and ahuja
Example from Yang and Ahuja

Stockman MSU Fall 2008