Csse463 image recognition day 30
1 / 15

CSSE463: Image Recognition Day 30 - PowerPoint PPT Presentation

  • Uploaded on

CSSE463: Image Recognition Day 30. This week Today: motion vectors and tracking Friday: Project workday. First status report due. Please bring short printed copy to class: what you did what you didn’t and the technical challenges what you will do next week. Questions?.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' CSSE463: Image Recognition Day 30' - zayit

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Csse463 image recognition day 30
CSSE463: Image Recognition Day 30

  • This week

    • Today: motion vectors and tracking

    • Friday: Project workday. First status report due. Please bring short printed copy to class:

      • what you did

      • what you didn’t and the technical challenges

      • what you will do next week.

  • Questions?

What is image flow
What is image flow?

  • Notice that we can take partial derivatives with respect to x, y, and time.

Image flow equations
Image flow equations

  • Goal: to find where each pixel in frame t moves in frame t+Dt

    • E.g. for 2 adjacent frames, Dt = 1

    • That is, Dx, Dy are unknown

  • Assume:

    • Illumination of object doesn’t change

    • Distances of object from camera or lighting don’t change

    • Each small intensity neighborhood can be observed in consecutive frames: f(x,y,t)f(x+Dx, y+Dy, t+Dt) for some Dx, Dy (the correct motion vector).

  • Compute a Taylor-series expansion around a point in (x,y,t) coordinates.

  • Gives edge gradient and temporal gradient

  • Solve for (Dx, Dy)

  • See answers to first quiz question now


  • Assumptions don’t always hold in real-world images.

  • Doesn’t give a unique solution for flow

    • Sometimes motion is ambiguous

    • Look at last question on last quiz.

    • “Live demo”

    • http://www.cs.mcgill.ca/~chundt/coherence/

Aperture problem
Aperture problem

  • When we only see a small part of the image, motion can be ambiguous

  • Lesson: By constraining ourselves to a small neighborhood, we lose the “big picture”

  • Solutions:

    • Assume pixels belong to same object and have same motion

      • Propagate constraints between pixels

        • Can still take many frames to converge

      • Doesn‘t work at occlusions (hidden parts of object)

      • Sensitive to outliers

    • Only track motion of “interesting points”


Interest point tracking
Interest point tracking

  • Idea: corners and other easily-identifiable points can be tracked, and their motion is unambiguous.

  • Technique for calculating a set of interest points:

    • Loop over image:

      • Calculate an interest score for a small neighborhood around each pixel

      • If the score is above threshold, then add it to a point set to be returned.


Stretch break
Stretch break

  • When is your group willing to present?


Interest scores
Interest scores

  • One solution:

    • Take minimum variance of those found in 4 directions:

  • (3x3 shown, but would use bigger window)

  • Another related texture-based operator is based on directions with significant edge gradient.

  • Could also detect corners using Kirsch operators (similar to other edge operators).

Tracking interest points
Tracking interest points

  • Search for them directly using a template.

    • Choose one with highest correlation

    • Just need to search in small neighborhood if frame rate is high enough

    • Shapiro’s Alg. 9.3

Review correlation
Review: Correlation

  • Just the dot product between the template and a neighborhood in the image.

  • Idea: high correlation when the template matches.

  • Problem: always high correlation when matching with bright region

  • Solution: Normalize each region by subtracting mean before taking dot product

  • Matlab libraries exist to extract frames (still images) from video for processing.

Computing trajectories
Computing trajectories

  • Problem: what if unlabeled moving points belong to multiple trajectories?

  • Idea: want to maximize smoothness, defined as change in direction and change of speed.

  • Given two points pt, pt+1, thought to be related, calculate vt = pt+1 – pt. Then:

Average speed compared to geom. mean


Computing trajectories1
Computing trajectories

  • Then the “Greedy-Exchange” algorithm is used to choose which points belong to which trajectory.

    • Operates by creating trajectories and exchanging points if it would increase total smoothness.

    • Local optimum.

Example tracking pedestrians with human models
Example: Tracking Pedestrians with Human Models

  • http://www.umiacs.umd.edu/users/hismail/Ghost_outline.htm

  • Thanks to Thomas Root for link

Another approach
Another approach

  • The Kalman filter is a probabilistic model that combines noisy measurements with the expected trajectory of the object. It works even with occlusion.

  • See http://www.cs.unc.edu/~welch/kalman/ for a start.