Loading in 5 sec....

CSSE463: Image Recognition Day 30PowerPoint Presentation

CSSE463: Image Recognition Day 30

- By
**zayit** - Follow User

- 107 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about ' CSSE463: Image Recognition Day 30' - zayit

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

CSSE463: Image Recognition Day 30

- This week
- Today: motion vectors and tracking
- Friday: Project workday. First status report due. Please bring short printed copy to class:
- what you did
- what you didn’t and the technical challenges
- what you will do next week.

- Questions?

What is image flow?

- Notice that we can take partial derivatives with respect to x, y, and time.

Image flow equations

- Goal: to find where each pixel in frame t moves in frame t+Dt
- E.g. for 2 adjacent frames, Dt = 1
- That is, Dx, Dy are unknown

- Assume:
- Illumination of object doesn’t change
- Distances of object from camera or lighting don’t change
- Each small intensity neighborhood can be observed in consecutive frames: f(x,y,t)f(x+Dx, y+Dy, t+Dt) for some Dx, Dy (the correct motion vector).

- Compute a Taylor-series expansion around a point in (x,y,t) coordinates.
- Gives edge gradient and temporal gradient
- Solve for (Dx, Dy)
- See answers to first quiz question now

Limitations

- Assumptions don’t always hold in real-world images.
- Doesn’t give a unique solution for flow
- Sometimes motion is ambiguous
- Look at last question on last quiz.
- “Live demo”
- http://www.cs.mcgill.ca/~chundt/coherence/

Aperture problem

- When we only see a small part of the image, motion can be ambiguous
- Lesson: By constraining ourselves to a small neighborhood, we lose the “big picture”
- Solutions:
- Assume pixels belong to same object and have same motion
- Propagate constraints between pixels
- Can still take many frames to converge

- Doesn‘t work at occlusions (hidden parts of object)
- Sensitive to outliers

- Propagate constraints between pixels
- Only track motion of “interesting points”

- Assume pixels belong to same object and have same motion

Q1

Interest point tracking

- Idea: corners and other easily-identifiable points can be tracked, and their motion is unambiguous.
- Technique for calculating a set of interest points:
- Loop over image:
- Calculate an interest score for a small neighborhood around each pixel
- If the score is above threshold, then add it to a point set to be returned.

- Loop over image:

Q2

Interest scores

- One solution:
- Take minimum variance of those found in 4 directions:

- (3x3 shown, but would use bigger window)
- Another related texture-based operator is based on directions with significant edge gradient.
- Could also detect corners using Kirsch operators (similar to other edge operators).

Tracking interest points

- Search for them directly using a template.
- Choose one with highest correlation
- Just need to search in small neighborhood if frame rate is high enough
- Shapiro’s Alg. 9.3

Review: Correlation

- Just the dot product between the template and a neighborhood in the image.
- Idea: high correlation when the template matches.
- Problem: always high correlation when matching with bright region
- Solution: Normalize each region by subtracting mean before taking dot product

- Matlab libraries exist to extract frames (still images) from video for processing.

Computing trajectories

- Problem: what if unlabeled moving points belong to multiple trajectories?
- Idea: want to maximize smoothness, defined as change in direction and change of speed.
- Given two points pt, pt+1, thought to be related, calculate vt = pt+1 – pt. Then:

Average speed compared to geom. mean

direction

Computing trajectories

- Then the “Greedy-Exchange” algorithm is used to choose which points belong to which trajectory.
- Operates by creating trajectories and exchanging points if it would increase total smoothness.
- Local optimum.

Example: Tracking Pedestrians with Human Models

- http://www.umiacs.umd.edu/users/hismail/Ghost_outline.htm
- Thanks to Thomas Root for link

Another approach

- The Kalman filter is a probabilistic model that combines noisy measurements with the expected trajectory of the object. It works even with occlusion.
- See http://www.cs.unc.edu/~welch/kalman/ for a start.

Download Presentation

Connecting to Server..