video event recognition multilevel pyramid matching n.
Download
Skip this Video
Download Presentation
Video Event Recognition: Multilevel Pyramid Matching

Loading in 2 Seconds...

play fullscreen
1 / 16

Video Event Recognition: Multilevel Pyramid Matching - PowerPoint PPT Presentation


  • 99 Views
  • Uploaded on

Video Event Recognition: Multilevel Pyramid Matching. Dong Xu and Shih-Fu Chang Digital Video and Multimedia Lab Department of Electrical Engineering Columbia University http://www.ntu.edu.sg/home/dongxu dongxu@ntu.edu.sg *Courtesy to Eric Zavesky for preparing for the slides.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Video Event Recognition: Multilevel Pyramid Matching' - fayola


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
video event recognition multilevel pyramid matching
Video Event Recognition:Multilevel Pyramid Matching
  • Dong Xu and Shih-Fu Chang
  • Digital Video and Multimedia Lab
  • Department of Electrical Engineering
  • Columbia University
  • http://www.ntu.edu.sg/home/dongxu
  • dongxu@ntu.edu.sg
  • *Courtesy to Eric Zavesky for preparing for the slides
video event recognition problem
Video Event Recognition: Problem
  • Online video search and video indexing
  • Events characterized by an evolution of scenes, objects and actions over time
  • 56 events are defined in LSCOM

Airplane Flying

Car Exiting

video event recognition challenges
Video Event Recognition: Challenges
  • Geometric and photometric variances
  • Clutter background
  • Complex camera motion and object motion
event recognition object tracking

Object Detection & Localization

Inference

Tracking

“Airplane Landing”

Event Recognition: Object Tracking
  • Detect interest object, track over time, and model spatio-temporal dynamics
  • Hard to detect events without explicit object motion, such as Riot

?

event recognition key frame based matching

Keyframe

Feature

Similarity

18%

15%

50%

...

...

Event Recognition: Key-Frame based Matching
  • Only key-frame is used for matching.
  • Low-level feature extraction, compare to other frames, overall decision on matching
event recognition multi level pyramid matching

feature extraction

concept detectors

EMD

distance

a

...

...

X

Event Recognition: Multi-level Pyramid Matching

multi-level pyramid matching

content representation low level features

edge directionhistogram

σ

Gabortexture

σ

σ

μ

γ

μ

γ

μ

γ

grid colormoment

Content Representation: Low-level Features
content representation mid level semantic concept scores

Image Database

+

-

Content Representation: Mid-level Semantic Concept Scores

Concept Detectors

  • Train detectors on low-level features
  • Mid-level semantic concept feature is more robust
  • Developed and released 374 semantic concept detectors
earth mover s distance emd approach
Earth Mover’s Distance (EMD): Approach

SupplierP is with agiven amount of goods

ReceiverQis with a

given limited capacity

dij

1

1/2

1/2

Weights:Solved by linear programming

  • Temporal shift:a frame at the beginning of P can be mapped to a frame at the end of Q
  • Scale variations: a frame from P can be mapped to multiple frames in Q
multi level pyramid matching motivations
Multi-level Pyramid Matching: Motivations
  • One Clip = several subclips(stages of event evolution)
  • No prior knowledge about the number of stages in an event
  • Videos of the same event may include only a subset of stages

Solution: Multi-level pyramid matching in temporal domain

slide11

Multi-level Pyramid Matching: Algorithm

Smoke

Fire

  • Temporally Constrained Hierarchical Agglomerative Clustering

Level-2

Level-2

Level-1

  • Alignment of different subclips (Level-1 as an example)

Level-1

Level-0

Level-0

EMD Distance

Matrix between

Sub-clips

Integer-value

Alignment

Level-2

Level-2

Level-1

Level-1

  • Fusion of information from different levels.

Smoke

Fire

experiments keyframe based feature performance
Experiments: Keyframe based feature performance

Evaluation Metric: Average Precision

Dataset: TRECVID2005

slide15

Experiments:

Benefits of multi-level pyramid fusion

slide16

Video Event Recognition: Conclusions

  • Single-level EMD outperforms key-frame based method. Multi-level Pyramid Matching further improves event detection accuracy.
  • First systematic study of diverse visual event recognition in the unconstrained broadcast news domain.