detecting categories in news video using image features n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Detecting Categories in News Video Using Image Features PowerPoint Presentation
Download Presentation
Detecting Categories in News Video Using Image Features

Loading in 2 Seconds...

play fullscreen
1 / 18

Detecting Categories in News Video Using Image Features - PowerPoint PPT Presentation


  • 113 Views
  • Uploaded on

Detecting Categories in News Video Using Image Features. Slav Petrov, Arlo Faria, Pascal Michaillat, Alex Berg, Andreas Stolcke, Dan Klein, Jitendra Malik. GB. SVM. Category correlation. Sequential context. Source combination. MFCC. GMM. 1-best selection. ASR. TFIDF. SVM.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Detecting Categories in News Video Using Image Features' - cody


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
detecting categories in news video using image features

Detecting Categories in News Video Using Image Features

Slav Petrov, Arlo Faria, Pascal Michaillat, Alex Berg, Andreas Stolcke, Dan Klein, Jitendra Malik

system overview

GB

SVM

Category correlation

Sequential context

Source

combination

MFCC

GMM

1-best

selection

ASR

TFIDF

SVM

Feature extraction

Primary systems

Higher-level systems

System Overview

Video

Images

Audio

image features in trecvid 05
IBM:

Color Histogram

Color Correlogram

Color Moments

CMU (local):

Color Histograms (in different color spaces)

Texture Histograms

Columbia (part based model):

Color

Texture

Tsinghua (local and global):

Color Auto-Correlograms

Color Coherence Vectors

Color Histograms

Co-occurence Texture

Wavelet Texture Grid

Edge Histogram Layout

Edge Histograms

Size

Spatial Relation

Color Moments

Edge Histograms

Wavelet Texture

Image Features in TrecVid ’05
exemplars for recognition
Exemplars for Recognition
  • Use exemplars for recognition
  • Compare query image and each exemplar using shape cues

Database of

Exemplars

Query

Image

geometric blur local appearance descriptor

Compute sparse channels from image

Extract a patch in each channel

Idealized

signal

Apply spatially varying

blur and sub-sample

Descriptor is robust to

small affine distortions

[Berg & Malik, CVPR’01]

Geometric Blur (Local Appearance Descriptor)

~

Geometric Blur

Descriptor

gb in practice

Horizontal Channel

Vertical Channel

Increasing Blur

GB in Practice
  • In practice compute discrete blur levels for whole image and sample as needed for each feature location.
comparing images

[Berg, Berg & Malik, CVPR’05]

Comparing Images
  • Sample 200 GB features from edge points
  • Dissimilarity from A to B is

where the Fx are the GB features.

caltech 101 dataset
Caltech 101 Dataset
  • Object Recognition Benchmark
  • 101 Categories:
    • Stereotypical pose
    • Little clutter
    • Objects centered
    • One object per image
caltech 101 results

uses GB features

[Zhang, Berg, Maire & Malik, CVPR’06]

Caltech 101 Results
primal features for svm

……….

0.9

0.1

0.7

0.7

0.8

Primal features for SVM
  • Compare to 50 prototypes from each class
  • Use distances as feature vector for an SVM

Query

..

..

……

..

Prototypes

Feature Vector

svm features interpretation
SVM features interpretation
  • Slices of the Kernel Matrix:
  • Fixed-points in a higher dimensional vector space:

q

ti

tj

tk

ti

tj

q

tk

svm specifics
SVM Specifics
  • SVMlight package
  • Same parameters for all categories:
    • Linear kernel
    • Default regularization parameter
    • Asymmetric cost doubling the weight of positive examples
results 06

Computer-TV-Screen

BestBerkeley-ShapeMedian

Sports

Meeting

Car

Results ’05Berkeley-Shape mAP = 0.38Best ’05 (IBM) mAP = 0.34

Results ’06

mAP = 0.11

limitations
Limitations
  • Several objects per image:
  • Features do not capture:
    • Different Scales
    • Color
conclusions
Conclusions
  • Shape is an important cue for object recognition.
  • System that uses shape features only can have competitive performance.
  • Shape features are orthogonal to features used in the past.
thank you

Thank You!

petrov@eecs.berkeley.edu