1 / 18

Detecting Categories in News Video Using Image Features

Detecting Categories in News Video Using Image Features. Slav Petrov, Arlo Faria, Pascal Michaillat, Alex Berg, Andreas Stolcke, Dan Klein, Jitendra Malik. GB. SVM. Category correlation. Sequential context. Source combination. MFCC. GMM. 1-best selection. ASR. TFIDF. SVM.

cody
Download Presentation

Detecting Categories in News Video Using Image Features

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Detecting Categories in News Video Using Image Features Slav Petrov, Arlo Faria, Pascal Michaillat, Alex Berg, Andreas Stolcke, Dan Klein, Jitendra Malik

  2. GB SVM Category correlation Sequential context Source combination MFCC GMM 1-best selection ASR TFIDF SVM Feature extraction Primary systems Higher-level systems System Overview Video Images Audio

  3. IBM: Color Histogram Color Correlogram Color Moments CMU (local): Color Histograms (in different color spaces) Texture Histograms Columbia (part based model): Color Texture Tsinghua (local and global): Color Auto-Correlograms Color Coherence Vectors Color Histograms Co-occurence Texture Wavelet Texture Grid Edge Histogram Layout Edge Histograms Size Spatial Relation Color Moments Edge Histograms Wavelet Texture Image Features in TrecVid ’05

  4. Image Features in TrecVid ’05

  5. Exemplars for Recognition • Use exemplars for recognition • Compare query image and each exemplar using shape cues Database of Exemplars Query Image

  6. Finding similar patches Exemplar Query

  7. Compute sparse channels from image Extract a patch in each channel Idealized signal Apply spatially varying blur and sub-sample Descriptor is robust to small affine distortions [Berg & Malik, CVPR’01] Geometric Blur (Local Appearance Descriptor) ~ Geometric Blur Descriptor

  8. Horizontal Channel Vertical Channel Increasing Blur GB in Practice • In practice compute discrete blur levels for whole image and sample as needed for each feature location.

  9. [Berg, Berg & Malik, CVPR’05] Comparing Images • Sample 200 GB features from edge points • Dissimilarity from A to B is where the Fx are the GB features.

  10. Caltech 101 Dataset • Object Recognition Benchmark • 101 Categories: • Stereotypical pose • Little clutter • Objects centered • One object per image

  11. uses GB features [Zhang, Berg, Maire & Malik, CVPR’06] Caltech 101 Results

  12. ………. … … … 0.9 0.1 0.7 0.7 0.8 Primal features for SVM • Compare to 50 prototypes from each class • Use distances as feature vector for an SVM Query .. .. …… .. Prototypes Feature Vector

  13. SVM features interpretation • Slices of the Kernel Matrix: • Fixed-points in a higher dimensional vector space: q ti tj tk ti tj q tk

  14. SVM Specifics • SVMlight package • Same parameters for all categories: • Linear kernel • Default regularization parameter • Asymmetric cost doubling the weight of positive examples

  15. Computer-TV-Screen BestBerkeley-ShapeMedian Sports Meeting Car Results ’05Berkeley-Shape mAP = 0.38Best ’05 (IBM) mAP = 0.34 Results ’06 mAP = 0.11

  16. Limitations • Several objects per image: • Features do not capture: • Different Scales • Color

  17. Conclusions • Shape is an important cue for object recognition. • System that uses shape features only can have competitive performance. • Shape features are orthogonal to features used in the past.

  18. Thank You! petrov@eecs.berkeley.edu

More Related