1 / 61

Recursive Composition in Computer Vision

Recursive Composition in Computer Vision. Leo Zhu CSAIL MIT Joint work with Chen, Yuille, Freeman and Torralba. Ideas behind Recursive Composition. Pattern Theory. Grenander 94 Compositionality. Geman 02, 06 Stochastic Grammar. Zhu and Mumford 06. How to deal with image complexity

clark
Download Presentation

Recursive Composition in Computer Vision

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Recursive Composition in Computer Vision Leo Zhu CSAIL MIT Joint work with Chen, Yuille, Freeman and Torralba

  2. Ideas behind Recursive Composition Pattern Theory. Grenander 94 Compositionality. Geman 02, 06 Stochastic Grammar. Zhu and Mumford 06 • How to deal with image complexity • A general framework for different vision tasks • Rich representation and tractable computation

  3. Recursive Composition • Representation • Recursive Compositional Models (RCMs) • Inference • Recursive Optimization • Learning • Supervised Parameter Estimation • Unsupervised Recursive Dictionary Learning • RCM-1: Deformable Object • RCM-2: Articulated Object • RCM-3: Scene (Entire Image)

  4. Model Deformable Object • Flat MRF • Nodes: object parts • Edges: spatial relations • Limitations: • Short range interaction • Sparse

  5. Recursive Composision

  6. Recursive Compositional Models:RCM-1 x: image y: (position, scale, orientation) graph=(nodes, edges) a: index of node b: child of a f: appearances on node a g: potentials on edges (a,b)

  7. RCM-1: the Recursive Formula Recursion x: image ; y: (position, scale, orientation); Vertical independency; Self-similarity;

  8. Recursive Composition • Representation • Recursive Compositional Models (RCMs) • Inference • Recursive Optimization • Learning • Supervised Parameter Estimation • Unsupervised Recursive Dictionary Learning

  9. Polynomial-time Inference Recursion • Polynomial-time Complexity: Inference task: Recursive Optimization:

  10. Supervised Learning Collins 02. Taskar et al. 04 • Supervised learning • Perceptron algorithm (MLE, max margin – svm) • Parameter estimation needs fast inference.

  11. Supervised learning by Perceptron Algorithm where Inference is critical for learning • Goal: • Input: a set of training images with ground truth . Initialize parameter vector. • Training algorithm (Collins 02): Loop over training samples: i = 1 to N Step 1: find the best using inference: Step 2: Update the parameters: End of Loop.

  12. Recursive Composition • Representation • Recursive Compositional Models (RCMs) • Inference • Recursive Optimization (Polynomial-time) • Learning • Supervised Parameter Estimation • RCM-1: Deformable Object

  13. RCM-1: Multi-level Potentials = * [ Gabor, Edge, …] Potentials for appearance

  14. RCM-1: Multi-level Potentials (position, scale, orientation) • Potentials for shape: triplet descriptors

  15. The Inference Results after Supervised Learning

  16. Segmentation Results

  17. Evaluations: Segmentation and Parsing • Segmentation (Accuracy of pixel labeling) • The proportion of the correct pixel labels (object or non-object) • Parsing (Average Position Error of matching) • The average distance between the positions of leaf nodes of the ground truth and those estimated in the parse tree

  18. Recursive Composition • Modeling: (Representation) • Recursive Compositional Models (RCMs) • Inference: (Computing) • Recursive Optimization (Polynomial-time) • Learning: • Supervised Parameter Estimation • Unsupervised Recursive Learning • RCM-1: deformable object

  19. Unsupervised Learning Correspondence is unknown ? Combinatorial Explosion problem • Task: given 10 training images, no labeling, no alignment, highly ambiguous features. • Induce the structure (nodes and edges) • Estimate the parameters.

  20. Recursive Dictionary Learning Recursion Barlow 94. • Multi-level dictionary (layer-wise greedy) • Bottom-Up and Top-Down recursive procedure • Three Principles: • Recursive Composition • Suspicious Coincidence • Competitive Exclusion

  21. 10 images for training

  22. Bottom-up Learning Suspicious Coincidence Clustering Composition Competitive Exclusion

  23. The Dictionary: From Generic Parts to Object Structures Unified representation (RCMs) and learning Bridge the gap between the generic features and specific object structures

  24. Dictionary Size, Part Sharing and Computational Complexity More Sharing

  25. Top-down refinement Fill in missing parts Examine every node from top to bottom

  26. Evaluations of Unsupervised Learning

  27. Scale up the System: Issue I More classes/viewpoints -> more training/detection cost

  28. Scale up the System: Issue II No enough data for rare viewpoints/classes

  29. Our Strategy Joint multi-class multi-view learning Appearance sharing Part sharing

  30. Joint Multi-Class Multi-View Learning 120 templates: 5 viewpoints & 26 classes

  31. Different Viewpoints Share same appearance

  32. Different Classes Share Common Parts

  33. Compact Hierarchical Dictionary

  34. Dense Part Sharing at Low Levels: Layer-2

  35. Less Part Sharing: Layer-3

  36. Sparse Part Sharing at High Levels: Layer-4

  37. Re-usable Parts: All Layers

  38. The more classes/viewpoints, the more amount of part sharing

  39. Multi-View Single Class Performance

  40. Recursive Composition • Representation • Recursive Compositional Models (RCMs) • Inference • Recursive Optimization (Polynomial-time) • Learning • Supervised Parameter Estimation • RCM-1: Deformable Object • RCM-2: Articulated Object

  41. RCM-2 for Articulated Object: Horses multiple poses y=(switch, position, scale, orientation) Composition Switch

  42. RCM-2 for Human Body

  43. Recursive Composition • Representation • Recursive Compositional Models (RCMs) • Inference • Recursive Optimization (Polynomial-time) • Learning • Supervised Parameter Estimation • RCM-1: Deformable Object • RCM-2: Articulated Object • RCM-3: Scene (Entire Image)

  44. Image Scene Parsing Task: Image Segmentation andLabeling

More Related