1 / 40

Learning Shared Body Plans

Learning Shared Body Plans. Ian Endres University of Illinois work with Derek Hoiem , Vivek Srikumar and Ming-Wei Chang. How should we represent multiple related object categories?. How should we represent multiple related object categories?.

elana
Download Presentation

Learning Shared Body Plans

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Learning Shared Body Plans Ian Endres University of Illinois work with Derek Hoiem, VivekSrikumar and Ming-Wei Chang

  2. How should we represent multiple related object categories?

  3. How should we represent multiple related object categories? Want to detect, localize, and estimate pose of broad range of objects, including new ones

  4. One option: independent detectors Basic-Level Categories Broad Categories Parts … Cat Detector Dog Detector Head Detector 4-Legged Animal Detector

  5. Our previous work: Train separate detectors, Joint spatial model Wheel Vehicle Animal Four-legged Mammal Head Leg Can run Can Jump Facing right Moves on road Facing right Farhadi Endres Hoiem (2010)

  6. Jointly trained multi-category models • Train part/category detectors to jointly predict object structure • Only need to perform well in context defined by others • Spatial model encodes likely part positions, number of parts, likely categories, etc. • Generalizes Felzenszwalb et al.: cross-category sharing, multiple parts with one model, variable size

  7. Deformable Part Models From Felzenszwalb et al.

  8. Detection with Deformable Part Models From Felzenszwalb et al.

  9. Shared mixture of deformable parts: Body Plans Include a body plan for background patches: No appearance models, just a bias

  10. Body Plan Overview High Scoring Detections + + Object Center + Head Anchors

  11. Anchor Point Score HOG based Deformable part model (Felzenszwalb et al.) Quadratic penalty in position and scale Sa = bias + appearance score - deformation cost Sa = bias + appearance score - deformation cost Overall score must be greater than 0 to be detected

  12. Inference: Head + + ✓ +

  13. Inference: Leg + + + + +

  14. Inference: Leg + + + ✓ + + Search Constraints: Count Pairwise Exclusion

  15. Inference: Leg + + + ✓ + +

  16. Inference: Leg + + + ✓ + + ✓

  17. Inference: Leg + + + ✓ + + ✓

  18. Inference: Leg + + + ✓ + ✓ + ✓

  19. Inference: Leg + + + ✓ + ✓ + ✓

  20. Inference: Leg ✓ + + + ✓ + ✓ + ✓

  21. Inference Score for each body plan: Overall score for an object hypothesis:

  22. Benefits of Joint Learning Only consider structures with:

  23. Benefits of Joint Learning No structures have

  24. (Latent) Max Margin Structured Learning Loss Highest Scoring Valid Structure Invalid Structure Soft margin slack

  25. Valid Structures Positive Examples Negative Examples Head Four-legged Elk Must select BG body plan LEG LEG LEG LEG Object Detectors: 50% Overlap with ground truth Part Detectors: 25% Overlap with ground truth

  26. Loss Positive Examples Negative Examples Head Four-legged Elk Non-BG body plan: +1 False Positives: +1 LEG LEG Head LEG LEG False Positives: +1 Duplicate Detections: +1 Missed Detections: + 1

  27. Optimization • Latent Structured SVM • Non-convex - CCCP • Stochastic gradient descent based cutting plane optimization

  28. Optimization Challenges • Expensive search for violated constraints • Mine many violated constraints at once • Speeds convergence • Large feature vectors (100k+) • Can’t store every mined violated constraint • Requires careful caching

  29. Experimental Setup • CORE: Train + Test • Familiar Categories: Camel, Dog, Elephant, Elk • Parts: Head, Leg, Torso • Unfamiliar Categories: Cat, Cow • Pascal 2008: Test • Unfamiliar Categories: Cat, Cow, Horse, Sheep

  30. Familiar Objects Unfamiliar Objects

  31. Mistakes

  32. Object Level Results AP

  33. Familiar four-legged parts AP

  34. Unfamiliar four-legged parts AP

  35. Mixed Supervision Four-legged Dog Head L E G L E G LEG LEG Learning Four-legged Dog Head L E G LEG L E G L E G

  36. Mixed Supervision Four-legged Dog Four-legged Dog Head + L E G L E G LEG LEG Learning Four-legged Dog Head L E G LEG L E G L E G

  37. Mixed Supervision - Learning • Unlabeled boxes become latent variables • Compute most likely positition • No loss for missed detections Loss Highest Scoring Valid Structure

  38. Mixed Supervision … Mixed Results AP

  39. Conclusions • Jointly representing related categories leads to better performance and generalization to unfamiliar categories • Joint training important to get full benefit of spatial model

  40. Thanks

More Related