1 / 44

Action Recognition with Exemplar Based 2.5D Graph Matching

Explore exemplar-based recognition in action recognition with 2.5D graph matching techniques, independent 3D pose, rich 2D appearance features, and more. Learn about dataset experiments and conclusions.

duyen
Download Presentation

Action Recognition with Exemplar Based 2.5D Graph Matching

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Action Recognition with Exemplar Based 2.5D Graph Matching Bangpeng Yao and Li Fei-Fei Computer Science Department Stanford University

  2. Action Recognition in Still Images Using computer

  3. Action Recognition in Still Images Using computer

  4. Action Recognition in Still Images Using computer 3D Recognition? Figure credit: Savarese & Fei-Fei, 2010

  5. Action Recognition in Still Images Using computer Felzenszwalb & Huttenlocher, 2005 Andriluka et al, 2009 Sapp et al, 2010 Yang & Ramanan, 2011 3D Recognition? Figure credit: Savarese & Fei-Fei, 2010

  6. Action Recognition in Still Images Using computer Lazebnik et al, 2006 Want et al, 2010 Delaitre et al., 2010 Yao et al., 2011

  7. Action Recognition in Still Images • 2.5D Representation • View independent 3D pose • Rich 2D appearance features Using computer

  8. Action Recognition in Still Images • 2.5D Representation • View independent 3D pose • Rich 2D appearance features Using computer

  9. Action Recognition in Still Images • 2.5D Representation • View independent 3D pose • Rich 2D appearance features Using computer • Exemplar based recognition • Only consider nearest image

  10. Outline • 2.5D Representation of Human Actions • Exemplar Based 2.5D Graph Matching • Dataset & Experiments • Conclusion

  11. Outline • 2.5D Representation of Human Actions • Exemplar Based 2.5D Graph Matching • Dataset & Experiments • Conclusion

  12. 2.5D Representation of Actions Original image Original image

  13. 2.5D Representation of Actions Original image 2D key points Original image 2D skeleton

  14. 2.5D Representation of Actions Original image 2D key points Original image 3D key points 3D skeleton 2D skeleton

  15. 2.5D Representation of Actions Original image 2D key points Original image 3D key points Appea-rance 3D skeleton 2D skeleton

  16. 2.5D Representation of Actions Original image 2D key points Original image 3D key points Appea-rance 3D skeleton 2.5D Graph 2.5D Graph 2D skeleton

  17. Estimating 2D Key Points Original image Pictorial Structure: 2D key points Not 100%? 3D key points Appea-rance See experiment … Felzenszwalb & Hunttenlocher, 2005 2.5D Graph Sapp et al, 2010

  18. Converting 2D Key Points to 3D Original image Taylor’s method: 2D key points points in 2D points in 3D 3D key points Appea-rance ± Resolving the “±” problem: 2.5D Graph • Configuration constraints (Lee & Chen, 1985) • Regression based refinement Taylor, 2000

  19. Outline • 2.5D Representation of Human Actions • Exemplar Based 2.5D Graph Matching • Dataset & Experiments • Conclusion

  20. Matching Two 2.5D Graphs Original image 2D key points G1 S(G1) A(G1) 3D key points Appea-rance G2 A(G2) S(G2) 2.5D Graph

  21. Matching Two 2.5D Graphs Original image 2D key points G1 S(G1) A(G1) 3D key points Appea-rance G2 A(G2) S(G2) 2.5D Graph Umeyama, 1991

  22. Exemplar Based Action Recognition Training images Test image

  23. Exemplar Based Action Recognition Training images Test image • Test image vs. each candidate image. • Time-consuming

  24. Exemplar Based Action Recognition Training images Test image • Our approach: Test image vs. a subset of training images.

  25. Exemplar Based Action Recognition Training images Test image • The smallest set of images that can recognize all within-class images in the exemplar based setting. • Dominating images. • Our approach: Test image vs. a subset of training images.

  26. Finding Dominating Images Training images Test image • Our approach: Test image vs. a subset of training images. • An iterative approach to find dominating images: • Maximize Coverage(I) for each image I;

  27. Finding Dominating Images Training images Test image • Our approach: Test image vs. a subset of training images. • An iterative approach to find dominating images: • Maximize Coverage(I) for each image I; • Find I* that maximizes Coverage(I*).

  28. Finding Dominating Images Training images Test image • Our approach: Test image vs. a subset of training images. • An iterative approach to find dominating images: • Maximize Coverage(I) for each image I; • Find I* that maximizes Coverage(I*). • Remove I* and Coverage(I*), return to step 2.

  29. Finding Dominating Images Training images Test image • Our approach: Test image vs. a subset of training images. • An iterative approach to find dominating images: • Maximize Coverage(I) for each image I; • Find I* that maximizes Coverage(I*). • Remove I* and Coverage(I*), return to step 2.

  30. Outline • 2.5D Representation of Human Actions • Exemplar Based 2.5D Graph Matching • Dataset & Experiments • Conclusion

  31. PPMI: Dataset • PPMI: People Playing (Interacting with) Musical Instruments • 24 classes of people interaction with different instruments. • 100 training & 100 testing for each class. Yao & Fei-Fei, 2010

  32. PPMI: Results 2D Pose only: Vs. 3D Pose only: Vs.

  33. PPMI: Results • Rich appearance info. • Pose can be wrong • Pose can be similar Lazebnik et al, 2006 Wang et al, 2010

  34. PPMI: Results

  35. PPMI: Results

  36. PPMI: Dominating Images

  37. PPMI: Dominating Images

  38. PASCAL 2011: Dataset • 3000 training & 3000 testing images each class Figure credit: Everingham Everingham et al, 2011

  39. PASCAL 2011: Pose Estimation • Human bounding box provided. • Two PS models: Full body and upper body.

  40. PASCAL 2011: Results HOBJ_DSAL: Prest et al, 2011 RF_SVM: Yao et al, 2011a POSELETS: Maji et al, 2011 ATTR_PART: Yao et al, 2011b

  41. PASCAL 2011: Results HOBJ_DSAL: Prest et al, 2011 RF_SVM: Yao et al, 2011a POSELETS: Maji et al, 2011 ATTR_PART: Yao et al, 2011b

  42. PASCAL 2011: Results HOBJ_DSAL: Prest et al, 2011 RF_SVM: Yao et al, 2011a POSELETS: Maji et al, 2011 ATTR_PART: Yao et al, 2011b

  43. Conclusion • 2.5D Representation • View independent 3D pose • Rich 2D appearance features Using computer • Exemplar based recognition • Only consider nearest image

  44. Acknowledgement

More Related