1 / 47

Proxemics Recognition

Proxemics Recognition. Yi Yang 1 , Simon Baker, Anitha Kannan, Deva Ramanan 1 1 Department of Computer Science, UC Irvine. Proxemics. Proxemics: the study of spatial arrangement of people as they interact - anthropologist Edward T. Hall in 1963. Brother and Sister Holding Hands.

manjit
Download Presentation

Proxemics Recognition

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Proxemics Recognition Yi Yang1, Simon Baker, Anitha Kannan, Deva Ramanan1 1Department of Computer Science, UC Irvine

  2. Proxemics Proxemics: the study of spatial arrangement of people as they interact - anthropologist Edward T. Hall in 1963

  3. Brother and Sister Holding Hands Friends Walking Side by Side Mom Holding Baby Husband Huggingand Holding Wife’s Hand

  4. Touch Code • Hand Touch Hand • Hand Touch Shoulder • Shoulder Touch Shoulder • Arm Touch Torso

  5. Applications • Personal Photo Search: • Find a specific interesting photo • Analysis of TV shows and movies • Kinect • Web Search • Auto-Movie/Auto-Slideshow • Locate interesting scenes

  6. Proxemics DataSet • 200 training images • 150 testing images • Collected From • Simon, Bing, Google, Gettyimage, Flickr • No video data • No Kinect 3d depth data

  7. Proxemics DataSet Complexity Number of People

  8. ANaïveApproach Input Image

  9. ANaïveApproach Input Image Image Feature

  10. ANaïveApproach Input Image Image Feature Pose Estimation i.e. Find skeletons

  11. ANaïveApproach Input Image Image Feature Hand touch Hand Interaction Recognition Pose Estimation i.e. Find skeletons

  12. Naïve Approach Results Hand touch Hand Hand touch Shoulder

  13. Human Pose Estimation • Not bad when no real interaction between people - Y. Yang & D. Ramanan, CVPR 2011

  14. Interactions Hurt Pose Estimation Occlusion + Ambiguous Parts

  15. OurApproachDirectProxemicsRecognition Input Image Image Feature

  16. OurApproachDirectProxemicsRecognition Input Image Image Feature Hand touch Hand Interaction Recognition

  17. Pictorial Structure Model • : Image • : Location of part - P. Felzenszwalb etc., PAMI 2009

  18. Pictorial Structure Model • : Unary template for part • : Local image features at location

  19. Pictorial Structure Model • : Spatial features between and • : Pairwise springs between part and part

  20. “Two Head Monster” Model i.e. “Hand-Touching-Hand”

  21. “Two Head Monster” Model i.e. “Hand-Touching-Hand”

  22. The models

  23. Match Model to Image • Inference: • Efficient algorithm: • Dynamic programming • Learning: • Structural SVM Solver

  24. Refinements + Extensions • Sub-categories Because of symmetry, 4 models for hand-hand etc R. Hand L. Hand L. Hand R. Hand L. Hand L. Hand R. Hand R. Hand

  25. Refinements + Extensions • Sub-categories Because of symmetry, 4 models for hand-hand etc • Co-occurrence of proxemics: Reduce redundancy, map Multi-Label -> Multi-Class R. Hand L. Hand L. Hand R. Hand L. Hand L. Hand R. Hand R. Hand

  26. Naïve Approach Results Hand touch Hand Hand touch Shoulder

  27. Direct Approach Results Hand touch Hand Hand touch Shoulder

  28. Quantitative Results [1] Y. Yang & D. Ramanan, CVPR 2011

  29. Improves Pose Estimation Y & D CVPR 2011 Our Model

  30. Improves Pose Estimation Y & D CVPR 2011 Our Model

  31. Improves Pose Estimation Y & D CVPR 2011 Our Model

  32. Conclusion • Proxemics and touch codes for human interaction

  33. Conclusion • Proxemics and touch codes for human interaction • Directly recognizing proxemics significantly outperforms

  34. Conclusion • Proxemics and touch codes for human interaction • Directly recognizing proxemics significantly outperforms • Recognizing proxemics helps pose estimation

  35. Acknowledgements Thank Simon and MSR for internship

  36. Acknowledgements Thank Anitha for a lot of suggestions

  37. Acknowledgements Thank Anarb for gettyimages

  38. Acknowledgements Thank Eletee for her beautiful smiling

  39. Acknowledgements Thank everybody for not falling asleep

  40. Thank you

  41. ArticulatedPoseEstimation - Yi Yang & Deva Ramanan, CVPR 2011

  42. Inference & Learning Inference For a tree graph (V,E): dynamicprogramming Given labeled positive and negative , write , and Learning

More Related