1 / 18

Exploring Vision-Based Interaction: Principles, Tools, and Applications

This resource delves into the realm of vision-based interaction, tracing its origins and development since the 1970s with pioneers like Myron Krueger. It outlines the fundamental principles, such as the separation of video from background using chroma key techniques, and introduces 'verbs' and 'nouns' in vision-based UIs, which involve detecting and recognizing objects and their interactions. Additionally, it highlights key tools, including Intel's OpenCV and the Papier-Mâché framework, and recommends foundational texts for further learning in computer vision and interactive systems.

trish
Download Presentation

Exploring Vision-Based Interaction: Principles, Tools, and Applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Vision-based Interaction Scott Klemmer 11 November 2004

  2. cs547: Michael Shiloh Teleo: Rapid Prototyping Toolkit Making Things Vision-based Interaction

  3. The first vision-based interface • Myron Krueger used computer vision to create Responsive Environments (1970s) • “Reaction is the Medium” • http://www.artmuseum.net/w2vr/timeline/videoplace_video.html Vision-based Interaction

  4. How it works • Video and background are separated in analog using chroma key techniques(think broadcast news) • The first and last points of each raster are stored in the computer, and represent the person’s outline Vision-based Interaction

  5. Vision-based UIs: “Verbs” • Detecting and Tracking elements of a certain type in a scene • Capturing contents of detected objects • Recognizing individual members in an object class Vision-based Interaction

  6. Vision-based UIs: “Verbs” • Detecting and Tracking elements of a certain type in a scene Vision-based Interaction

  7. Vision-based UIs: “Verbs” • Capturing contents of detected objects Vision-based Interaction

  8. Vision-based UIs: “Verbs” • Recognizing individual members in a class Vision-based Interaction

  9. Vision-based UIs: “Nouns” • People (one or multiple) • Bodies • Faces • Hands • Documents • Objects Vision-based Interaction

  10. Vision-based UIs: “Nouns” • People (one or multiple) • Bodies • Faces • Hands • Documents • Objects Vision-based Interaction

  11. Vision-based UIs: “Nouns” • People (one or multiple) • Bodies • Faces • Hands • Documents • Objects Vision-based Interaction

  12. I N F R A S T R U C T U R E Background Subtraction Vision-based Interaction

  13. Image Moments (of Inertia) • 0th moment is mass(total number of pixels) Vision-based Interaction

  14. Image Moments (of Inertia) • 1st moment is center Vision-based Interaction

  15. Image Moments (of Inertia) • 2nd moment is orientation Vision-based Interaction

  16. Tools for Vision apps • Intel’s OpenCV • C API to highly optimized image processing functions (threshold, dilate, optical flow, …) • http://www.intel.com/research/mrl/research/opencv • Fast to run! Slow to develop • Great for vision folks; too low-level for app folks • Papier-Mâché • Java API (and to some extent visual UI) for vision (and other physical input) • http://guir.berkeley.edu/papier-mache • Fast to develop! Slow to run • Great for app folks; ~5 fps can sometimes be too slow Vision-based Interaction

  17. Good Vision Books • Computer Vision: A Modern Approach • David Forsyth and Jean Ponce (2003) • Fantastic book; but goal is more theoretical understanding than practical application • Robot Vision • Berthold Horn (1987) • More focused on apps and interactive algorithms • Somewhat out of date Vision-based Interaction

  18. iRoom demo after class… Vision-based Interaction

More Related