1 / 22

Hand-Eye Coordination and Vision-based Interaction

This course focuses on vision for registering, understanding, and generating motion, as well as how to act based on visual information. Topics include motion estimation, sensory-motor control, user interfaces for computers and robots, and more.

charlesrobinson
Download Presentation

Hand-Eye Coordination and Vision-based Interaction

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Hand-Eye Coordination and Vision-based Interactionhttp://www.cs.ualberta.ca/~jag/courses/HandEye/ cmput610 Martin Jagersand

  2. Today: • Fun (hopefully) intro to some of the topics in the course. Give flavor of course. • Suggestions for some exciting projects you can choose. You are also welcome to propose your own. • (Boring) administrative details: Course schedule, room, participation and examination, class list etc…

  3. Most other courses are about static vision. What is in the image? This course: Vision for regestering, understanding and generating motion. How to act based on visual information. Main focus of course Symbol Interp Im Processing Images Agent doing physical motion Real world

  4. Where/What or Action/Perception division also in biological vision

  5. Main topics we will cover • Motion in the real world and its image projection • Vision and other sensory based control • Biological motor control. • User interfaces for computers, robots and other machines.

  6. Motion estimation:Image subtraction Detecting motion: 50 Candidate areas for motion

  7. Motion estimation 2 • A vector flow is a more useful representation

  8. Motion estimation 3 • Non-rigid motions are also important

  9. Motion estimation 4 • Generalize to many freedooms (DOFs)

  10. One application: Tracking • Goal: Stabilizing motion. Find move params

  11. Vision and Sensory based motion control

  12. Uncalibrated Visual Servoing • Let y = visual observation; x = motor control • Linear system model: y= • Linear p-controller • Estimate the Visual-Motor Jacobian

  13. Shortcut in Image based visual servoing

  14. Biological sensory-motor control • Sensory and motor areas in cortex

  15. Example: Eye movements • Relatively well studied area • Yet complex nonlinear kinematics • Interesting adaption behaviour Paths: 1 flash Multple targets Remembered location

  16. User Interface applications:Gesture and motion tracking

  17. User Interfaces 2:Robot Control • Vision based “Tele Assistance” • Describe task and objects by gestures and pointing • Visual language maps to physical actions

  18. Composite Task: Solving a Puzzle

  19. Proposed projects 1 • Neurosci Georgopoulos, Kalaska, Shieber... • PercAct Goodale, Lomis, • Characterizing motion: Activity recognition: Nelson,Alimonios Fleet, Jepson... Jagersand Matlab implementaton • Animating motion. Gleicher, Jagersand, Thalman

  20. Proposed projects 2 • Tracking: Hager, Black, XVision: practical experiments with. Filtering, Bar-Shalom: • Robotics: Controller HW and SW for the small robot arm (or the PUMA's) • Real time systems Real time linux

  21. Participation and Examination • Course readings • Class discussion, active class participation • Present some papers of your choice • Do your own reading or practical project and a writeup

  22. Administrative • Register for cmput610 • Proposed schedule: Tue, Thu 9-10am • Room TBA • Course literature and readings are given on-line when possible. • Announcements via email. Make sure you are on the email list. • Course web page (important):http://www.cs.ualberta.ca/~jag/courses/HandEye/

More Related