710 088 robot vision messen aus bildern 2vo 1ku matthias r ther
Skip this Video
Download Presentation
710.088 ROBOT VISION („Messen aus Bildern“) 2VO 1KU Matthias Rüther

Loading in 2 Seconds...

play fullscreen
1 / 31

710.088 ROBOT VISION („Messen aus Bildern“) 2VO 1KU Matthias Rüther - PowerPoint PPT Presentation

  • Uploaded on

710.088 ROBOT VISION („Messen aus Bildern“) 2VO 1KU Matthias Rüther. Kawada Industries Inc. DLR. Organization. VO : Tuesday 14:15-15:45 Seminarraum ICG Exam: Written Exam Oral Exam if Requested KU:implementation of lecture topics in the real

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about '710.088 ROBOT VISION („Messen aus Bildern“) 2VO 1KU Matthias Rüther' - boyce

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

VO: Tuesday 14:15-15:45

Seminarraum ICG

Exam: Written Exam

Oral Exam if Requested

KU:implementation of lecture topics in the real

world (on the lab-robots)

Groups of three students

Possible problems on the last slide

Scheduling of topics: 8.3.2005

If you are interested: excursions to industrial vision

companies (Alicona Imaging, M&R)

time table
Time Table

1.3. : Introduction and Overview

8.3. : Projective Geometry (1)

15.3. : Projective Geometry (2)

12.4. : Projective Geometry (3)

19.4. : Projective Geometry (4)

26.4. : Camera Technologies

3.5. : Shape From X (1)

10.5. : Shape From X (2)

24.5. : Shape From X (3)

31.5. : Robot Kinematics (1)

7.6. : Robot Kinematics (2)

14.6. : Tracking of Moving Objects

21.6. : Visual Servoing / Hand Eye Coordination

  • Sciavicco, L., Siciliano, B., Modelling and Control of Robot Manipulators 2nd Ed., Springer, 2000
  • Ballard D.H., Brown C.M., "Computer Vision", Prentice-Hall, 1982
  • Sonka M., Hlavac V., Boyle Image Processing, Analysis and Machine Vision, Chapman Hall, 1998
  • Nalva V.S., "A Guided Tour of Computer Vision", Addison-Wesley Publishing Company, 1993
  • Horn B.K.P., "Robot Vision", MIT Press, Cambridge, 1986
  • Shirai Y., "Three- Dimensional Computer Vision", Springer Verlag, 1987
  • Faugeras O., Three-Dimensional Computer Vision A Geometric Viewpoint, MIT Press, 1993
  • Hartley R., Zissermann A., Multiple View Geometry in Computer Vision, Cambridge, 2001.
  • What is a robot?

"A reprogrammable, multifunctional manipulator designed to move material, parts, tools, or specialized devices through various programmed motions for the performance of a variety of tasks"

Robot Institute of America, 1979

… in a three-dimensional environment.

  • Industrial
    • Mostly automatic manipulation of rigid parts with well-known shape in a specially prepared environment.
  • Medical
    • Mostly semi-automatic manipulation of deformable objects in a naturally created, space limited environment.
  • Field Robotics
    • Autonomous control and navigation of a mobile vehicle in an arbitrary environment.
robot vs human
Robot vs Human
  • Human advantages:
    • Intelligence
    • Flexibility
    • Adaptability
    • Skill
    • Can Learn
    • Can Estimate
  • Robot Advantages:
    • Strength
    • Accuracy
    • Speed
    • Does not tire
    • Does repetitive tasks
    • Can Measure
robotics goals and applications
Robotics: Goals and Applications
  • Robotics does not intend to develop the artificial human!

[Whitney, D. E., Lozinski, C. A. and Rourke, J. M. (1986) Industrial robot forward calibration method and results.]

  • Goal: combine robot and human abilities.
  • Applications:
    • Automation (Production)
    • Inspection (Quality control)
    • Remote Sensing (Mapping)
    • Man-Machine interaction („Cobot“)
    • Robot Companion (Physically challenged people)
    • See [Brady, M. et. al. (eds). „Robot Motion: Planning and Control“]
what can computer vision do for robotics
What can Computer Vision do for Robotics?
  • Accurate Robot-Object Positioning
  • Keeping Relative Position under Movement
  • Visualization / Teaching / Telerobotics
  • Performing measurements
  • Object Recognition (see LV „Bildverarbeitung u. Mustererkennung“, „Bildverstehen“, „AK Computer Vision“)
  • Registration

Visual Servoing

Combining Computer Vision and Robotics


Motion Planning: Given a known world and a cooperative mechanism, how do I get there from here ?

Localization: Given sensors and a map, where am I ?

Vision: If my sensors are eyes, what do I do?

Mapping: Given sensors, how do I create a useful map?

Bug Algorithms: Given an unknowable world but a known goal and local sensing, how can I get there from here?

Abstraction level

Kinematics: if I move this motor somehow, what happens in other coordinate systems ?

Control (PID): what voltage should I set over time ?


Motor Modeling: what voltage should I set now ?

computer vision
Computer Vision
  • What is Computer Vision?

"Computer Vision describes the automatic deduction of the structure and the properties of a (possible dynamic) three-dimensional world from either a single or multiple two-dimensional images of the world"

[Nalva VS, "A Guided Tour of Computer Vision"]

  • Measurement
    • Measure shape and material properties in a 3D environment. Accuracy is important.
  • Recognition
    • Cognitive systems interpret a 3D environment (object classification, categorization). Systems are allowed to fail to a certain extent (similar to humans).
  • Navigation
    • Navigation Systems orient themselves in a 3D environment. Robustness and time are important.
  • „Shape from X“ techniques measure shape properties of objects from 2D digital images.
    • Shape from Stereo: two cameras obeserve an object from different viewpoints (similar to human eye).
    • Shape from focus: limited depth of focus allows to measure object-camera-distance.
    • Shape from structured light: a light pattern is projected on the object, the pattern deformation gives shape information.
    • Shape from Shading: an object is illuminated from a single direction. Light reflection depends on object shape and follows a reflectance function.
shape from structured light
Shape from Structured Light
  • Structured Light Sensor

Figures from PRIP, TU Vienna

  • SLAM: Simultaneous Localization and Mapping.
    • Where am I on my map?
    • If the place is unknown, build a new map, try to merge it with the original map.
  • Visual Odometry: calculate the relative motion of the camera between two frames. Summing up the motion gives the camera path. Error propagation!
  • Visual Servoing: move to / maintain a relative position between robot end effector and an object.
  • Tracking: continuously measure the position of an object within the sensor coordinate frame.



The final map:



  • Registration of CAD models to scene features:

Figures from P.Wunsch: Registration of CAD-Models to Images by Iterative Inverse Perspective Matching

ku student problems
KU: Student Problems
  • Shape from Stereo 3 students
  • Shape from Focus 3 students
  • Shape from Structured Light:Laser 3 students
  • Shape from Structured Light:Pattern 3 students
  • Shape from Shading 3 students
  • Robot Kinematics 3 students
  • 2D Grip Planning 2..3 students
  • 2D Visual Servoing 3 students
  • 2D Tracking 3 students
  • Registration / Model Fitting 3 students
  • Visual Odometry + Randomized RANSAC 3 students