710 088 robot vision messen aus bildern 2vo 1ku matthias r ther
1 / 31

710.088 ROBOT VISION („Messen aus Bildern“) 2VO 1KU Matthias Rüther - PowerPoint PPT Presentation

  • Uploaded on

710.088 ROBOT VISION („Messen aus Bildern“) 2VO 1KU Matthias Rüther. Kawada Industries Inc. DLR. Organization. VO : Tuesday 14:15-15:45 Seminarraum ICG Exam: Written Exam Oral Exam if Requested KU:implementation of lecture topics in the real

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' 710.088 ROBOT VISION („Messen aus Bildern“) 2VO 1KU Matthias Rüther' - boyce

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
710 088 robot vision messen aus bildern 2vo 1ku matthias r ther
710.088 ROBOT VISION(„Messen aus Bildern“) 2VO 1KUMatthias Rüther

Kawada Industries Inc.



VO: Tuesday 14:15-15:45

Seminarraum ICG

Exam: Written Exam

Oral Exam if Requested

KU:implementation of lecture topics in the real

world (on the lab-robots)

Groups of three students

Possible problems on the last slide

Scheduling of topics: 8.3.2005

If you are interested: excursions to industrial vision

companies (Alicona Imaging, M&R)

Time table
Time Table

1.3. : Introduction and Overview

8.3. : Projective Geometry (1)

15.3. : Projective Geometry (2)

12.4. : Projective Geometry (3)

19.4. : Projective Geometry (4)

26.4. : Camera Technologies

3.5. : Shape From X (1)

10.5. : Shape From X (2)

24.5. : Shape From X (3)

31.5. : Robot Kinematics (1)

7.6. : Robot Kinematics (2)

14.6. : Tracking of Moving Objects

21.6. : Visual Servoing / Hand Eye Coordination


  • Sciavicco, L., Siciliano, B., Modelling and Control of Robot Manipulators 2nd Ed., Springer, 2000

  • Ballard D.H., Brown C.M., "Computer Vision", Prentice-Hall, 1982

  • Sonka M., Hlavac V., Boyle Image Processing, Analysis and Machine Vision, Chapman Hall, 1998

  • Nalva V.S., "A Guided Tour of Computer Vision", Addison-Wesley Publishing Company, 1993

  • Horn B.K.P., "Robot Vision", MIT Press, Cambridge, 1986

  • Shirai Y., "Three- Dimensional Computer Vision", Springer Verlag, 1987

  • Faugeras O., Three-Dimensional Computer Vision A Geometric Viewpoint, MIT Press, 1993

  • Hartley R., Zissermann A., Multiple View Geometry in Computer Vision, Cambridge, 2001.


  • What is a robot?

    "A reprogrammable, multifunctional manipulator designed to move material, parts, tools, or specialized devices through various programmed motions for the performance of a variety of tasks"

    Robot Institute of America, 1979

    … in a three-dimensional environment.

  • Industrial

    • Mostly automatic manipulation of rigid parts with well-known shape in a specially prepared environment.

  • Medical

    • Mostly semi-automatic manipulation of deformable objects in a naturally created, space limited environment.

  • Field Robotics

    • Autonomous control and navigation of a mobile vehicle in an arbitrary environment.

Robot vs human
Robot vs Human

  • Human advantages:

    • Intelligence

    • Flexibility

    • Adaptability

    • Skill

    • Can Learn

    • Can Estimate

  • Robot Advantages:

    • Strength

    • Accuracy

    • Speed

    • Does not tire

    • Does repetitive tasks

    • Can Measure

Robotics goals and applications
Robotics: Goals and Applications

  • Robotics does not intend to develop the artificial human!

    [Whitney, D. E., Lozinski, C. A. and Rourke, J. M. (1986) Industrial robot forward calibration method and results.]

  • Goal: combine robot and human abilities.

  • Applications:

    • Automation (Production)

    • Inspection (Quality control)

    • Remote Sensing (Mapping)

    • Man-Machine interaction („Cobot“)

    • Robot Companion (Physically challenged people)

    • See [Brady, M. et. al. (eds). „Robot Motion: Planning and Control“]

What can computer vision do for robotics
What can Computer Vision do for Robotics?

  • Accurate Robot-Object Positioning

  • Keeping Relative Position under Movement

  • Visualization / Teaching / Telerobotics

  • Performing measurements

  • Object Recognition (see LV „Bildverarbeitung u. Mustererkennung“, „Bildverstehen“, „AK Computer Vision“)

  • Registration

Visual Servoing

Combining Computer Vision and Robotics


Motion Planning: Given a known world and a cooperative mechanism, how do I get there from here ?

Localization: Given sensors and a map, where am I ?

Vision: If my sensors are eyes, what do I do?

Mapping: Given sensors, how do I create a useful map?

Bug Algorithms: Given an unknowable world but a known goal and local sensing, how can I get there from here?

Abstraction level

Kinematics: if I move this motor somehow, what happens in other coordinate systems ?

Control (PID): what voltage should I set over time ?


Motor Modeling: what voltage should I set now ?

Computer vision
Computer Vision

  • What is Computer Vision?

    "Computer Vision describes the automatic deduction of the structure and the properties of a (possible dynamic) three-dimensional world from either a single or multiple two-dimensional images of the world"

    [Nalva VS, "A Guided Tour of Computer Vision"]

  • Measurement

    • Measure shape and material properties in a 3D environment. Accuracy is important.

  • Recognition

    • Cognitive systems interpret a 3D environment (object classification, categorization). Systems are allowed to fail to a certain extent (similar to humans).

  • Navigation

    • Navigation Systems orient themselves in a 3D environment. Robustness and time are important.


  • „Shape from X“ techniques measure shape properties of objects from 2D digital images.

    • Shape from Stereo: two cameras obeserve an object from different viewpoints (similar to human eye).

    • Shape from focus: limited depth of focus allows to measure object-camera-distance.

    • Shape from structured light: a light pattern is projected on the object, the pattern deformation gives shape information.

    • Shape from Shading: an object is illuminated from a single direction. Light reflection depends on object shape and follows a reflectance function.

Shape from structured light
Shape from Structured Light

  • Structured Light Sensor

Figures from PRIP, TU Vienna


  • SLAM: Simultaneous Localization and Mapping.

    • Where am I on my map?

    • If the place is unknown, build a new map, try to merge it with the original map.

  • Visual Odometry: calculate the relative motion of the camera between two frames. Summing up the motion gives the camera path. Error propagation!

  • Visual Servoing: move to / maintain a relative position between robot end effector and an object.

  • Tracking: continuously measure the position of an object within the sensor coordinate frame.




The final map:




  • Registration of CAD models to scene features:

Figures from P.Wunsch: Registration of CAD-Models to Images by Iterative Inverse Perspective Matching

Ku student problems
KU: Student Problems

  • Shape from Stereo 3 students

  • Shape from Focus 3 students

  • Shape from Structured Light:Laser 3 students

  • Shape from Structured Light:Pattern 3 students

  • Shape from Shading 3 students

  • Robot Kinematics 3 students

  • 2D Grip Planning 2..3 students

  • 2D Visual Servoing 3 students

  • 2D Tracking 3 students

  • Registration / Model Fitting 3 students

  • Visual Odometry + Randomized RANSAC 3 students