1 / 82

Overview of Our Sensors For Robotics

Overview of Our Sensors For Robotics. Machine vision. Computer vision To recover useful information about a scene from its 2-D projections. To take images as inputs and produce other types of outputs (object shape, object contour, etc.) Geometry + Measurement + Interpretation

Download Presentation

Overview of Our Sensors For Robotics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Overview of Our Sensors For Robotics

  2. Machine vision • Computer vision • To recover useful information about a scene from its 2-D projections. • To take images as inputs and produce other types of outputs (object shape, object contour, etc.) • Geometry + Measurement + Interpretation • To create a model of the real world from images.

  3. Topics • Computer vision system • Image enhancement • Image analysis • Pattern Classification

  4. Related fields • Image processing • Transformation of images into other images • Image compression, image enhancement • Useful in early stages of a machine vision system • Computer graphics • Pattern recognition • Artificial intelligence • Psychophysics

  5. Vision system hardware

  6. Image Processing System

  7. Image Representation

  8. Image • Image : a two-dimensional array of pixels • The indices [i, j] of pixels : integer values that specify the rows and columns in pixel values

  9. Sampling, pixeling and quantization • Sampling • The real image is sampled at a finite number of points. • Sampling rate : image resolution • how many pixels the digital image will have • e.g.) 640 x 480, 320 x 240, etc. • Pixel • Each image sample • At the sample point, an integer value of the image intensity

  10. Quantization • Each sample is represented with the finite word size of the computer. • How many intensity levels can be used to represent the intensity value at each sample point. • e.g.) 28 = 256, 25 = 32, etc.

  11. Color models • Color models for images, • RGB, CMY • Color models for video, • YIQ, YUV (YCbCr) • Relationship between color models :

  12. 6.7. Digital Cameras

  13. Digital Cameras • Technology • CCD (charge coupled devices) • CMOS (complementary metal oxide semiconductor) • Resolution • 60x80 black/white up to • several Mega-Pixels in 32bit color However:Embedded system has to have computing power to deal with this large amount of data!

  14. Vision (camera + framegrabber)

  15. Digital Cameras • Performance of embedded system: 10% - 50% of standard PC

  16. Interfacing Digital Cameras to CPU • Interfacing to CPU: • Completely depends on sensor chip specs • Many sensors provide several different interfacing protocols • versatile in hardware design • software gets very complicated • Typically: 8 bit parallel (or 4, 16, serial) • Numerous control signals required

  17. Interfacing Digital Cameras to CPU • Digital camera sensors are very complex units. • In many respects they are themselves similar to an embedded controller chip. • Some sensors buffer camera data and allow slow reading via handshake(ideal for slow microprocessors) • Most sensors send full imageas a streamafter start signal • (CPU must be fast enough to read or use hardware buffer or DMA) • We will not go into further details in this course. However, we consider camera access routines

  18. Simplified diagram of camera to CPU interface

  19. Problem with Digital Cameras • Problem • Every pixel from the camera causes an interrupt • Interrupt service routines take long, since they need to store register contents on the stack • Everything is slowed down • Solution • Use RAM buffer for image and read full image with single interrupt

  20. Idea • Use FIFO as image data buffer • FIFO is similar to dual-ported RAM, it is required since there is no synchronization between camera and CPU • When FIFO is half full, interrupt is generated • Interrupt service routine then reads FIFO until empty • (Assume delay is small enough to avoid FIFO overrun)

  21. Bayer Pattern

  22. De-Mosaic

  23. Conversion in Digital Cameras • Bayer Pattern • Output format of most digital cameras • Note: 2x2 pattern is not spatially located in a single point! • Can be simply converted to RGB (drop one green byte) 160x120 Bayer → 80x60 RGB • Can be better converted using “demosaicing” technique 160x120 Bayer → 160x120 RGB

  24. CMUCAM2+ CAMERA www.seattlerobotics.com • The camera can trackuser defined color blobs at up to 50 fps (frames per second) • Track motion using frame differencing at 26 fps • Find the centroid of any tracking data • Gather mean color and variance data • Gather a 28 bin histogram of each color channel • Manipulate horizontal pixel differenced images • Arbitrary image windowing • Adjust the camera’s image properties This camera can do a lot of processing

  25. This camera can do a lot of processing • Dump a raw image • Up to 160 X 255 resolution • Support multiple baud rates • Control 5 servos outputs • Slave parallel image processing mode off of single camera bus • Automatically use servos to dotwo axis color tracking • B/W analog video output (Pal or NTSC) • Flexible output packet customization • Multiple pass image processing on a buffered image

  26. Vision Guided Robotics and Applications in Industry and Medicine

  27. Contents • Robotics in General • Industrial Robotics • Medical Robotics • What can Computer Vision do for Robotics? • Vision Sensors • Issues / Problems • Visual Servoing • Application Examples • Summary

  28. Industrial Robot vs Human • Human advantages: • Intelligence • Flexibility • Adaptability • Skill • Can Learn • Can Estimate • Robot Advantages: • Strength • Accuracy • Speed • Does not tire • Does repetitive tasks • Can Measure Robot needs vision

  29. Industrial Robot • Requirements: • Accuracy • Tool Quality • Robustness • Strength • Speed • Price Production Cost • Maintenance Production Quality

  30. Medical (Surgical) Robot • Requirements • Safety • Accuracy • Reliability • Tool Quality • Price • Maintenance • Man-Machine Interface

  31. What can Computer Vision do for (industrial and medical) Robotics? • Accurate Robot-Object Positioning • Keeping Relative Position under Movement • Visualization / Teaching / Telerobotics • Performing measurements • Object Recognition • Registration Visual Servoing

  32. Vision Sensors • Single Perspective Camera • Multiple Perspective Cameras (e.g. Stereo Camera Pair) • Laser Scanner • Omnidirectional Camera • Structured Light Sensor

  33. Vision Sensors • Single Perspective Camera Single projection

  34. Vision Sensors • Multiple Perspective Cameras (e.g. Stereo Camera Pair)

  35. Vision Sensors • Multiple Perspective Cameras (e.g. Stereo Camera Pair)

  36. Vision Sensors • Laser Scanner

  37. Vision Sensors • Laser Scanner

  38. Vision Sensors • Omnidirectional Camera

  39. Vision Sensors • Omnidirectional Camera

  40. Vision Sensors • Structured Light Sensor Figures from PRIP, TU Vienna

  41. Issues/Problems of Vision Guided Robotics • Measurement Frequency • Measurement Uncertainty • Occlusion, Camera Positioning • Sensor dimensions

  42. Visual Servoing • Vision System operates in a closed control loop. • Better Accuracy than „Look and Move“ systems Figures from S.Hutchinson: A Tutorial on Visual Servo Control

More Related