1 / 33

An Introduction to Real-time Machine Vision in Mechatronics

An Introduction to Real-time Machine Vision in Mechatronics. Dr. Onur TOKER. Outline. RT Machine Vision ? Mechatronics ? Review of previous experiments Image sensors (CMOS versus CCD) CMUCam, and cwCAM Interfacing a CCD camera to an 8-bit uC Difficulties in real-time machine vision

jenifer-nelson
Download Presentation

An Introduction to Real-time Machine Vision in Mechatronics

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. An Introduction to Real-time Machine Vision in Mechatronics Dr. Onur TOKER

  2. Outline • RT Machine Vision ? Mechatronics ? • Review of previous experiments • Image sensors (CMOS versus CCD) • CMUCam, and cwCAM • Interfacing a CCD camera to an 8-bit uC • Difficulties in real-time machine vision • Conclusion Dr. Onur TOKER

  3. RT Machine Vision ? Mechatronics ? • Machine vision is the ability of a computation machine to "see." • Visual object tracking • Object recognition • Automated inspection, sorting • Pattern recognition, etc. RT: There is no strict real-time system. There are systems with very short event response latency times. Dr. Onur TOKER

  4. Experiment #1 • 1-D tracking system • Analog video camera & PCI grabber • VB 6 & VFW based • Simple algorithm • PID control • Pentium 2/350MHz Dr. Onur TOKER

  5. Experiment #2 • Line following • Wireless video camera and ToyCar • Processing on a remote PC • VC++ & DirectX based • Simple algorithm • Pentium 3/1GHz Dr. Onur TOKER

  6. Experiment #3 • Intel 8051 • Very primitive machine vision • Rapid prototyping board • LDR sensor • MOSFET driver Dr. Onur TOKER

  7. Experiment #3 Dr. Onur TOKER

  8. Prototyping / Final product Final design EPROM based minimum size PCB Prototyping board Serial download, EEPROM based, 9V battery Dr. Onur TOKER

  9. A Student Project Line following robot Phototransistor based sensors Dr. Onur TOKER

  10. BOE-BOT kit Simple kit PBASIC Not very flexible Very small RAM IR LEDs & photo transistors Dr. Onur TOKER

  11. BOE-BOT demo Dr. Onur TOKER

  12. Other demos CMUCam demo (Color tracking) WAM demo (MIT 1995) (Tracking by stereo machine vision) Dr. Onur TOKER

  13. Image sensor types • Charged coupled devices (CCD) • Charge injection devices (CID) • CMOS Active Pixel Sensors (CMOS) • They all convert incident light (photons) into electronic charge (electrons) by a photo-conversion process. • Color sensors can be made by coating each individual pixel with a filter color (e.g. red, green, and blue). • Beyond that point, everything is different. Dr. Onur TOKER

  14. CMOS image sensors Digital output Easy to interface A CMOS sensor (OV7620) CUMCam uses such a sensor 2nd PCB has a Scenix uC DALSA CMOS Sensor Dr. Onur TOKER

  15. CCD image sensors Analog output Difficult to interface Require several support chips DALSA CCD Sensor Dr. Onur TOKER

  16. CMOS versus CCD Under same lightning, same distance, comparable budget, CCD image is better. CMOS sensor 640x480 mode CCD sensor 640x480 (NTSC output) Dr. Onur TOKER

  17. CMUCam architecture CMUCam SX28 uC CMOS sensor uC/DSP Serial I/O • “User device” issues high level commands • SX28 does the processing (Limited built-in functions) • SX28 replies Dr. Onur TOKER

  18. What is wrong with CMUCam ? • Serial I/O (Low bandwidth) • Low frame rate (Max. 17fps) • CMOS sensor • Processing done by SX28 • Limited to built in functions • Not much flexibility • Instead of FPGA, uses SX28 • Very compact design Dr. Onur TOKER

  19. Proposed cwCam architecture cwCam uC/DSP CCD camera FPGA uC/DSP Video ADC uC/DSP • Co-operating windowing approach (Discussed later) • Parallel processors • Parallel application specific digital architectures in the FPGA • ASIC CPU cores in FPGA Dr. Onur TOKER

  20. Machine Vision with an Analog Industrial camera • NTSC/30fps or PAL/25fps • Even/odd field interlacing: 60fips/50fips rate • 31ms VSYNC, 4.7us HSYNC for NTSC • Needs a high speed ADC (AD9048 is 35 MHz) • Most 8-bit uCs are too slow for this task • Scenix SX28AC/DP 13.3 ns instruction cycle • FPGA for accurate and high resolution capture Dr. Onur TOKER

  21. Digitized video signal One field One frame VSYNC Dr. Onur TOKER

  22. A single field VSYNC Several HSYNCs Dr. Onur TOKER

  23. Video ADC speed ? HSYNC ??? VSYNC Conclusion:Use 10MHz ADC Dr. Onur TOKER

  24. Scenix SX28AC/DP • 13.3 ns instruction cycle (75MHz clock) • 10MHz video sampling = 100 ns loop time • 1 Branch=3 cycles • 4 instruction loop OK, but int. RAM too small • 8051 too slow ! • PIC16F877 too slow ! • USE AN FPGA ! Dr. Onur TOKER

  25. Our FPGAs (Prototyping boards) Spartan II FPGA 50 Kgate 8MB RAM 8051 Dr. Onur TOKER

  26. Our ADC (AD9048) Actual photo of AD9048 used in our video digitizer • 35MSPS, 8-bit Flash ADC, Bipolar, 550mW, DIP 28 available • AD9203, 40MSPS, 10-bit,CMOS, 74mW, No DIP available Dr. Onur TOKER

  27. Cortex-I approach • Bederson, 1992 • Logarithmic structured space variant pixel geometry • Based on human vision system • For real-time machine vision, reduce data to < 1500 pixels Dr. Onur TOKER

  28. Co-operating windowing (1) • Nassif & Capson, 1997 • 2 Watch windows (200x20) • 1 Peripheral window (40x40 … 200x200) • 1 Foveal window (20x20) • Object tracking at 113Hz Dr. Onur TOKER

  29. Co-operating windowing (2) Dr. Onur TOKER

  30. Where we are at cwCAM ? • AD9048 Video ADC board design completed (PCB layout !) • AD9048 interfaced to 8051 prototyping board and tested • Logic design is being done by Xilinx ISE software • Mixed VHDL and graphical logic designs • Tedious and long task cwCam CCD camera FPGA Video ADC Dr. Onur TOKER

  31. Human Vision ? PUMA robot arm and dual camera set HMD and Dual monitor support Dr. Onur TOKER

  32. Conclusion • Real time machine vision requires innovative use of software and hardware techniques. • Cortex-I (Human Eye), Co-operating windowing, etc. • Innovative use of FPGAs and uC/DSPs. • High frame rate CCD sensors. • Optimum designs likely to be an application specific one. • cwCAM is based on co-operating windowing approach and innovative hardware/software techniques. Dr. Onur TOKER

  33. QUESTIONS ? Dr. Onur TOKER

More Related