An introduction to real time machine vision in mechatronics
This presentation is the property of its rightful owner.
Sponsored Links
1 / 33

An Introduction to Real-time Machine Vision in Mechatronics PowerPoint PPT Presentation


  • 127 Views
  • Uploaded on
  • Presentation posted in: General

An Introduction to Real-time Machine Vision in Mechatronics. Dr. Onur TOKER. Outline. RT Machine Vision ? Mechatronics ? Review of previous experiments Image sensors (CMOS versus CCD) CMUCam, and cwCAM Interfacing a CCD camera to an 8-bit uC Difficulties in real-time machine vision

Download Presentation

An Introduction to Real-time Machine Vision in Mechatronics

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


An introduction to real time machine vision in mechatronics

An Introduction to Real-time Machine Vision in Mechatronics

Dr. Onur TOKER


Outline

Outline

  • RT Machine Vision ? Mechatronics ?

  • Review of previous experiments

  • Image sensors (CMOS versus CCD)

  • CMUCam, and cwCAM

  • Interfacing a CCD camera to an 8-bit uC

  • Difficulties in real-time machine vision

  • Conclusion

Dr. Onur TOKER


Rt machine vision mechatronics

RT Machine Vision ? Mechatronics ?

  • Machine vision is the ability of a computation machine to "see."

  • Visual object tracking

  • Object recognition

  • Automated inspection, sorting

  • Pattern recognition, etc.

RT: There is no strict real-time system. There are systems with very short event response latency times.

Dr. Onur TOKER


Experiment 1

Experiment #1

  • 1-D tracking system

  • Analog video camera & PCI grabber

  • VB 6 & VFW based

  • Simple algorithm

  • PID control

  • Pentium 2/350MHz

Dr. Onur TOKER


Experiment 2

Experiment #2

  • Line following

  • Wireless video camera and ToyCar

  • Processing on a remote PC

  • VC++ & DirectX based

  • Simple algorithm

  • Pentium 3/1GHz

Dr. Onur TOKER


Experiment 3

Experiment #3

  • Intel 8051

  • Very primitive machine vision

  • Rapid prototyping board

  • LDR sensor

  • MOSFET driver

Dr. Onur TOKER


Experiment 31

Experiment #3

Dr. Onur TOKER


Prototyping final product

Prototyping / Final product

Final design EPROM based minimum size PCB

Prototyping board Serial download, EEPROM based, 9V battery

Dr. Onur TOKER


A student project

A Student Project

Line following robot

Phototransistor based sensors

Dr. Onur TOKER


Boe bot kit

BOE-BOT kit

Simple kit

PBASIC

Not very flexible

Very small RAM

IR LEDs &

photo transistors

Dr. Onur TOKER


Boe bot demo

BOE-BOT demo

Dr. Onur TOKER


Other demos

Other demos

CMUCam demo

(Color tracking)

WAM demo (MIT 1995)

(Tracking by stereo machine vision)

Dr. Onur TOKER


Image sensor types

Image sensor types

  • Charged coupled devices (CCD)

  • Charge injection devices (CID)

  • CMOS Active Pixel Sensors (CMOS)

  • They all convert incident light (photons) into electronic charge (electrons) by a photo-conversion process.

  • Color sensors can be made by coating each individual pixel with a filter color (e.g. red, green, and blue).

  • Beyond that point, everything is different.

Dr. Onur TOKER


Cmos image sensors

CMOS image sensors

Digital output

Easy to interface

A CMOS sensor (OV7620)

CUMCam uses such a sensor 2nd PCB has a Scenix uC

DALSA CMOS Sensor

Dr. Onur TOKER


Ccd image sensors

CCD image sensors

Analog output

Difficult to interface

Require several support chips

DALSA CCD Sensor

Dr. Onur TOKER


Cmos versus ccd

CMOS versus CCD

Under same lightning,

same distance,

comparable budget,

CCD image is better.

CMOS sensor

640x480 mode

CCD sensor

640x480 (NTSC output)

Dr. Onur TOKER


Cmucam architecture

CMUCam architecture

CMUCam

SX28

uC

CMOS sensor

uC/DSP

Serial I/O

  • “User device” issues high level commands

  • SX28 does the processing (Limited built-in functions)

  • SX28 replies

Dr. Onur TOKER


What is wrong with cmucam

What is wrong with CMUCam ?

  • Serial I/O (Low bandwidth)

  • Low frame rate (Max. 17fps)

  • CMOS sensor

  • Processing done by SX28

  • Limited to built in functions

  • Not much flexibility

  • Instead of FPGA, uses SX28

  • Very compact design

Dr. Onur TOKER


Proposed cwcam architecture

Proposed cwCam architecture

cwCam

uC/DSP

CCD camera

FPGA

uC/DSP

Video ADC

uC/DSP

  • Co-operating windowing approach (Discussed later)

  • Parallel processors

  • Parallel application specific digital architectures in the FPGA

  • ASIC CPU cores in FPGA

Dr. Onur TOKER


Machine vision with an analog industrial camera

Machine Vision with an Analog Industrial camera

  • NTSC/30fps or PAL/25fps

  • Even/odd field interlacing: 60fips/50fips rate

  • 31ms VSYNC, 4.7us HSYNC for NTSC

  • Needs a high speed ADC (AD9048 is 35 MHz)

  • Most 8-bit uCs are too slow for this task

  • Scenix SX28AC/DP 13.3 ns instruction cycle

  • FPGA for accurate and high resolution capture

Dr. Onur TOKER


Digitized video signal

Digitized video signal

One field

One frame

VSYNC

Dr. Onur TOKER


A single field

A single field

VSYNC

Several HSYNCs

Dr. Onur TOKER


Video adc speed

Video ADC speed ?

HSYNC ???

VSYNC

Conclusion:Use 10MHz ADC

Dr. Onur TOKER


Scenix sx28ac dp

Scenix SX28AC/DP

  • 13.3 ns instruction cycle (75MHz clock)

  • 10MHz video sampling = 100 ns loop time

  • 1 Branch=3 cycles

  • 4 instruction loop OK, but int. RAM too small

  • 8051 too slow !

  • PIC16F877 too slow !

  • USE AN FPGA !

Dr. Onur TOKER


Our fpgas prototyping boards

Our FPGAs (Prototyping boards)

Spartan II FPGA 50 Kgate

8MB RAM

8051

Dr. Onur TOKER


Our adc ad9048

Our ADC (AD9048)

Actual photo of AD9048 used in our video digitizer

  • 35MSPS, 8-bit Flash ADC, Bipolar, 550mW, DIP 28 available

  • AD9203, 40MSPS, 10-bit,CMOS, 74mW, No DIP available

Dr. Onur TOKER


Cortex i approach

Cortex-I approach

  • Bederson, 1992

  • Logarithmic structured space variant pixel geometry

  • Based on human vision system

  • For real-time machine vision, reduce data to < 1500 pixels

Dr. Onur TOKER


Co operating windowing 1

Co-operating windowing (1)

  • Nassif & Capson, 1997

  • 2 Watch windows (200x20)

  • 1 Peripheral window (40x40 … 200x200)

  • 1 Foveal window (20x20)

  • Object tracking at 113Hz

Dr. Onur TOKER


Co operating windowing 2

Co-operating windowing (2)

Dr. Onur TOKER


Where we are at cwcam

Where we are at cwCAM ?

  • AD9048 Video ADC board design completed (PCB layout !)

  • AD9048 interfaced to 8051 prototyping board and tested

  • Logic design is being done by Xilinx ISE software

  • Mixed VHDL and graphical logic designs

  • Tedious and long task

cwCam

CCD camera

FPGA

Video ADC

Dr. Onur TOKER


Human vision

Human Vision ?

PUMA robot arm and dual camera set

HMD and Dual monitor support

Dr. Onur TOKER


Conclusion

Conclusion

  • Real time machine vision requires innovative use of software and hardware techniques.

  • Cortex-I (Human Eye), Co-operating windowing, etc.

  • Innovative use of FPGAs and uC/DSPs.

  • High frame rate CCD sensors.

  • Optimum designs likely to be an application specific one.

  • cwCAM is based on co-operating windowing approach and innovative hardware/software techniques.

Dr. Onur TOKER


An introduction to real time machine vision in mechatronics

QUESTIONS ?

Dr. Onur TOKER


  • Login