mobile robotics 6 vision 1 l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Mobile Robotics: 6. Vision 1 PowerPoint Presentation
Download Presentation
Mobile Robotics: 6. Vision 1

Loading in 2 Seconds...

play fullscreen
1 / 35

Mobile Robotics: 6. Vision 1 - PowerPoint PPT Presentation


  • 285 Views
  • Uploaded on

Dr. Brian Mac Namee ( www.comp.dit.ie/bmacnamee ) Mobile Robotics: 6. Vision 1 Acknowledgments These notes are based (heavily) on those provided by the authors to accompany “Introduction to Autonomous Mobile Robots” by Roland Siegwart and Illah R. Nourbakhsh

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Mobile Robotics: 6. Vision 1' - Samuel


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
acknowledgments
Acknowledgments
  • These notes are based (heavily) on those provided by the authors to accompany “Introduction to Autonomous Mobile Robots” by Roland Siegwart and Illah R. Nourbakhsh

More information about the book is available at:http://autonomousmobilerobots.epfl.ch/

The book can be bought at:The MIT Press and Amazon.com

today s lecture
Today’s Lecture
  • Why is vision hard?
  • Brief historical overview
    • From early cameras to digital cameras
  • Low-level robot vision
    • Camera as sensor
    • Color representation
    • Object detection
vision in general
Vision In General
  • Vision is our most powerful sense providing us with an enormous amount of information about our environment and enables us to interact intelligently with the environment
  • It is therefore not surprising that an enormous amount of effort has occurred to give machines a sense of vision
  • Vision is also our most complicated sense
    • Whilst we can reconstruct views with high resolution on photographic paper, understanding how the brain processes the information from our eyes is still in its infancy
vision in general cont
Vision In General (cont…)
  • When an image is recorded through a camera, a 3-D scene is projected onto a 2-D plane
  • In order to try and recover some “useful information” from the scene, usually edge detectors are used to find the contours of the objects
  • From these edges or edge fragments, much research time has to been spent attempting to produce fool proof algorithms which can provide all the necessary information required to reconstruct the 3-D scene which produced the 2-D image
  • The interpretation of 3-D scenes from 2-D images is not a trivial task
vision is hard brightness adaptation
Vision Is Hard! (Brightness Adaptation)

For more great illusion examples take a look at: http://web.mit.edu/persci/gaz/

vision is hard illusions
Vision Is Hard! (Illusions)
  • Our visual systems play lots of interesting tricks on us
vision is hard illusions12
Vision Is Hard! (Illusions)
  • Stare at the cross in the middle of the image and think circles
camera obscura

Photograph of camera obscura interior:

Portmerion Village, North Wales

Camera Obscura
  • Mo Ti, Chinese philosopher, 5th Century B.C.
    • Described linear light paths, pinhole image formation
  • Leonardo da Vinci (1452-1519)
    • Demonstrated camera obscura (lens added later)

Frisius (1544)

toward photography
Toward Photography
  • People sought a way to “fix” the images at the back of the camera obscura
  • Pursued decades of experimentation with light-sensitive salts, acids, etc.
  • First photograph produced when?
first photograph
First Photograph
  • Joseph Nicéphore Niépce “View from the Window at Le Gras”, c. 1826
  • Aluminum plate coated with light-sensitive material

Joseph Nicéphore Niépce

Harry Ransom Center

Kodak (reproduction)

More information on the first photograph is available at: http://www.hrc.utexas.edu/exhibitions/permanent/wfp/

first digital cameras
First Digital Cameras
  • Photoelectric effect (Hertz 1887; Einstein 1905)
  • Charge-coupled devices as storage (late 1960’s)
  • Light sensing, pixel row readout (early 1970’s)
  • First electronic CCD still- image camera (1975):
    • Fairchild CCD element 
    • Resolution: 100 x 100 b&w
    • Image capture time: 23 sec.,mostly writing cassette tape
    • Total weight: 8½ pounds

Kodak, c. 1975

modern digital cameras
Modern Digital Cameras
  • Today, fifty Euro buys a camera with:
    • 640 x 480 pixel resolution at 30Hz
    • 1280 x 960 still image resolution
    • 24-bit RGB pixels (8 bits per channel)
    • Automatic gain control, color balancing
    • On-chip lossy compression algorithms
    • Uncompressed images if desired
    • Integrated microphone, USB interface
    • Limitations
      • Narrow dynamic range
      • Narrow FOV, with fixed spatial resolution
      • No motion / active vision capabilities
vision based sensors hardware
Vision-Based Sensors: Hardware
  • CCD (light-sensitive, discharging capacitors of 5 to 25 micron)
  • CMOS (Complementary Metal Oxide Semiconductor) technology
what is a digital image
What Is A Digital Image?
  • A digital image is a 2-D representation of a 3-D scene as a finite set of digital values, called picture elements or pixels
what is a digital image cont

1 pixel

What Is A Digital Image? (cont…)
  • Pixel values typically represent gray levels, colours, heights, opacities etc
  • Rememberdigitization implies that a digital image is an approximation of a real scene
digital image contents

v

height

width

IO

u

Image coordinates (pixels)

Digital Image Contents
  • Why are pixels represented as “RGB”?
    • Is world made of red, green, and blue “stuff”?
  • … Answer requires a digression (or two)about human vision, cameras as sensors
visible light spectrum
Visible Light Spectrum

Solar (ECI, Oxford)

Incandescent (Wikipedia)

Freedman & Kaufmann, Universe

image as measurement

Illumination spectrum

Reflection spectrum

IJVS

Image As Measurement
  • What does eye/camera actually observe?
  • … the product of illumination spectrum
  • with absorption or reflection spectrum!

=

(at each image point)

X

eye anatomy
Eye Anatomy
  • Spectrum incident on light-sensitive retina

(View of R eye from above)

Rods and cones

Incident spectral

distribution

After Isaka (2004)

blind spot experiment
Blind-Spot Experiment
  • Draw an image similar to that below on a piece of paper (the dot and cross are about 6 inches apart)
  • Close your right eye and focus on the cross with your left eye
  • Hold the image about 20 inches away from your face and move it slowly towards you
  • The dot should disappear!
cone sensitivities

4 mm

Cone Sensitivities
  • Three cone types (S, M, and L) are roughly blue, green, and red sensors, respectively
  • Their peak sensitivities occur at ~430nm, ~560nm, and ~610nm for the “average” human

Rods & cones, ~1.35 mm fromcenter of fovea

Rods & cones, ~8 mm fromcenter of fovea

Cone sensitivities as a

function of wavelength

color perception

IJVS

Color Perception
  • The cones form a spectral “basis” for visible light; incident spectral distribution differentially excites S,M,L cones, leading to color vision

=

(at each cone site)

X

X

origin of rgb ccd sensors

(Vaytek)

Origin Of RGB CCD Sensors
  • So, in a concrete sense, CCD chips are designed as RGB sensors in order to emulate the human visual system
rgb colour model

(0,0,1) – pure blue

(1,1,1) - white

(0,0,0) - black (hidden)

(0,1,0) – pure green

(1,0,0) pure red

RGB Colour Model
  • Think of R, G, B as color orthobasis
hsv colour model
HSV Colour Model
  • More robust against illumination changes
  • Still must confront noise, specularity etc.
object detection

v

height

width

IO

u

Image coordinates (pixels)

Object Detection
  • Suppose we want to detect an object (e.g., a colored ball) in the field of view
  • We simply need to identify the pixels of some desired colour in the image … right?
real world images

Noise!

Real-World Images

Occluded light source

Specularhighlights

Complexsurfacegeometry

(self-

shadowing)

Mixedpixels

summary
Summary
  • Vision is our most useful sense, but is also the most difficult to replicate
  • Digital cameras have evolved to be powerful, cheap and accurate
  • Artificial vision systems tend to be modelled after the human eye
  • How do we do it?