slide1 l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Object Lesson: Discovering and Learning to Recognize Objects PowerPoint Presentation
Download Presentation
Object Lesson: Discovering and Learning to Recognize Objects

Loading in 2 Seconds...

play fullscreen
1 / 49

Object Lesson: Discovering and Learning to Recognize Objects - PowerPoint PPT Presentation


  • 140 Views
  • Uploaded on

Object Lesson: Discovering and Learning to Recognize Objects. – Paul Fitzpatrick –. MIT CSAIL USA. robots and learning. Robots have access to physics, and physics is a good teacher Physics won’t let you believe the wrong thing for long

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Object Lesson: Discovering and Learning to Recognize Objects' - lizbeth


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

Object Lesson:

Discovering and

Learning to Recognize Objects

– Paul Fitzpatrick –

MIT CSAIL

USA

robots and learning
robots and learning
  • Robots have access to physics, and physics is a good teacher
  • Physics won’t let you believe the wrong thing for long
  • Robot perception should ideally integrate experimentation, or at least learn from (non-fatal) mistakes

forward

seems

safe…

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

a challenge object perception
a challenge: object perception
  • Object perception is a key enabling technology
  • Many components:
    • Object detection
    • Object segmentation
    • Object recognition
  • Typical systems require human-prepared training data; can we use autonomous experimentation?

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

a challenge object perception4
a challenge: object perception
  • Object perception is a key enabling technology
  • Many components:
    • Object detection
    • Object segmentation
    • Object recognition
  • Typical systems require human-prepared training data; can we use autonomous experimentation?

Fruit detection

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

a challenge object perception5
a challenge: object perception
  • Object perception is a key enabling technology
  • Many components:
    • Object detection
    • Object segmentation
    • Object recognition
  • Typical systems require human-prepared training data; can we use autonomous experimentation?

Fruit segmentation

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

a challenge object perception6
a challenge: object perception
  • Object perception is a key enabling technology
  • Many components:
    • Object detection
    • Object segmentation
    • Object recognition
  • Typical systems require human-prepared training data – can’t adapt to new situations autonomously

Fruit recognition

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

talk overview
talk overview
  • Perceiving through experiment
    • Example: active segmentation
  • Learning new perceptual abilities opportunistically
    • Example: detecting edge orientation
    • Example: object detection, segmentation, recognition
  • An architecture for opportunistic learning
    • Example: learning about, and through, search activity

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

talk overview8
talk overview
  • Perceiving through experiment
    • Example: active segmentation
  • Learning new perceptual abilities opportunistically
    • Example: detecting edge orientation
    • Example: object detection, segmentation, recognition
  • An architecture for opportunistic learning
    • Example: learning about, and through, search activity

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

active segmentation
active segmentation
  • Object boundaries are not always easy to detect visually
  • Solution: Cog sweeps arm through ambiguous area
  • Any resulting object motion helps segmentation
  • Robot can learn to recognize and segment object without further contact

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

active segmentation10

1

2

3

4

5

6

7

8

9

10

active segmentation
  • Detect contact between arm and object using fast, coarse processing on optic flow signal
  • Do detailed comparison of motion immediately before and after collision
  • Use minimum-cut algorithm to generate best segmentation

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

active segmentation11
active segmentation

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

active segmentation12
active segmentation
  • Not always practical!
  • No good for objects the robot can view but not touch
  • No good for very big or very small objects
  • But fine for objects the robot is expected to manipulate

Head segmentation

the hard way!

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

listening to physics

car

table

listening to physics
  • Active segmentation is useful even if robot normally depends on other segmentation cues (color, stereo)
  • If passive segmentation is incorrect and robot fails to grasp object, active segmentation can use even clumsy collision to get truth
  • Seems silly not to use this feedback from physics and keep making the same mistake

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

talk overview14
talk overview
  • Perceiving through experiment
    • Example: active segmentation
  • Learning new perceptual abilities opportunistically
    • Example: detecting edge orientation
    • Example: object detection, segmentation, recognition
  • An architecture for opportunistic learning
    • Example: learning about, and through, search activity

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

talk overview15
talk overview
  • Perceiving through experiment
    • Example: active segmentation
  • Learning new perceptual abilities opportunistically
    • Example: detecting edge orientation
    • Example: object detection, segmentation, recognition
  • An architecture for opportunistic learning
    • Example: learning about, and through, search activity

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

opportunistic learning
opportunistic learning
  • To begin with, Cog has three categories of perceptual abilities:
    • Judgements it can currently make (e.g. about color, motion, time)
    • Judgements it can sometimes make (e.g. boundary of object, identity of object)
    • Judgements it cannot currently make (e.g. counting objects)
  • With opportunistic learning, the robot takes judgements in the sometimes category and works to promote them to the can category by finding reliable correlated features that are more frequently available
    • Example: analysis of boundaries detected through motion yields purely visual features that are predictive of edge orientation
    • Example: assuming a non-hostile environment (some continuity in time and space) segmented views of objects can be grouped and purely visual features inferred that are characteristic of distinct objects

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

training a model of edge appearance
training a model of edge appearance
  • Robot initially only perceives oriented edges through active segmentation procedure
  • Robot collects samples of edge appearance along boundary, and builds a look-up table from appearance to orientation angle
  • Now can perceive orientation directly
  • This is often built in, but it doesn’t have to be

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

most frequent samples
most frequent samples

1st

21st

41st

61st

81st

101st

121st

141st

161st

181st

201st

221st

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

some tests
some tests

Red = horizontalGreen = vertical

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

natural images
natural images

00

22.50

900

22.50

450

22.50

450

22.50

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

talk overview21
talk overview
  • Perceiving through experiment
    • Example: active segmentation
  • Learning new perceptual abilities opportunistically
    • Example: detecting edge orientation
    • Example: object detection, segmentation, recognition
  • An architecture for opportunistic learning
    • Example: learning about, and through, search activity

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

on to object detection
on to object detection…

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

on to object detection23
on to object detection…

look for this…

…in this

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

on to object detection24
on to object detection…

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

on to object detection25
on to object detection…

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

on to object detection26
on to object detection…

geometry alone

geometry + color

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

other examples
other examples

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

other examples28
other examples

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

other examples29
other examples

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

just for fun
just for fun

look for this…

…in this

result

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

real object in real images
real object in real images

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

yellow on yellow
yellow on yellow

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

multiple objects
multiple objects

camera image

implicated edges

found and grouped

response for

each object

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

attention
attention

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

first time seeing a ball
first time seeing a ball

robot’s

current

view

recognized

object (as seen

during poking)

pokes,

segments

ball

sees ball,

“thinks” it is cube

correctly differentiates

ball and cube

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

open object recognition
open object recognition

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

talk overview37
talk overview
  • Perceiving through experiment
    • Example: active segmentation
  • Learning new perceptual abilities opportunistically
    • Example: detecting edge orientation
    • Example: object detection, segmentation, recognition
  • An architecture for opportunistic learning
    • Example: learning about, and through, search activity

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

talk overview38
talk overview
  • Perceiving through experiment
    • Example: active segmentation
  • Learning new perceptual abilities opportunistically
    • Example: detecting edge orientation
    • Example: object detection, segmentation, recognition
  • An architecture for opportunistic learning
    • Example: learning about, and through, search activity

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

physically grounded perception

object segmentation

affordance exploitation

[with Giorgio Metta]

(rolling)

manipulator detection

(robot, human)

edge catalog

object detection

(recognition, localization,

contact-free segmentation)

physically-grounded perception

active segmentation

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

socially grounded perception
socially-grounded perception

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

socially grounded perception41
socially-grounded perception

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

opportunistic architecture a virtuous circle

use constraint of familiar activity to

discover unfamiliar entity used within it

reveal the structure of unfamiliar activities by tracking familiar entities into and through them

opportunistic architecture: a virtuous circle

familiar activities

familiar entities (objects, actors, properties, …)

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

a virtuous circle
a virtuous circle

poking, chatting

discover car, ball, and cube through poking; discover their names through chatting

car, ball, cube, and their names

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

a virtuous circle44
a virtuous circle

poking, chatting, search

follow named objects into search activity, and observe the structure of search

car, ball, cube, and their names

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

learning about search
learning about search

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

a virtuous circle46
a virtuous circle

poking, chatting, search

follow named objects into search activity, and observe the structure of search

car, ball, cube, and their names

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

a virtuous circle47
a virtuous circle

poking, chatting, searching

discover novel object through poking, learn its name (e.g. ‘toma’) indirectly during search

car, ball, cube, toma, and their names

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

finding the toma
finding the toma

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003

conclusion why do this
conclusion: why do this?
  • The quest for truly flexible robots
  • Humanoid form is general-purpose, mechanically flexible
  • Robots that really live and work amongst us will need to be as general-purpose and adaptive perceptually as they are mechanically

Paul Fitzpatrick, MIT CSAIL, Humanoids 2003