AN ACTIVE VISION APPROACH
Download
1 / 44

AN ACTIVE VISION APPROACH TO OBJECT SEGMENTATION - PowerPoint PPT Presentation


  • 123 Views
  • Updated On :

AN ACTIVE VISION APPROACH TO OBJECT SEGMENTATION. – Paul Fitzpatrick – MIT CSAIL. flexible perception. Humanoid form is general-purpose, mechanically flexible

Related searches for AN ACTIVE VISION APPROACH TO OBJECT SEGMENTATION

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'AN ACTIVE VISION APPROACH TO OBJECT SEGMENTATION' - leander


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Slide1 l.jpg

AN ACTIVE VISION APPROACH

TO OBJECT SEGMENTATION

–Paul Fitzpatrick –

MIT CSAIL


Flexible perception l.jpg
flexible perception

  • Humanoid form is general-purpose, mechanically flexible

  • Robots that really live and work amongst us will need to be as general-purpose and adaptive perceptually as they are mechanically


Flexible visual perception l.jpg
flexible visual perception

  • In robotics, vision is often used to guide manipulation

  • But manipulation can also guide vision

  • Important for…

    • Correction – detecting and recovering from incorrect perception

    • Experimentation – disambiguating inconclusive perception

    • Development – creating or improving perceptual abilities through experience


Flexible visual object perception l.jpg
flexible visual object perception

Tsikos, Bajcsy, 1991

“Segmentation via manipulation”

Simplify cluttered scenes by moving overlapping objects

Sandini et al, 1993

“Vision during action”

Interpret motion during manipulation to deduce object boundaries


Segmentation via manipulation l.jpg
segmentation via manipulation

Tsikos, Bajcsy, 1991


Vision during action l.jpg
vision during action

Sandini et al, 1993


Active segmentation l.jpg
active segmentation

  • Object boundaries are not always easy to detect visually

  • Solution: Cog sweeps arm through ambiguous area

  • Any resulting object motion helps segmentation

  • Robot can learn to recognize and segment object without further contact



Evidence for segmentation l.jpg
evidence for segmentation

  • Areas where motion is observed upon contact

    • classify as ‘foreground’

  • Areas where motion is observed immediately before contact

    • classify as ‘background’

  • Textured areas where no motion was observed

    • classify as ‘background’

  • Textureless areas where no motion was observed

    • no information

  • No need to model the background!


Minimum cut l.jpg
minimum cut

foreground

node

allegiance to foreground

  • “allegiance” = cost of assigning two nodes to different layers (foreground versus background)

pixel-to-pixel

allegiance

pixel nodes

allegiance to background

background

node


Minimum cut11 l.jpg
minimum cut

foreground

node

allegiance to foreground

  • “allegiance” = cost of assigning two nodes to different layers (foreground versus background)

pixel-to-pixel

allegiance

pixel nodes

allegiance to background

background

node


Grouping on synthetic data l.jpg
grouping (on synthetic data)

proposed segmentation

“figure” points

(known motion)

“ground”

points

(stationary,

or gripper)


Point of contact l.jpg
point of contact

1

2

3

4

5

6

7

8

9

10

Motion spreads suddenly, faster than the arm itself  contact

Motion spreads continuously (arm or its shadow)



Segmentation examples l.jpg
segmentation examples

Side

tap

Back

slap

Impact event

Motion caused

(red = novel,

Purple/blue = discounted)

Segmentation

(green/yellow)



Segmentation examples17 l.jpg

car

table

segmentation examples


Boundary fidelity l.jpg

0.2

cube

car

bottle

ball

0.16

0.12

0.08

0.04

0

0

0.05

0.1

0.15

0.2

0.25

0.3

boundary fidelity

measure of anisiotropy

(square root of) second Hu moment

measure of size (area at poking distance)



Active segmentation20 l.jpg
active segmentation

  • Not always practical!

  • No good for objects the robot can view but not touch

  • No good for very big or very small objects

  • But fine for objects the robot is expected to manipulate

Head segmentation

the hard way!


Other approaches l.jpg

human manipulation,

first person perspective

(Charlie Kemp)

human manipulation,

external perspective

(Artur Arsenio)

other approaches

robot manipulation,

first person perspective

(Paul Fitzpatrick,

Giorgio Metta)


From first contact to close encounters l.jpg

segmentation catalog

affordance exploitation

(rolling)

manipulator detection

(robot, human)

edge catalog

object detection

(recognition, localization,

contact-free segmentation)

from first contact to close encounters

active segmentation


Object recognition l.jpg
object recognition

  • Geometry-based

    • Objects and images modeled as set of point/surface/volume elements

    • Example real-time method: store geometric relationships in hash table

  • Appearance-based

    • Objects and images modeled as set of features closer to raw image

    • Example real-time method: use histograms of simple features (e.g. color)


Geometry appearance l.jpg
geometry+appearance

  • Advantages: more selective; fast

  • Disadvantages: edges can be occluded; 2D method

  • Property: no need for offline training

angles + colors + relative sizes

invariant to scale, translation,

In-plane rotation


Details of features l.jpg
details of features

  • Distinguishing elements:

    • Angle between regions (edges)

    • Position of regions relative to their projected intersection point (normalized for scale, orientation)

    • Color at three sample points along line between region centroids

  • Output of feature match:

    • Predicts approximate center and scale of object if match exists

  • Weighting for combining features:

    • Summed at each possible position of center; consistency check for scale

    • Weighted by frequency of occurrence of feature in object examples, and edge length






Just for fun l.jpg
just for fun

look for this…

…in this

result


Multiple objects l.jpg
multiple objects

camera image

implicated edges

found and grouped

response for

each object



First time seeing a ball l.jpg
first time seeing a ball

robot’s

current

view

recognized

object (as seen

during poking)

pokes,

segments

ball

sees ball,

“thinks” it is cube

correctly differentiates

ball and cube



The attention system l.jpg
the attention system

low-level

salience filters

poking

sequencer

motor control

(arm)

object recognition/

localization (wide)

tracker

motor control

(eyes, head, neck)

object recognition/

localization (foveal)

egocentric

map




Finding manipulators l.jpg
finding manipulators

  • Analogous to finding objects

  • Object

    • Definition: physically coherent structure

    • How to find one: poke around and see what moves together

  • Actor

    • Definition: something that acts on objects

    • How to find one: see what pokes objects


Similar human and robot actions l.jpg
similar human and robot actions

Object connects robot and human action


Catching manipulators in the act l.jpg
catching manipulators in the act

manipulator approaches object

contact!




Theoretical goal a virtuous circle l.jpg

use constraint of familiar activity to

discover unfamiliar entity used within it

reveal the structure of unfamiliar activities by tracking familiar entities into and through them

theoretical goal: a virtuous circle

familiar activities

familiar entities (objects, actors, properties, …)


Conclusions l.jpg

Ow!

conclusions

  • Active segmentation can make any segmentation strategy better

  • Ideal for learning about manipulable objects – an important class of object for humanoid robots

  • Doesn’t require a database of objects to be built up by hand

  • (at least, not by human hand)


ad