Human computer interaction segmentation
1 / 38

Human-Computer Interaction Segmentation - PowerPoint PPT Presentation

  • Uploaded on

Human-Computer Interaction Segmentation. Hanyang University Jong-Il Park. Why segment images?. Form large chunks of pixels that can be dealt with together for efficiency because these might represent objects Join up image tokens that together convey information. Grouping.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Human-Computer Interaction Segmentation' - edana

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Human computer interaction segmentation

Human-Computer InteractionSegmentation

Hanyang University

Jong-Il Park

Why segment images
Why segment images?

  • Form large chunks of pixels that can be dealt with together

    • for efficiency

    • because these might represent objects

  • Join up image tokens that together convey information


  • Humans interpret image information collectively

    • in “groups”

    • Eg. Muller-Lyer illusion


  • Shot boundary detection

    • summarize video by

      • find shot boundaries

      • obtain “most representative” frame

  • Background subtraction

    • find “interesting bits” of image by subtracting known background

      • Eg. find person in an office

      • Eg. find cars on a road

  • Interactive segmentation

    • user marks some foreground/background pixels

    • system cuts object out of image

      • useful for image editing, etc.

Technique shot boundary detection
Technique: Shot Boundary Detection

  • Find the shots in a sequence of video

    • shot boundaries cause big differences between succeeding frames

  • Strategy:

    • compute interframe distances

    • declare a boundary where these are big

  • Possible distances

    • frame differences

    • histogram differences

    • block comparisons

    • edge differences

Technique background subtraction
Technique: Background Subtraction

  • If we know the background, easy to find “interesting bits”

  • Approach:

    • use a moving average to estimate background image

    • subtract from current frame

    • large absolute values are interesting pixels

  • trick: use morphological operations to clean up pixels

Interactive segmentation
Interactive segmentation

  • Goals

    • User cuts an object out of one image to paste into another

    • User forms a matte

      • weights between 0 and 1 to mix pixels with background

      • to cope with, say, hair

  • Interactions

    • mark some foreground, background pixels with strokes

    • put a box around foreground

  • Technical problem

    • allocate pixels to foreground/background class

    • consistent with interaction information

    • segments are internally coherent and different from one another


  • Pixels are too small and too detailed a representation

    • for recognition

    • for some kinds of reconstruction

  • Replace with superpixels

    • small groups of pixels that are

      • clumpy

      • like one another

      • a reasonable representation of the underlying pixels

Segmentation as clustering
Segmentation as clustering

  • Cluster together (pixels, tokens, etc.) that belong together

  • Agglomerative clustering

    • attach closest to cluster it is closest to

    • repeat

  • Divisive clustering

    • split cluster along best boundary

    • repeat

The watershed algorithm
The watershed algorithm

  • An agglomerative clusterer with a special metric

Clustering pixels
Clustering pixels

  • Natural to use k-means

    • represent pixels with

      • intensity vector; color vector; vector of nearby filter responses

      • perhaps position

The mean shift algorithm
The Mean Shift Algorithm

  • Originally intended to find modes in scattered data

  • Strategy

    • start at a promising estimate of mode

    • iterate until the estimate doesn’t change

      • fit a model of probability density to some points near estimate

      • find the peak of this model

  • Model

    • smoothing kernel

    • the update takes a special form

      • shift the mode to a weighted mean of the nearby points

      • hence the name.

Clustering with mean shift
Clustering with Mean Shift

  • Model data points as samples from a probability model

    • clusters are associated with modes

    • but it might be hard to find one mode per cluster

      • if there’s more than one mode per cluster, they should be close together

  • Apply mean shift to find modes

    • modes should form small, widely separated clusters

  • Now cluster the modes with (say) agglomerative clusterer

    • easy, because there are small, widely separated clusters

  • Point belongs to cluster that its closest mode belongs to

Mean shift segmentation
Mean Shift Segmentation

  • Cluster pixels using mean shift

    • each cluster is a segment

  • Represent with color, position

    • important

      • color distances are not the same as position distances

      • choose one scale for each

Evaluating segmenters
Evaluating Segmenters

  • Collect “correct” segmentations

    • from human labellers

    • these may not be perfect, but ...

  • Now apply your segmenter

    • Count

      • % human boundary pixels close to your boundary pixels -- Recall

      • % of your boundary pixels close to human boundary pixels -- Precision