human computer interaction segmentation n.
Skip this Video
Download Presentation
Human-Computer Interaction Segmentation

Loading in 2 Seconds...

play fullscreen
1 / 38

Human-Computer Interaction Segmentation - PowerPoint PPT Presentation

  • Uploaded on

Human-Computer Interaction Segmentation. Hanyang University Jong-Il Park. Why segment images?. Form large chunks of pixels that can be dealt with together for efficiency because these might represent objects Join up image tokens that together convey information. Grouping.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Human-Computer Interaction Segmentation' - edana

Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
human computer interaction segmentation

Human-Computer InteractionSegmentation

Hanyang University

Jong-Il Park

why segment images
Why segment images?
  • Form large chunks of pixels that can be dealt with together
    • for efficiency
    • because these might represent objects
  • Join up image tokens that together convey information
  • Humans interpret image information collectively
    • in “groups”
    • Eg. Muller-Lyer illusion
  • Shot boundary detection
    • summarize video by
      • find shot boundaries
      • obtain “most representative” frame
  • Background subtraction
    • find “interesting bits” of image by subtracting known background
      • Eg. find person in an office
      • Eg. find cars on a road
  • Interactive segmentation
    • user marks some foreground/background pixels
    • system cuts object out of image
      • useful for image editing, etc.
technique shot boundary detection
Technique: Shot Boundary Detection
  • Find the shots in a sequence of video
    • shot boundaries cause big differences between succeeding frames
  • Strategy:
    • compute interframe distances
    • declare a boundary where these are big
  • Possible distances
    • frame differences
    • histogram differences
    • block comparisons
    • edge differences
technique background subtraction
Technique: Background Subtraction
  • If we know the background, easy to find “interesting bits”
  • Approach:
    • use a moving average to estimate background image
    • subtract from current frame
    • large absolute values are interesting pixels
  • trick: use morphological operations to clean up pixels
interactive segmentation
Interactive segmentation
  • Goals
    • User cuts an object out of one image to paste into another
    • User forms a matte
      • weights between 0 and 1 to mix pixels with background
      • to cope with, say, hair
  • Interactions
    • mark some foreground, background pixels with strokes
    • put a box around foreground
  • Technical problem
    • allocate pixels to foreground/background class
    • consistent with interaction information
    • segments are internally coherent and different from one another
  • Pixels are too small and too detailed a representation
    • for recognition
    • for some kinds of reconstruction
  • Replace with superpixels
    • small groups of pixels that are
      • clumpy
      • like one another
      • a reasonable representation of the underlying pixels
segmentation as clustering
Segmentation as clustering
  • Cluster together (pixels, tokens, etc.) that belong together
  • Agglomerative clustering
    • attach closest to cluster it is closest to
    • repeat
  • Divisive clustering
    • split cluster along best boundary
    • repeat
the watershed algorithm
The watershed algorithm
  • An agglomerative clusterer with a special metric
clustering pixels
Clustering pixels
  • Natural to use k-means
    • represent pixels with
      • intensity vector; color vector; vector of nearby filter responses
      • perhaps position
the mean shift algorithm
The Mean Shift Algorithm
  • Originally intended to find modes in scattered data
  • Strategy
    • start at a promising estimate of mode
    • iterate until the estimate doesn’t change
      • fit a model of probability density to some points near estimate
      • find the peak of this model
  • Model
    • smoothing kernel
    • the update takes a special form
      • shift the mode to a weighted mean of the nearby points
      • hence the name.
clustering with mean shift
Clustering with Mean Shift
  • Model data points as samples from a probability model
    • clusters are associated with modes
    • but it might be hard to find one mode per cluster
      • if there’s more than one mode per cluster, they should be close together
  • Apply mean shift to find modes
    • modes should form small, widely separated clusters
  • Now cluster the modes with (say) agglomerative clusterer
    • easy, because there are small, widely separated clusters
  • Point belongs to cluster that its closest mode belongs to
mean shift segmentation
Mean Shift Segmentation
  • Cluster pixels using mean shift
    • each cluster is a segment
  • Represent with color, position
    • important
      • color distances are not the same as position distances
      • choose one scale for each
evaluating segmenters
Evaluating Segmenters
  • Collect “correct” segmentations
    • from human labellers
    • these may not be perfect, but ...
  • Now apply your segmenter
    • Count
      • % human boundary pixels close to your boundary pixels -- Recall
      • % of your boundary pixels close to human boundary pixels -- Precision