human computer interaction segmentation
Download
Skip this Video
Download Presentation
Human-Computer Interaction Segmentation

Loading in 2 Seconds...

play fullscreen
1 / 38

Human-Computer Interaction Segmentation - PowerPoint PPT Presentation


  • 52 Views
  • Uploaded on

Human-Computer Interaction Segmentation. Hanyang University Jong-Il Park. Why segment images?. Form large chunks of pixels that can be dealt with together for efficiency because these might represent objects Join up image tokens that together convey information. Grouping.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Human-Computer Interaction Segmentation' - edana


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
human computer interaction segmentation

Human-Computer InteractionSegmentation

Hanyang University

Jong-Il Park

why segment images
Why segment images?
  • Form large chunks of pixels that can be dealt with together
    • for efficiency
    • because these might represent objects
  • Join up image tokens that together convey information
grouping
Grouping
  • Humans interpret image information collectively
    • in “groups”
    • Eg. Muller-Lyer illusion
applications
Applications
  • Shot boundary detection
    • summarize video by
      • find shot boundaries
      • obtain “most representative” frame
  • Background subtraction
    • find “interesting bits” of image by subtracting known background
      • Eg. find person in an office
      • Eg. find cars on a road
  • Interactive segmentation
    • user marks some foreground/background pixels
    • system cuts object out of image
      • useful for image editing, etc.
technique shot boundary detection
Technique: Shot Boundary Detection
  • Find the shots in a sequence of video
    • shot boundaries cause big differences between succeeding frames
  • Strategy:
    • compute interframe distances
    • declare a boundary where these are big
  • Possible distances
    • frame differences
    • histogram differences
    • block comparisons
    • edge differences
technique background subtraction
Technique: Background Subtraction
  • If we know the background, easy to find “interesting bits”
  • Approach:
    • use a moving average to estimate background image
    • subtract from current frame
    • large absolute values are interesting pixels
  • trick: use morphological operations to clean up pixels
interactive segmentation
Interactive segmentation
  • Goals
    • User cuts an object out of one image to paste into another
    • User forms a matte
      • weights between 0 and 1 to mix pixels with background
      • to cope with, say, hair
  • Interactions
    • mark some foreground, background pixels with strokes
    • put a box around foreground
  • Technical problem
    • allocate pixels to foreground/background class
    • consistent with interaction information
    • segments are internally coherent and different from one another
superpixels
Superpixels
  • Pixels are too small and too detailed a representation
    • for recognition
    • for some kinds of reconstruction
  • Replace with superpixels
    • small groups of pixels that are
      • clumpy
      • like one another
      • a reasonable representation of the underlying pixels
segmentation as clustering
Segmentation as clustering
  • Cluster together (pixels, tokens, etc.) that belong together
  • Agglomerative clustering
    • attach closest to cluster it is closest to
    • repeat
  • Divisive clustering
    • split cluster along best boundary
    • repeat
the watershed algorithm
The watershed algorithm
  • An agglomerative clusterer with a special metric
clustering pixels
Clustering pixels
  • Natural to use k-means
    • represent pixels with
      • intensity vector; color vector; vector of nearby filter responses
      • perhaps position
the mean shift algorithm
The Mean Shift Algorithm
  • Originally intended to find modes in scattered data
  • Strategy
    • start at a promising estimate of mode
    • iterate until the estimate doesn’t change
      • fit a model of probability density to some points near estimate
      • find the peak of this model
  • Model
    • smoothing kernel
    • the update takes a special form
      • shift the mode to a weighted mean of the nearby points
      • hence the name.
clustering with mean shift
Clustering with Mean Shift
  • Model data points as samples from a probability model
    • clusters are associated with modes
    • but it might be hard to find one mode per cluster
      • if there’s more than one mode per cluster, they should be close together
  • Apply mean shift to find modes
    • modes should form small, widely separated clusters
  • Now cluster the modes with (say) agglomerative clusterer
    • easy, because there are small, widely separated clusters
  • Point belongs to cluster that its closest mode belongs to
mean shift segmentation
Mean Shift Segmentation
  • Cluster pixels using mean shift
    • each cluster is a segment
  • Represent with color, position
    • important
      • color distances are not the same as position distances
      • choose one scale for each
evaluating segmenters
Evaluating Segmenters
  • Collect “correct” segmentations
    • from human labellers
    • these may not be perfect, but ...
  • Now apply your segmenter
    • Count
      • % human boundary pixels close to your boundary pixels -- Recall
      • % of your boundary pixels close to human boundary pixels -- Precision
ad