today image segmentation n.
Skip this Video
Download Presentation
Today: Image Segmentation

Loading in 2 Seconds...

play fullscreen
1 / 32

Today: Image Segmentation - PowerPoint PPT Presentation

  • Uploaded on

Today: Image Segmentation. Image Segmentation Techniques Snakes Scissors Graph Cuts Mean Shift Wednesday (2/28) Texture analysis and synthesis Multiple hypothesis tracking Final Project Presentations Q: one two-hour slot or two one-hour slots? March 12 or 14?. From Images to Objects.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Today: Image Segmentation' - lesley-crawford

Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
today image segmentation
Today: Image Segmentation
  • Image Segmentation Techniques
    • Snakes
    • Scissors
    • Graph Cuts
    • Mean Shift
  • Wednesday (2/28)
    • Texture analysis and synthesis
    • Multiple hypothesis tracking
  • Final Project Presentations
    • Q: one two-hour slot or two one-hour slots?
    • March 12 or 14?
from images to objects
From Images to Objects
  • What Defines an Object?
    • Subjective problem, but has been well-studied
    • Gestalt Laws seek to formalize this
      • proximity, similarity, continuation, closure, common fate
      • see notes by Steve Joordens, U. Toronto
image segmentation
Image Segmentation
  • Many Approaches Proposed
    • color cues
    • region cues
    • contour cues
  • Today: Three Approaches
    • Mean Shift (color-based)
      • D. Comaniciu, P. Meer, Robust Analysis of Feature Spaces: Color Image Segmentation, Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR'97), 1997, 750-755.
    • Normalized Cuts (region-based)
      • J. Shi and J. Malik, Normalized Cuts and Image Segmentation, IEEE Conf. Computer Vision and Pattern Recognition(CVPR), 1997
    • Snakes and Scissors (contour-based)
      • M. Kass, A. Witkin, and D. Terzopoulos, Snakes: Active contour models. International Journal of Computer Vision. v. 1, n. 4, pp. 321-331, 1987.
      • E. N. Mortensen and W. A. Barrett, Intelligent Scissors for Image Composition, in ACM Computer Graphics (SIGGRAPH `95), pp. 191-198, 1995
color based segmentation
Color-Based Segmentation
  • Segmentation by Histogram Processing
    • Given image with N colors, choose K
    • Each of the K colors defines a region
      • not necessarily contiguous
    • Performed by computing color histogram, looking for modes
    • This is what happens when you downsample image color range, for instance in Photoshop
finding modes in a histogram
Finding Modes in a Histogram
  • How Many Modes Are There?
    • Easy to see, hard to compute
mean shift comaniciu meer
Mean Shift [Comaniciu & Meer]
  • Iterative Mode Search
    • Initialize random seed, and fixed window
    • Calculate center of gravity of the window (the “mean”)
    • Translate the search window to the mean
    • Repeat Step 2 until convergence

More Examples:

region based segmentation
Region-Based Segmentation
  • We Want Regions
    • why not build this in as a constraint?
images as graphs shi malik
Images as Graphs [Shi & Malik]
  • Graph G = (V, E, W)
    • node for every pixel
    • edge between every pair of pixels, p,q
    • weight wpq for each edge
      • wpq measures similarity
        • similarity: difference in color and position





segmentation by graph cuts


Segmentation by Graph Cuts
  • Break Graph into Segments
    • Delete edges that cross between segments
    • Easiest to break edges that have low weight:
      • similar pixels should be in the same segments
      • dissimilar pixels should be in different segments




cuts in a graph

Normalized Cut

    • a cut penalizes large segments
    • fix by normalizing for size of segments



Cuts in a graph
  • Edge Cut
    • set of edges whose removal makes a graph disconnected
    • cost of a cut:



the normalized cut ncut criterion
The Normalized Cut (NCut) criterion
  • Given a Graph G = (V, E, W)
    • Find A ½ V that minimizes


eigenvalue problem
Eigenvalue Problem
  • After lot’s of math, we get:
  • This is a Rayleigh Quotient
    • Solution given by “generalized” eigenvalue problem:
    • Solved by converting to standard eigenvalue problem:
  • Subtleties
    • optimal solution is second smallest eigenvector
    • gives real result—must convert into discrete values of y
interpretation as a dynamical system
Interpretation as a Dynamical System
  • Weights are Springs
    • eigenvectors correspond to vibration modes
interpretation as a dynamical system1
Interpretation as a Dynamical System
  • Weights are Springs
    • eigenvectors correspond to vibration modes
graph cuts formulations

The formulation is different however

    • In Boykov approach, each pixel was connected to a label node
    • Boykov approach gives better optimality guarantees, but requires discrete labels
Graph Cuts Formulations
  • We’ve seen graph cuts before
    • Stereo (Y. Boykov, O. Veksler, and Ramin Zabih, Fast Approximate Energy Minimization via Graph Cuts)
segmentation using edges
Segmentation Using Edges
  • Edges  Segments
    • Spurious edges
    • Missing edges

How could user-interaction help?

snakes energy minimization
Snakes: Energy Minimization
  • Ingredients:
    • A contour
    • An energy function
  • Try to make the contour snap to edges
    • minimize Esnake
  • Issues
    • contour representation
    • energy function
    • minimization algorithm
  • Contour representation
    • splines [Kass et al., IJCV 89, original formulation]
    • discrete set of points [Williams & Shah, CVGIP 55(1), 92]
    • implicit function [Malladi et al., 95 (see readings]
  • Energy functional
    • Variations on edge term (gradient magnitude, laplacian)
    • Minor variations in smoothness terms (first/second order)
    • Volume expansion (balloon) or contraction
    • Deformable templates [Yuille, IJCV 92]
    • Learned shape models [Isard & Blake, others]
  • Minimization algorithm
    • Numerical Euler integration [e.g., Kass]
    • Greedy algorithms [e.g., Williams]
    • Level set evolution [e.g., Malladi]
issues scale
Issues: Scale

Lowpass Images

Bandpass Images

Pyramid snakes for coarse-to-fine processing

problem with snakes
Problem with Snakes
  • You can’t control the shape evolution*
  • * to be fair: Kass et al described how you can add forces to push and pull the snake as desired, but still kind of kludgy
intelligent scissors
Intelligent Scissors
  • Approach answers a basic question
    • Q: how to find a path from seed to mouse that follows object boundary as closely as possible?
    • A: define a path that stays as close as possible to edges
intelligent scissors1
Intelligent Scissors
  • Basic Idea
    • Define edge score for each pixel
      • edge pixels have low cost
    • Find lowest cost path from seed to mouse



  • Questions
    • How to define costs?
    • How to find the path?
defining costs
Defining Costs
  • Link Costs
    • Costs are assigned to a pair of adjacent pixels (a link)




fz(q) = 0 if Laplacian is 0 at q

fz(q) = 1 otherwise

fd(q) measures difference in

gradient direction

path search
Path Search
  • Graph Search Algorithm
    • Computes minimum cost path from seed to allotherpixels