Today image segmentation
1 / 32

Today: Image Segmentation - PowerPoint PPT Presentation

  • Uploaded on

Today: Image Segmentation. Image Segmentation Techniques Snakes Scissors Graph Cuts Mean Shift Wednesday (2/28) Texture analysis and synthesis Multiple hypothesis tracking Final Project Presentations Q: one two-hour slot or two one-hour slots? March 12 or 14?. From Images to Objects.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Today: Image Segmentation' - lesley-crawford

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Today image segmentation
Today: Image Segmentation

  • Image Segmentation Techniques

    • Snakes

    • Scissors

    • Graph Cuts

    • Mean Shift

  • Wednesday (2/28)

    • Texture analysis and synthesis

    • Multiple hypothesis tracking

  • Final Project Presentations

    • Q: one two-hour slot or two one-hour slots?

    • March 12 or 14?

From images to objects
From Images to Objects

  • What Defines an Object?

    • Subjective problem, but has been well-studied

    • Gestalt Laws seek to formalize this

      • proximity, similarity, continuation, closure, common fate

      • see notes by Steve Joordens, U. Toronto

Image segmentation
Image Segmentation

  • Many Approaches Proposed

    • color cues

    • region cues

    • contour cues

  • Today: Three Approaches

    • Mean Shift (color-based)

      • D. Comaniciu, P. Meer, Robust Analysis of Feature Spaces: Color Image Segmentation, Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR'97), 1997, 750-755.

    • Normalized Cuts (region-based)

      • J. Shi and J. Malik, Normalized Cuts and Image Segmentation, IEEE Conf. Computer Vision and Pattern Recognition(CVPR), 1997

    • Snakes and Scissors (contour-based)

      • M. Kass, A. Witkin, and D. Terzopoulos, Snakes: Active contour models. International Journal of Computer Vision. v. 1, n. 4, pp. 321-331, 1987.

      • E. N. Mortensen and W. A. Barrett, Intelligent Scissors for Image Composition, in ACM Computer Graphics (SIGGRAPH `95), pp. 191-198, 1995

Color based segmentation
Color-Based Segmentation

  • Segmentation by Histogram Processing

    • Given image with N colors, choose K

    • Each of the K colors defines a region

      • not necessarily contiguous

    • Performed by computing color histogram, looking for modes

    • This is what happens when you downsample image color range, for instance in Photoshop

Finding modes in a histogram
Finding Modes in a Histogram

  • How Many Modes Are There?

    • Easy to see, hard to compute

Mean shift comaniciu meer
Mean Shift [Comaniciu & Meer]

  • Iterative Mode Search

    • Initialize random seed, and fixed window

    • Calculate center of gravity of the window (the “mean”)

    • Translate the search window to the mean

    • Repeat Step 2 until convergence

More Examples:

Region based segmentation
Region-Based Segmentation

  • We Want Regions

    • why not build this in as a constraint?

Images as graphs shi malik
Images as Graphs [Shi & Malik]

  • Graph G = (V, E, W)

    • node for every pixel

    • edge between every pair of pixels, p,q

    • weight wpq for each edge

      • wpq measures similarity

        • similarity: difference in color and position





Segmentation by graph cuts


Segmentation by Graph Cuts

  • Break Graph into Segments

    • Delete edges that cross between segments

    • Easiest to break edges that have low weight:

      • similar pixels should be in the same segments

      • dissimilar pixels should be in different segments




Cuts in a graph

  • Normalized Cut

    • a cut penalizes large segments

    • fix by normalizing for size of segments



Cuts in a graph

  • Edge Cut

    • set of edges whose removal makes a graph disconnected

    • cost of a cut:



The normalized cut ncut criterion
The Normalized Cut (NCut) criterion

  • Given a Graph G = (V, E, W)

    • Find A ½ V that minimizes


Eigenvalue problem
Eigenvalue Problem

  • After lot’s of math, we get:

  • This is a Rayleigh Quotient

    • Solution given by “generalized” eigenvalue problem:

    • Solved by converting to standard eigenvalue problem:

  • Subtleties

    • optimal solution is second smallest eigenvector

    • gives real result—must convert into discrete values of y

Interpretation as a dynamical system
Interpretation as a Dynamical System

  • Weights are Springs

    • eigenvectors correspond to vibration modes

Interpretation as a dynamical system1
Interpretation as a Dynamical System

  • Weights are Springs

    • eigenvectors correspond to vibration modes

Graph cuts formulations

Graph Cuts Formulations

  • We’ve seen graph cuts before

    • Stereo (Y. Boykov, O. Veksler, and Ramin Zabih, Fast Approximate Energy Minimization via Graph Cuts)

Segmentation using edges
Segmentation Using Edges

  • Edges  Segments

    • Spurious edges

    • Missing edges

How could user-interaction help?

Snakes kass et al
Snakes [Kass et al.]

Snakes energy minimization
Snakes: Energy Minimization

  • Ingredients:

    • A contour

    • An energy function

  • Try to make the contour snap to edges

    • minimize Esnake

  • Issues

    • contour representation

    • energy function

    • minimization algorithm


  • Contour representation

    • splines [Kass et al., IJCV 89, original formulation]

    • discrete set of points [Williams & Shah, CVGIP 55(1), 92]

    • implicit function [Malladi et al., 95 (see readings]

  • Energy functional

    • Variations on edge term (gradient magnitude, laplacian)

    • Minor variations in smoothness terms (first/second order)

    • Volume expansion (balloon) or contraction

    • Deformable templates [Yuille, IJCV 92]

    • Learned shape models [Isard & Blake, others]

  • Minimization algorithm

    • Numerical Euler integration [e.g., Kass]

    • Greedy algorithms [e.g., Williams]

    • Level set evolution [e.g., Malladi]

Issues scale
Issues: Scale

Lowpass Images

Bandpass Images

Pyramid snakes for coarse-to-fine processing

Problem with snakes
Problem with Snakes

  • You can’t control the shape evolution*

  • * to be fair: Kass et al described how you can add forces to push and pull the snake as desired, but still kind of kludgy

Intelligent scissors mortensen barrett
Intelligent Scissors [Mortensen & Barrett]

Intelligent scissors
Intelligent Scissors

  • Approach answers a basic question

    • Q: how to find a path from seed to mouse that follows object boundary as closely as possible?

    • A: define a path that stays as close as possible to edges

Intelligent scissors1
Intelligent Scissors

  • Basic Idea

    • Define edge score for each pixel

      • edge pixels have low cost

    • Find lowest cost path from seed to mouse



  • Questions

    • How to define costs?

    • How to find the path?

Defining costs
Defining Costs

  • Link Costs

    • Costs are assigned to a pair of adjacent pixels (a link)




fz(q) = 0 if Laplacian is 0 at q

fz(q) = 1 otherwise

fd(q) measures difference in

gradient direction

Path search
Path Search

  • Graph Search Algorithm

    • Computes minimum cost path from seed to allotherpixels