images as graphs n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Images as graphs PowerPoint Presentation
Download Presentation
Images as graphs

Loading in 2 Seconds...

play fullscreen
1 / 54

Images as graphs - PowerPoint PPT Presentation


  • 105 Views
  • Uploaded on

Images as graphs. Fully-connected graph node for every pixel link between every pair of pixels, p , q similarity w ij for each link. i. w ij. c. j. Source: Seitz. Segmentation by Graph Cuts. w. Break Graph into Segments Delete links that cross between segments

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Images as graphs' - caron


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
images as graphs
Images as graphs
  • Fully-connected graph
    • node for every pixel
    • link between every pair of pixels, p,q
    • similarity wij for each link

i

wij

c

j

Source: Seitz

segmentation by graph cuts
Segmentation by Graph Cuts

w

  • Break Graph into Segments
    • Delete links that cross between segments
    • Easiest to break links that have low cost (low similarity)
      • similar pixels should be in the same segments
      • dissimilar pixels should be in different segments

A

B

C

Source: Seitz

cuts in a graph
Cuts in a graph
  • Link Cut
    • set of links whose removal makes a graph disconnected
    • cost of a cut:

B

A

  • One idea: Find minimum cut
    • gives you a segmentation
    • fast algorithms exist for doing this

Source: Seitz

cuts in a graph1
Cuts in a graph

B

A

  • Normalized Cut
    • a cut penalizes large segments
    • fix by normalizing for size of segments
    • volume(A) = sum of costs of all edges that touch A

Source: Seitz

finding minimum normalized cut
Finding Minimum Normalized-Cut
  • Finding the Minimum Normalized-Cut is NP-Hard.
  • Polynomial Approximations are generally used for segmentation

* Slide from Khurram Hassan-Shafique CAP5415 Computer Vision 2003

finding minimum normalized cut1
Finding Minimum Normalized-Cut

* From Khurram Hassan-Shafique CAP5415 Computer Vision 2003

finding minimum normalized cut2
Finding Minimum Normalized-Cut
  • It can be shown that

such that

  • If y is allowed to take real values then the minimization can be done by solving the generalized eigenvalue system

* Slide from Khurram Hassan-Shafique CAP5415 Computer Vision 2003

algorithm
Algorithm
  • Compute matrices W & D
  • Solve for eigen vectors with the smallest eigen values
  • Use the eigen vector with second smallest eigen value to bipartition the graph
  • Recursively partition the segmented parts if necessary.

* Slide from Khurram Hassan-Shafique CAP5415 Computer Vision 2003

recursive normalized cuts
Recursive normalized cuts
  • Given an image or image sequence, set up a weighted graph: G=(V, E)
    • Vertex for each pixel
    • Edge weight for nearby pairs of pixels
  • Solve for eigenvectors with the smallest eigenvalues: (D − W)y = λDy
    • Use the eigenvector with the second smallest eigenvalue to bipartition the graph
    • Note: this is an approximation

4. Recursively repartition the segmented parts if necessary

http://www.cs.berkeley.edu/~malik/papers/SM-ncut.pdf

Details:

markov random fields
Markov Random Fields

Node yi: pixel label

Edge: constrained pairs

Cost to assign a label to each pixel

Cost to assign a pair of labels to connected pixels

solving mrfs with graph cuts
Solving MRFs with graph cuts

Source (Label 0)

Cost to assign to 0

Cost to split nodes

Cost to assign to 1

Sink (Label 1)

solving mrfs with graph cuts1
Solving MRFs with graph cuts

Source (Label 0)

Cost to assign to 0

Cost to split nodes

Cost to assign to 1

Sink (Label 1)

graph cuts boykov and jolly 2001

Foreground (source)

Min Cut

Background(sink)

Cut: separating source and sink; Energy: collection of edges

Min Cut: Global minimal enegry in polynomial time

Graph cutsBoykov and Jolly (2001)

Image

Source: Rother

optimization formulation boykov jolly 01
Optimization Formulation - Boykov & Jolly ‘01
  • A – Proposed Segmentation
  • E(A) – Overall Energy
  • R(A) – Degree to which pixels fits model
  • B(A) – Degree to which the cuts breaks up similar pixels
  • - Balance A() and B()
  • Goal: Find Segmentation, A, which minimizes E(A)
link weights
Link Weights
  • Pixel links based on color/intensity similarities
  • Source/Target links based on histogram models of fore/background
grab cuts and graph cuts
Grab cuts and graph cuts

Magic Wand(198?)

Intelligent ScissorsMortensen and Barrett (1995)

GrabCut

User Input

Result

Regions

Regions & Boundary

Boundary

Source: Rother

colour model
Colour Model

Gaussian Mixture Model (typically 5-8 components)

R

R

Iterated graph cut

Foreground &Background

Foreground

G

Background

G

Background

Source: Rother

difficult examples

GrabCut – Interactive Foreground Extraction11

Difficult Examples

Camouflage &

Low Contrast

Fine structure

Harder Case

Initial Rectangle

InitialResult

border matting

GrabCut – Interactive Foreground Extraction23

Border Matting

Hard Segmentation

Automatic Trimap

Soft Segmentation

to

comparison

GrabCut – Interactive Foreground Extraction24

Comparison

With no regularisation over alpha

Input

Knockout 2Photoshop Plug-In

Bayes MattingChuang et. al. (2001)

Shum et. al. (2004):Coherence matting in “Pop-up light fields”

natural image matting

GrabCut – Interactive Foreground Extraction25

Natural Image Matting

Mean Colour Foreground

Mean ColourBackground

Solve

Ruzon and Tomasi (2000):Alpha estimation in natural images

border matting1

GrabCut – Interactive Foreground Extraction26

Border Matting

Foreground

Noisy alpha-profile

1

Mix

Back-ground

0

Foreground

Background

Mix

Fit a smooth alpha-profile with parameters

dynamic programming

GrabCut – Interactive Foreground Extraction27

Dynamic Programming

t+1

t

DP

Result using DP Border Matting

Regularisation

Noisy alpha-profile

grabcut border matting colour

GrabCut – Interactive Foreground Extraction28

GrabCut BorderMatting -Colour
  • Compute MAP of p(F|C,alpha) (marginalize over B)
  • To avoid colour bleeding use colour stealing (“exemplar based inpainting” – Patches do not work)

Grabcut Border Matting

[Chuang et al. ‘01]