1 / 39

Graph Cut Selections

Graph Cut Selections. Stephen J. Guy. GrabCut – Interactive Foreground Extraction 1. Photomontage. Agenda. Background Segmentation Intro Graph Cut for Segmentation (Boykov & Jolly ‘01) Copying Lazy Snapping (Li et al. ’04) GrabCut (Rother et al. ’04) Other

lana
Download Presentation

Graph Cut Selections

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Graph Cut Selections Stephen J. Guy

  2. GrabCut – Interactive Foreground Extraction1 Photomontage

  3. Agenda • Background • Segmentation Intro • Graph Cut for Segmentation (Boykov & Jolly ‘01) • Copying • Lazy Snapping (Li et al. ’04) • GrabCut (Rother et al. ’04) • Other • Texture & Video Synthesis (Kwatra et al. ’04)

  4. Segmentation • Goal • Splitting Image into similar, meaningful(?) regions • E.g. Clustering, Edge Detection, *Graph Cuts* GrabCut “SuperPixels” Rother et al. 2004 X. Ren and J. Malik. Learning a classification model for segmentation. ICCV 2003.

  5. Optimization Formulation - Boykov & Jolly ‘01 • A – Proposed Segmentation • E(A) – Overall Energy • R(A) – Degree to which pixels fits model • B(A) – Degree to which the cuts breaks up similar pixels • - Balance A() and B() • Goal: Find Segmentation, A, which minimizes E(A)

  6. Graph Cut Basics • Image As Graph • Each pixel is a node • Create links between each pixel • More nodes/links possible (e.g. foreground node) • Graph Cuts • Assign energy for each link based on similarity • Break the links with least energy, but • Require each pixel is still connected to exactly one foreground or background node • Benefits • No Topological constraints on regions (can be concave, unconnected, etc.) • Fast (polynomial time) solution – Ford-Fulkerson Algorithm

  7. Graph Cut Overview

  8. Link Weights • Pixel links based on color/intensity similarities • Source/Target links based on histogram models of fore/background

  9. Gestalt Example • Effect of • Recall:

  10. Lazy Snapping

  11. Lazy Snapping Overview • Main Steps: • Mark strokes as foreground & background • Perform Boykov & Jolly style graphcut • *Edit Boundary* • Familiar Formulation • Minimize Gibbs Energy: • E1(X) – Cluster (K-means) known foreground/background colors to find representative colors • E2(X) – Similar to Boykov & Jolly

  12. Lazy Snapping – Issue • Too Slow to allow for real-time refinement • Solution? Superpixels!

  13. Lazy Snaping – Speedup

  14. Boundary Editing • Graph Cut is only step 1 • Allow user to edit boundary directly • Formulate as Graph Cut • Allow user to guide boundary regions w/ brush

  15. Lazy Snapping – Video

  16. User Study • Compared Lazy Snapping to Photoshop

  17. User Study • Results • Easy of Use – Lazy Snapping 20% less mistakes • Speed – Lazy Snapping 60% less time • Accuracy – Lazy Snapping 60% less pixels wrong • Feedback • “Almost Magic” • “Much Easier” • Suggestions • Get rid of 2 step process • Make it easy to switch between graph cut strokes and boundary editing

  18. More Results

  19. GrabCutInteractive Foreground Extraction using Iterated Graph CutsCarsten RotherVladimir KolmogorovAndrew BlakeMicrosoft Research Cambridge-UK

  20. Big Changes • New model of foreground & background pixels: • GMM vs. K-mean colors vs. Intensity Histogram • More automation: • No need to specify foreground • Iterative Graph Cuts

  21. GrabCut – Interactive Foreground Extraction3 What GrabCut does Magic Wand(198?) Intelligent ScissorsMortensen and Barrett (1995) GrabCut User Input Result Regions Regions & Boundary Boundary

  22. Grab Cut Basics • Assign Selection box or Lasso as background • Assign other pixels as unknown • Learn Model of Fore/Background • Assign Link Energy, GraphCut (Boykov & Jolly) • Use GraphCut results to assign Fore/Background • Goto Step 3 • Allow user to add cleanup strokes

  23. Foreground (source) Min Cut Background(sink) Cut: separating source and sink; Energy: collection of edges Min Cut: Global minimal enegry in polynomial time GrabCut – Interactive Foreground Extraction5 Graph CutsBoykov and Jolly (2001) Image

  24. GrabCut – Interactive Foreground Extraction6 Iterated Graph Cut ? User Initialisation Graph cuts to infer the segmentation K-means for learning colour distributions

  25. GrabCut – Interactive Foreground Extraction7 Iterated Graph Cuts Guaranteed toconverge 1 3 4 2 Result Energy after each Iteration

  26. GrabCut – Interactive Foreground Extraction8 Colour Model Gaussian Mixture Model (typically 5-8 components) R R Iterated graph cut Foreground &Background Foreground G Background G Background

  27. GrabCut – Interactive Foreground Extraction10 Moderately straightforward examples … GrabCut completes automatically

  28. GrabCut – Interactive Foreground Extraction11 Difficult Examples Camouflage & Low Contrast Fine structure No telepathy Initial Rectangle InitialResult

  29. Error Rate: 0.72% GrabCut – Interactive Foreground Extraction13 Comparison Boykov and Jolly (2001) GrabCut User Input Result Error Rate: 1.87% Error Rate: 1.81% Error Rate: 1.32% Error Rate: 1.25% Error Rate: 0.72%

  30. GrabCut – Interactive Foreground Extraction16 Border Matting Hard Segmentation Automatic Trimap Soft Segmentation

  31. GrabCut – Interactive Foreground Extraction17 Natural Image Matting Mean Colour Foreground Mean ColourBackground Solve Ruzon and Tomasi (2000):Alpha estimation in natural images

  32. GrabCut – Interactive Foreground Extraction19 Border Matting Foreground Noisy alpha-profile 1 Mix Back-ground 0 Foreground Background Mix Fit a smooth alpha-profile with parameters

  33. GrabCut – Interactive Foreground Extraction21 Results

  34. GrabCut – Interactive Foreground Extraction22 Summary Intelligent Scissors Mortensen and Barrett (1995) LazySnappingLi et al. (2004) Graph Cuts Boykov and Jolly (2001) Magic Wand (198?) GrabCutRother et al. (2004)

  35. Comparison

  36. Bonus Paper • Texture Synthesis Revisited

  37. Basic Idea • Copy overlapping patches one at a times • Find the seamwhich minimize change in neighboring pixels • Formulate as Graph Cut

  38. Interesting Advantage • Extends well into 3D

More Related