“ grabcut ” - Interactive Foreground Extraction using Iterated Graph Cuts

1. “grabcut”- Interactive Foreground Extraction using Iterated Graph Cuts An weizhi 2161230233

2. overview • Background • Previous Approach • Start From Graph Cut • Grab Cut • Conclusion

3. Background • Foreground-background segmentation

4. Previews work white brush : foreground yellow brush :boundary red brush :background

5. Previews work • Compared to the previews work • Graph cut simplied the iteraction • Robust for the segmentation in complex situation.

6. Start from Graph Cut foreground background

7. Start from Graph Cut • Define image as an array (gray image) • Define the segmentation as opacity for each pixel if (is background if ( = 1) is foreground

8. Start from Graph Cut • Define to describe image grey-level distribution • Create histogram distributions for foreground and background respectively

9. Graph Cut • The problem is how to infer with the given array z and model ? Energy function Data term Smooth term

10. Data term estimated using background histogram distribution model estimated using foreground histogram distribution model

11. Personal understanding • Imaging there are two histograms • When uncertain pixels put into the two histograms, it will output a value which can be seen as a probability. • The higher probability means the pixels are more likely belong to the correspondent When uncertain pixels has been classified correctly, will convergence

12. Smoothness term • represent works on boundary • When minimize , it aims to get a “large pixel distance” • boundary • When in ,it switch apporprately between high and low contrast

13. Objective funtion • The segmentation can be estimated as a global minimun: How to solve this optimization problem? Min-Cut / Max-Flow Algorithm

14. Image to Graph • Treat an image as a graph • Graph: • Nodes • A background node • A foreground node • n-nodes corresponds to n-pixels • Edges • Every node connect with both S and T • Every node connect with its neighbors • Treat Cut as segmentation

16. New Challenge How about take image into RGB colour space ? How to succeed more simple users interaction?

17. Motivation use the value histogram? Too sparse GMM (Gaussian Mixture Model) estimation

18. Colour data modelling background Background is exactly fixed Assumption: the image array z satisfied a probability distribution

19. Colour data modeling • Define There are two GMMs ,one for background and one for foreground • The GMMs are full-covariance Gaussian mixture with K components(K=5) • Define vector

20. Colour data modeling • Energy function Smoothness term Data term • Data term

21. Data term Gaussian Probability Formula

22. Smoothness term The V is unchanged from the previous term except the pixel distance calculation Our aimis to get opacity covariance GMM components means weight

23. Method:EM algorithm • Initialisation • Background: • Initial foreground updated

24. Initialisation • Initialize k Use k-means clustering • For each pixel belongs to a GMM component • Initialize for GMMs components

25. (1) • is an image pixels array • Each pixel already assigned to foreground or background, is known. • has been initialized • Our aim:

26. learning GMM paramaters • learn GMM parameters from data z • (2) • GMM parameters： ) ) set of foreground pixels assigned to component k

27. Estimate segmentation • Estimate segmentation use min cut • Repeat above steps (1)(2)(3) until convergence

28. Optimizaiton result

29. Border Matting Our aim:forproduce continuous in the boundary • Define a Contour C • （previous segmentation） • Recompute nearby • Caculate centre distance width

30. Border Matting Smoothness term Data term • Using DP algorithm to minimize E

31. Border Matting • Smoothness term • Data term mean covariance Gaussian probability

32. Foreground estimation • For estimate foregroundpixelnot from background(Bayes matte), grabcut has no blackgroundcolours bleeding Comparing methods for border matting

33. Result

34. Result More difficult situation

35. Result

36. Failures situation • Regions of low contrast（reduce V penalty） • Camouflage, with overlap in distribution • Background material inside the user rectangle happens important to the background total distribution

37. Conclusion • Grab cut could cope with moderately difficult images with simple user interaction • It combines hard segmentationby iteration • It use border matting to make the hard boundary more smooth

38. Q&A • Q: what does mean? • A: It means and each could output a probability • Q:Why does the grabcut not use the original histograms instead of using GMM • A:For the image in colourspace,the image will have 3 channels, and it is too sparse to use the histograms.So the grabcut proposed a more intuitive model GMM to

39. Q&A • Replace the histograms • Q:What’s the requirements when drawing a rectangle on the images? • A:In fact,we need to ensure the background is outside the rectangle.For we have emphasized in the failure situations that the background distributions need a abundant information. • Q:In grabcut,theis uncertaion,and how to

40. Q&A solve • A:We set a initial at first, and then we use EM methods to do a iteration.In the interation, we could get a optimization of

41. Thank you