Image Foresting Transform

1. Image Foresting Transform for Image Segmentation • Presented by: • Michael Fang • Weilong Yang

2. A Few Things to Recall • Image Segmentation • Finding homogeneous regions • Graph-based Methods • Treating images as graphs • Image Foresting Transform • Unification • Efficiency • Simplicity

3. Directed Graphs A directed graph is a pair (I, A), where I is a set of nodes and A is a set of ordered pairs of nodes.

4. Paths • A path is a sequence t1, t2, …, tk of distinct nodes in the graph, such that (ti, ti+1)  A for 1  i k – 1. • A path is trivial if k = 1; • Path    denotes the concatenation of two paths,  and , where  ends at t and  begins at t. • Path  =   s, t denotes theconcatenation of the longest prefix  of  and the last arc (s, t).

5. Path Costs • A path-cost function is a mapping that assigns to each path  a cost (), in some ordered set  of cost values. • A function  is said to be monotonic-incremental (MI) when (t) = h(t), (  s, t) = ()  (s, t),where h(t) is a handicap cost value and  satisfies: x’  x  x’  (s, t)  x (s, t) and x  (s, t)  x,for x, x’   and (s, t)  A.

6. Examples of MI Cost Functions • Additive cost function sum(t) = h(t), sum(  s, t) = sum() + w(s, t),where w(s, t) is a fixed non-negative arc weight. • Max-arc cost function max(t) = h(t), max(  s, t) = max{max(),w(s, t)}, where w(s, t) is a fixed arc weight.

7. Predecessor Map and Spanning Forest • A predecessor map is a function P that assigns to each node t I either some other node in I, or a distinctive marker nil  I – in which case t is the root of the map. • A spanning forest is a predecessor map which takes every node to nil in a finite number of iterations (i.e., it contains no cycles).

8. Paths of the Forest P • For any node t I, there is a path P*(t) which is obtained in backward by following the predecessor nodes along the path. P*(c) = a, b, c, where P(c) = b, P(b) = a, P(a) = nil P*(i) = i, where P(i) = nil

9. Optimum-path Forest An optimum-path forest is a spanning forest P, where (P*(t)) is minimum for all nodes t I. Consider cost function sum in the example below.

10. An Image as a Directed Graph • A grayscale image I is a pair (I, I), where I is a finite set of pixels (points in Z2) and I assigns to each pixel t I a value I(t) in some arbitrary value space. • An adjacency relation A is a binary relation between pixels of I, which is usually translation-invariant. • Once A has been fixed, image I can be interpreted as a directed graph, whose nodes are the image pixels in I and whose arcs are defined by A.

11. Seed Pixels In some applications, we would like to use a predefined path-cost function  but constrain the search to paths that start in a given set SI of seed pixels. This constraint can be modeled by defining

12. IFT Algorithm for Image Segmentation • Path Cost • Four-Connected Adjacency

13. IFT algorithm with FIFO policy(1) Initialization C(t) I t

14. IFT algorithm with FIFO policy(2) ∝ ∝ ∝ ∝ 5 5 5 5 1 2 3 4 ∝ ∝ ∝ ∝ 5 5 5 5 5 6 7 8 ∝ ∝ ∝ ∝ 5 5 5 5 9 10 11 12 ∝ ∝ 0 0 0 5 5 0 0 0 13 14 15 16 13 0 0 16

15. IFT algorithm with FIFO policy(3) Growing Process

16. IFT algorithm with FIFO policy(4) ∝ ∝ ∝ ∝ 5 5 5 5 1 2 3 4 ∝ ∝ ∝ ∝ 5 5 5 5 5 6 7 8 5 ∝ ∝ ∝ 5 5 5 5 9 10 11 12 ∝ 0 5 0 0 5 5 0 13 14 15 16 5 5 0 5 5 0 16 9 14

17. IFT algorithm with FIFO policy(4) ∝ ∝ ∝ ∝ 5 5 5 5 1 2 3 4 ∝ ∝ ∝ ∝ 5 5 5 5 5 6 7 8 5 ∝ ∝ 5 5 5 5 5 9 10 11 12 5 0 5 0 0 5 5 0 13 14 15 16 5 5 5 5 5 5 5 5 14 15 9 12

18. IFT algorithm with FIFO policy(4) ∝ ∝ ∝ ∝ 5 5 5 5 1 2 3 4 5 ∝ ∝ ∝ 5 5 5 5 5 6 7 8 5 5 ∝ 5 5 5 5 5 9 10 11 12 5 0 5 0 0 5 5 0 13 14 15 16 5 5 5 5 5 5 5 5 5 5 12 15 10 14 5

19. IFT algorithm with FIFO policy(4) ∝ ∝ ∝ ∝ 5 5 5 5 1 2 3 4 5 ∝ ∝ ∝ 5 5 5 5 5 6 7 8 5 5 ∝ 5 5 5 5 5 9 10 11 12 5 0 5 0 0 5 5 0 13 14 15 16 5 5 5 5 5 5 5 5 12 5 15 10

20. IFT algorithm with FIFO policy(4) ∝ ∝ ∝ ∝ 5 5 5 5 1 2 3 4 5 ∝ ∝ 5 5 5 5 5 5 6 7 8 5 5 5 5 5 5 5 5 9 10 11 12 5 0 5 0 0 5 5 0 13 14 15 16 5 5 5 5 5 5 5 5 5 5 5 10 11 15 8

21. IFT algorithm with FIFO policy(4) 5 5 5 5 5 5 5 5 1 2 3 4 5 5 5 5 5 5 5 5 5 6 7 8 5 5 5 5 5 5 5 5 9 10 11 12 5 0 5 0 0 5 5 0 13 14 15 16

22. Another Example

23. Framework of Image segmentation by IFT Input Image Gradient Image Seeds Labeling IFT

24. Experiment Results (1)

25. Experiment Results (2)

26. Experiment Results (3)

27. Experiment Results (4)

28. Summary • Basic concept of the Image Foresting Transform • IFT for image segmentation • Experiment results

29. References (1) • Martin, D., Fowlkes, C., Tal, D., and Malik, J., A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics, ICCV(2), No. 7-14, January 2001, pp. 416–425. • Vincent, L., and Soille, P.[Pierre], Watersheds in Digital Spaces: An Efficient Algorithm Based on Immersion Simulations, PAMI(13), No. 6, June 1991, pp. 583-598. • Makrogiannis, S., Economou, G., Fotopoulos, S., A region dissimilarity relation that combines feature-space and spatial information for color image segmentation, SMC-B(35), No. 1, February 2005, pp. 44-53. • Haris, K., Efstratiadis, S.N., Maglaveras, N., Katsaggelos, A.K., Hybrid Image Segmentation Using Watersheds And Fast Region Merging, IP(7), No. 12, December 1998, pp. 1684-1699. • Shi, J.[Jianbo], Malik, J.[Jitendra], Normalized Cuts and Image Segmentation, PAMI(22), No. 8, August 2000, pp. 888-905. • Zabih, R.[Ramin], Kolmogorov, V.[Valdimir], Spatially coherent clustering using graph cuts, CVPR04(II: 437-444). • Boykov, Y.Y.[Yuri Y.], Jolly, M.P.[Marie-Pierre], Interactive Graph Cuts for Optimal Boundary and Region Segmentation of Objects in N-D Images, ICCV01(I: 105-112). • Andrew, Y.N., Jordan, M. and Weiss, Y., On Spectral Clustering: Analysis and an Algorithm, NIPS(14), 2002, pp. 849–856. • Weiss, Y.[Yair], Segmentation using Eigenvectors: A Unifying View, ICCV99(975-982). • Felzenszwalb, P.F.[Pedro F.], Huttenlocher, D.P.[Daniel P.], Image Segmentation Using Local Variation, CVPR98(98-104).

30. References (2) • Felzenszwalb, P.F.[Pedro F.], Huttenlocher, D.P.[Daniel P.], Efficient Graph-Based Image Segmentation, IJCV(59), No. 2, September 2004, pp. 167-181. • Haxhimusa, Y.[Yll] and Kropatsch, W.G.[Walter G.], Segmentation Graph Hierarchies, SSPR&SPR(18-20) August 2004, pp. 343–351. • Haxhimusa, Y.[Yll], Ion, A.[Adrian], Kropatsch, W.G.[Walter G.], Illetschko, T.[Thomas], Evaluating Minimum Spanning Tree Based Segmentation Algorithms, CAIP05(579). • Falcão, A.X.[Alexandre X.], Stolfi, J.[Jorge], de Alencar Lotufo, R.[Roberto], The Image Foresting Transform: Theory, Algorithms, and Applications, PAMI(26), No. 1, January 2004, pp. 19-29. • Falcão, A.X.[Alexandre X.], Bergo, F.[Felipe] and Miranda, P.[Paulo], Image Segmentation by Tree Pruning, Proceedings of the Computer Graphics and Image Processing, XVII Brazilian Symposium, 2004, pp. 65–71. • Ding, J., Ma, R., Chen, S. and Wang, B., A Fast Directed Tree Based Neighborhood Clustering for Image Segmentation, NIP(4233), 2006, pp. 369–378. • Li, K.[Kang], Wu, X.D.[Xiao-Dong], Chen, D.Z.[Danny Z.], Sonka, M.[Milan], Optimal Surface Segmentation in Volumetric Images: A Graph-Theoretic Approach, PAMI(28), No. 1, January 2006, pp. 119-134. • Grady, L.[Leo], Random Walks for Image Segmentation, PAMI(28), No. 11, November 2006, pp. 1768-1783. • Pednekar, A.S.[Amol S.], Kakadiaris, I.A.[Ioannis A.], Image Segmentation Based on Fuzzy Connectedness Using Dynamic Weights, IP(15), No. 6, June 2006, pp. 1555-1562. • Luo, Q., Khoshgoftaar, T.M., Unsupervised Multiscale Color Image Segmentation Based on MDL Principle, IP(15), No. 9, August 2006, pp. 2755-2761.

31. References (3) • Arbeláez, P.A.[Pablo A.], Cohen, L.D.[Laurent D.], A Metric Approach to Vector-Valued Image Segmentation, IJCV(69), No. 1, August 2006, pp. 119-126. • Cremers, D.[Daniel], Osher, S.J.[Stanley J.], Soatto, S.[Stefano], Kernel Density Estimation and Intrinsic Alignment for Shape Priors in Level Set Segmentation, IJCV(69), No. 3, September 2006, pp. 335-351. • Bresson, X.[Xavier], Vandergheynst, P.[Pierre], Thiran, J.P.[Jean-Philippe], A Variational Model for Object Segmentation Using Boundary Information and Shape Prior Driven by the Mumford-Shah Functional, IJCV(68), No. 2, June, 2006, pp. 145-162. • Boykov, Y.Y.[Yuri Y.], Funka-Lea, G.[Gareth], Graph Cuts and Efficient N-D Image Segmentation, IJCV(70), No. 2, November 2006, pp. 109-131. • Bresson, X.[Xavier], Vandergheynst, P.[Pierre], Thiran, J.P.[Jean-Philippe], Multiscale Active Contours, IJCV(70), No. 3, December 2006, pp. 197-211 • Tu, Z.W.[Zhuo-Wen], Zhu, S.C.[Song-Chun], Parsing Images into Regions, Curves, and Curve Groups, IJCV(69), No. 2, August 2006, pp. 223-249. • Seghers, D., Loeckx, D.[Dirk], Maes, F.[Frederik], Vandermeulen, D., Suetens, P.[Paul], Minimal Shape and Intensity Cost Path Segmentation, MedImg(26), No. 8, August 2007, pp. 1115-1129. • Papandreou, G., Maragos, P., Multigrid Geometric Active Contour Models, IP(16), No. 1, January 2007, pp. 229-240. • Tao, W., Jin, H., Zhang, Y., Color Image Segmentation Based on Mean Shift and Normalized Cuts, SMC-B(37), No. 5, October 2007, pp. 1382-1389. • Wu, J., Chung, A.C.S., A Segmentation Model Using Compound Markov Random Fields Based on a Boundary Model, IP(16), No. 1, January 2007, pp. 241-252.

32. References (4) • Pavan, M.[Massimiliano], Pelillo, M.[Marcello], Dominant Sets and Pairwise Clustering, PAMI(29), No. 1, January 2007, pp. 167-172. • Tai, Y.W.[Yu-Wing], Jia, J.[Jiaya], Tang, C.K.[Chi-Keung], Soft Color Segmentation and Its Applications, PAMI(29), No. 9, September 2007, pp. 1520-1537. • Pyun, K., Lim, J., Won, C.S., Gray, R.M., Image Segmentation Using Hidden Markov Gauss Mixture Models, IP(16), No. 7, July 2007, pp. 1902-1911. • Arias, P., Pini, A., Sanguinetti, G., Sprechmann, P., Cancela, P., Fernandez, A., Gomez, A., Randall, G., Ultrasound Image Segmentation With Shape Priors: Application to Automatic Cattle Rib-Eye Area Estimation, IP(16), No. 6, June 2007, pp. 1637-1645. • Nikou, C.[Christophoros], Galatsanos, N.P.[Nikolaos P.], Likas, A.C.[Aristidis C.], A Class-Adaptive Spatially Variant Mixture Model for Image Segmentation, IP(16), No. 4, April 2007, pp. 1121-1130. • Benboudjema, D.[Dalila], Pieczynski, W.[Wojciech], Unsupervised Statistical Segmentation of Nonstationary Images Using Triplet Markov Fields, PAMI(29), No. 8, August 2007, pp. 1367-1378. • Michailovich, O.V.[Oleg V.], Rathi, Y.[Yogesh], Tannenbaum, A.[Allen], Image Segmentation Using Active Contours Driven by the Bhattacharyya Gradient Flow, IP(16), No. 11, November 2007, pp. 2787-2801. • Zoller, T.[Thomas], Buhmann, J.M.[Joachim M.], Robust Image Segmentation Using Resampling and Shape Constraints, PAMI(29), No. 7, July 2007, pp. 1147-1164. • Qiu, H.J.[Huai-Jun], Hancock, E.R.[Edwin R.], Clustering and Embedding Using Commute Times, PAMI(29), No. 11, November 2007, pp. 1873-1890. • Rivera, M., Ocegueda, O., Marroquin, J.L., Entropy-Controlled Quadratic Markov Measure Field Models for Efficient Image Segmentation, IP(16), No. 12, December 2007, pp. 3047-3057.

33. References (5) • Chen, S.F.[Shi-Feng], Cao, L.L.[Liang-Liang], Liu, J.Z.[Jian-Zhuang], Tang, X.[Xiaoou], Iterative MAP and ML Estimations for Image Segmentation, CVPR07(1-6). • Allili, M.S.[Mohand Said], Ziou, D.[Djemel], Object of Interest segmentation and Tracking by Using Feature Selection and Active Contours, CVPR07(1-8). • Liu, T.[Tie], Sun, J.[Jian], Zheng, N.N.[Nan-Ning], Tang, X.[Xiaoou], Shum, H.Y.[Heung-Yeung], Learning to Detect A Salient Object, CVPR07(1-8). • Cremers, D.[Daniel], Rousson, M.[Mikael], Deriche, R.[Rachid], A Review of Statistical Approaches to Level Set Segmentation: Integrating Color, Texture, Motion and Shape, IJCV(72), No. 2, April 2007, pp. 195-215. • Tai, X.C.[Xue-Cheng], Christiansen, O.[Oddvar], Lin, P.[Ping], SkjÆlaaen, I.[Inge], Image Segmentation Using Some Piecewise Constant Level Set Methods with MBO Type of Projection, IJCV(73), No. 1, June 2007, pp. 61-76. • Riklin-Raviv, T.[Tammy], Kiryati, N.[Nahum], Sochen, N.A.[Nir A.], Prior-based Segmentation and Shape Registration in the Presence of Perspective Distortion, IJCV(72), No. 3, May 2007, pp. 309-328. • Sofou, A., Maragos, P.[Petros], Generalized Flooding and Multicue PDE-Based Image Segmentation, IP(17), No. 3, March 2008, pp. 364-376. • Ding, J., Ma, R., Chen, S., A Scale-Based Connected Coherence Tree Algorithm for Image Segmentation, IP(17), No. 2, February 2008, pp. 204-216. • Du, X., Bui, T.D., A New Model for Image Segmentation, SPLetters(15), 2008, pp. 182-185. • Orbanz, P.[Peter], Buhmann, J.M.[Joachim M.], Nonparametric Bayesian Image Segmentation, IJCV(77), No. 1-3, May 2008, pp. 25-45. Thank You!Questions?