A Seeded Image Segmentation Framework Unifying Graph Cuts and Random Walker Which Yields A New Algorithm

1. A Seeded Image Segmentation Framework Unifying Graph Cuts and Random Walker Which Yields A New Algorithm Ali Kemal Sinop * Computer Science Department Carnegie Mellon University Leo Grady Department of Imaging and Visualization Siemens Corporate Research * work done while author was at Siemens Corporate Research

2. Outline • Review of seeded segmentation – Graph Cuts and Random Walker • Our generalized framework • The q = ∞ case • Comparison • Conclusion

3. Seeded Segmentation Review Interactive segmentation • Four parts: • Input foreground/background pixels (seeds) from the user • Use image content to establish affinity (metric) relationships between pixels • Perform energy minimization over the space of functions defined on pixels • Assign a foreground/background label to each pixel corresponding to the value of the function at that pixel

4. abstraction Seeded Segmentation Review – Graph Cuts Graph cuts Abstract image to a weighted graph Compute min-cut/max-flow 44 image 44 weighted graph 3 1 2 2 1 4 S T 5 6 1

5. abstraction 3 1V 1 2 2 1 4 5 6 1 Seeded Segmentation Review – Random Walker Random Walker Abstract image to a weighted graph 44 image 44 weighted graph Compute probability that a random walker arrives at seed Random walk view Steady-state circuit view

6. Outline • Review of seeded segmentation – Graph Cuts and Random Walker • Our generalized framework • The q = ∞ case • Comparison • Conclusion

7. Choice of q determines solution properties: q = 1 Graph Cuts q = 2 Random Walker q =∞ ? Generalized seeded segmentation framework ‘Algorithm A’

8. Generalized seeded segmentation: q = 1 Graph Cuts Note: If x is binary, energy represents cut size Unary terms implicit

9. Z 1 2 [ ] ( ( ) ) D r u g u x y = ; 2 ­ r r 0 ¢ g u = Generalized seeded segmentation: q = 2 Random Walker Solution to random walk problem equivalent to minimization of the Dirichlet integral with appropriate boundary conditions. The solution is given by a harmonic function, i.e., a function satisfying

10. 1 1 T T T 1 0 ¡ ¢ [ ] ( ) D A C A L x x = = F B x x x x x = = ; 2 2 r L r 0 0 ¢ g x u = = Generalized seeded segmentation: q = 2 Random Walker Energy functional: Subject to boundary conditions at seed locations Euler-Lagrange:

11. Outline • Review of seeded segmentation – Graph Cuts and Random Walker • Our generalized framework • The q = ∞ case • Comparison • Conclusion

12. where is the minimum distance from pixel i to a background seed The q = ∞ case q = ∞ How to optimize?

13. The q = ∞ case Problem: Uniqueness Multiple solutions minimize functional Solution: Find the solution that additionally minimizes the (q = 2) energy

14. Outline • Review of seeded segmentation – Graph Cuts and Random Walker • Our generalized framework • The q = ∞ case • Comparison • Conclusion

15. Comparison - Theoretical Metrication q = ∞ q = 1 (Graph Cuts) q = 2 (Random Walker)

16. Comparison - Quantitative Stability relationship

17. Comparison - Qualitative q = 1 (Graph Cuts) q = 2 (Random Walker) q = ∞

18. Outline • Review of seeded segmentation – Graph Cuts and Random Walker • Our generalized framework • The q = ∞ case • Comparison • Conclusion

19. Conclusion 1) Graph Cuts and Random Walker algorithms may be seen as minimizing the same functional with respect to an L1 or L2 norm, respectively 2) The L∞ case was previously unexplored, may be optimized efficiently and produces “tight” segmentations with minimum sensitivity to seed number More information L∞ paper: http://cns.bu.edu/~lgrady/sinop2007linf.pdf Random walkers paper: http://cns.bu.edu/~lgrady/grady2006random.pdf Random walkers MATLAB code: http://cns.bu.edu/~lgrady/random_walker_matlab_code.zip MATLAB toolbox for graph theoretic image processing at: http://eslab.bu.edu/software/graphanalysis/