A Distributed Mincut/Maxflow Algorithm Combining Path Augmentation and Push-Relabel

1. A Distributed Mincut/Maxflow Algorithm Combining Path Augmentation and Push-Relabel Alexander Shekhovtsov and Václav Hlaváč shekhole@fel.cvut.cz, hlavac@fel.cvut.cz This work was supported bu EU projects FP7-ICT-247870 NIFTi and FP7-ICT-247525 HUMAVIPS and the Czech project 1M0567 CAK Czech Technical University in Prague EMMCVPR 25-27 July, 2011 Center for Machine Perception TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAA

2. Overview • Goals • Main Results Preliminary results presented on workshop 2010 in Kiev (not entirely correct) Alexander Shekhovtsov & Vaclav Hlavac, 2011

3. Mincut in Computer Vision • Test Problems (University of Western Ontario) Alexander Shekhovtsov & Vaclav Hlavac, 2011

4. Mincut in Computer Vision • Discrete Energy Minimization via Mincut Alexander Shekhovtsov & Vaclav Hlavac, 2011

5. Mincut • Capacitated network Alexander Shekhovtsov & Vaclav Hlavac, 2011

6. Mincut • Minimum s-t Cut Alexander Shekhovtsov & Vaclav Hlavac, 2011

7. Augmenting Path Approach • Path Augmentation Alexander Shekhovtsov & Vaclav Hlavac, 2011

8. Augmenting Path Approach • Residual Network Alexander Shekhovtsov & Vaclav Hlavac, 2011

9. Augmenting Path Approach • Costs of all cuts are changed by a constant Alexander Shekhovtsov & Vaclav Hlavac, 2011

10. Augmenting Path Approach • Augment next path Alexander Shekhovtsov & Vaclav Hlavac, 2011

11. Augmenting Path Approach • Minimum Cut in Residual Network Alexander Shekhovtsov & Vaclav Hlavac, 2011

12. Push-Relabel Approach • Extended transformation excess excess Alexander Shekhovtsov & Vaclav Hlavac, 2011

13. Push-Relabel Approach Alexander Shekhovtsov & Vaclav Hlavac, 2011

14. Push-Relabel Approach Alexander Shekhovtsov & Vaclav Hlavac, 2011

15. Push-Relabel Approach Alexander Shekhovtsov & Vaclav Hlavac, 2011

16. Push-Relabel Approach Alexander Shekhovtsov & Vaclav Hlavac, 2011

17. Push-Relabel Approach Alexander Shekhovtsov & Vaclav Hlavac, 2011

18. Distributed Algorithms Shared Memory Parallel Sequential Memory Memory CPU CPU CPU • Distributed Model – Divide Computation AND Memory Distributed Sequential Distributed Parallel Mem Mem Mem Quick Quick CPU CPU CPU Slow Slow Disk Alexander Shekhovtsov & Vaclav Hlavac, 2011

19. Existing Distributed Algorithms • Push-Relabel [Goldberg ‘94] • Region Discharge [Delong and Boykov, CVPR‘08] Alexander Shekhovtsov & Vaclav Hlavac, 2011

20. Existing Distributed Algorithms • Adaptive Bottom-up Merging [Liu and J. Sun, CVPR‘10] Alexander Shekhovtsov & Vaclav Hlavac, 2011

21. Existing Distributed Algorithms • Dual Decomposition for Mincut [Strandmark and Kahl, CVPR‘10] Alexander Shekhovtsov & Vaclav Hlavac, 2011

22. Existing Distributed Algorithms • Dual Decomposition for Mincut [Strandmark and Kahl, CVPR‘10] Alexander Shekhovtsov & Vaclav Hlavac, 2011

23. New Algorithm • New Distance Function • The Algorithm • Parallel Version • Complexity Bound • Experimental Confirmation Alexander Shekhovtsov & Vaclav Hlavac, 2011

24. New Distance Function length of the path = number of boundary edges distance = length of a shortest path to the sink Valid Labeling – distance underestimate Alexander Shekhovtsov & Vaclav Hlavac, 2011

25. The Algorithm: Augmenting Path Region Discharge sweep 1. Augment paths to the sink 2. Augment paths the boundary with label 0 3. Augment paths the boundary with label 1 ... estimate of the shortest way to the sink Relabel the interior vertices Alexander Shekhovtsov & Vaclav Hlavac, 2011

26. Parallel Version parallel sweep all regions discharged concurrently Conflicts are resolved by canceling one of the flows (similar to asynchronous parallel push-relabel [Goldberg and Tarjan 88]) Alexander Shekhovtsov & Vaclav Hlavac, 2011

27. Complexity Bound Alexander Shekhovtsov & Vaclav Hlavac, 2011

28. Experimental Confirmation S-PRD: Sequential Push-Relabel Region DischargeS-ARD: Sequential Augmenting path Region Discharge Test problems: 2D grid with regular connectivity and random costs S-PRD details: highest label first push-relabel (HI-PR), region-gap heuristic, region-relabel heuristic, global gap heuristic S-ARD details: global gap heuristic Alexander Shekhovtsov & Vaclav Hlavac, 2011

29. Efficient Implementation • Standard Heuristics: global relabel and global gap • Boundary Relabel • Partial Discharges • Boundary Search Trees Alexander Shekhovtsov & Vaclav Hlavac, 2011

30. Efficient Implementation • Global Relabel Heuristic = compute exact distance, O(m) time • Global Gap Heuristic (sufficient condition of sink unreachability) … cannot reach sink cannot be in the sink set of a minimum cut “decided nodes” Alexander Shekhovtsov & Vaclav Hlavac, 2011

31. not possible Efficient Implementation • Boundary Relabel Improve labeling, knowing only the information on the boundary? do not know how the vertices linked inside Group nodes by their label within each region Compute exact distanceon the auxiliary graph A valid labeling ofboundary nodes Add possible links (red) Result: a (better) valid labeling. In the limit of small regions = global relabel. Alexander Shekhovtsov & Vaclav Hlavac, 2011

32. Efficient Implementation • Partial Discharges execute only this step in sweep 0 1. Augment paths to the sink 2. Augment paths the boundary with label 0 3. Augment paths the boundary with label 1 ... Relabel the interior vertices Alexander Shekhovtsov & Vaclav Hlavac, 2011

33. Efficient Implementation • Partial Discharges execute up to here in sweep 1 1. Augment paths to the sink 2. Augment paths the boundary with label 0 3. Augment paths the boundary with label 1 ... Relabel the interior vertices Alexander Shekhovtsov & Vaclav Hlavac, 2011

34. Efficient Implementation • Partial Discharges execute up to here in sweep 2 1. Augment paths to the sink 2. Augment paths the boundary with label 0 3. Augment paths the boundary with label 1 ... Relabel the interior vertices Prevents sending the flow in a wrong direction (redundant work) Boundary Relabel + Partial Discharges effect: Alexander Shekhovtsov & Vaclav Hlavac, 2011

35. Efficient Implementation • Boundary Search Trees Find augmenting paths and encode labeling. Search trees of thesink and boundaryvetices Residual region network Labels of the innervertices are determinedby their tree root Tree with higher root cannot overtake lower Saves computation between sweeps, Integrates region-relabel into augmentation. Alexander Shekhovtsov & Vaclav Hlavac, 2011

36. Results • Dependence on Partition • Sequential Competition • Parallel Competition • Local Problem Reduction Alexander Shekhovtsov & Vaclav Hlavac, 2011

37. BL06-gargoyle-sml cell complex 32x45x32x24 partitioned by vertex number Results: Dependence on Partition Stable over partitions LB07-bunny-med 3D 6-connected 202x199x157 sparse data sliced along dimensions liver.n6.c10 3D 6-connected 170x170x144 sliced along dimensions Alexander Shekhovtsov & Vaclav Hlavac, 2011

38. Results: Sequential Competition BK – Augmenting Path [Boykov-Kolmogorov] HIPR – Highest level Push-Relabel [Goldberg-Tarjan, Cherkassky] S-ARD – Sequential Augmenting Path Region Discharge S-PRD – Sequential Push-Relable Region Discharge ([Delong-Boykov], our impl.) Alexander Shekhovtsov & Vaclav Hlavac, 2011

39. Results: Sequential Competition BK – Augmenting Path [Boykov-Kolmogorov] HIPR – Highest level Push-Relabel [Goldberg-Tarjan, Cherkassky] S-ARD – Sequential Augmenting Path Region Discharge S-PRD – Sequential Push-Relable Region Discharge ([Delong-Boykov], our impl.) ... Alexander Shekhovtsov & Vaclav Hlavac, 2011

40. best parameters not applicable not applicable reduced graph Results: Parallel Competition (4 CPUs) DD – Dual Decomposition [ Strandmark and Kahl ’10] 2/4 Regions RPR – Region Push-Relable [Delong and Boykov ‘08] impl. by Sameh Khamis shared memory P-ARD speed-up over BK: 0.8 – 4, robust method, few sweeps (message exchanges) Adaptive Bottom-up Merging [Liu and J. Sun ‘10] achieves near linear speed-up over BK Alexander Shekhovtsov & Vaclav Hlavac, 2011

41. Local Problem Reduction • How many vertices can be decided optimally by consideringregions separately? 16 regions 64 regions 2D stereo problems are largely decided locally. Other problems are significantly harder. Alexander Shekhovtsov & Vaclav Hlavac, 2011

42. Conclusion • New distributed algorithm • Terminates in at most B2+1 sweeps (few in practice) • Sequential Algorithm • competitive with sequential solvers • uses few sweeps (= loads/unloads of regions) • suitable to run in the limited memory model • Parallel Algorithm • competitive with shared memory algorithms • uses few sweeps (= rounds of message exchange) • suitable for execution on a computer cluster • Implementation can be specialized for regular grids (less memory/faster) • (?) no good worst case complexity bound in terms of elementary operations Thanks for you comments Alexander Shekhovtsov & Vaclav Hlavac, 2011