A New Approach to the Maximum-Flow Problem

1. A New Approach to the Maximum-Flow Problem Andrew V. Goldberg, Robert E. Tarjan Presented by Andrew Guillory

2. Outline • Background • Definitions • Push-Relabel Algorithm • Correctness / Termination Proofs • Implementation

3. Maximum Flow Problem • Classic problem in operations research • Many problems reduce to max flow • Maximum cardinality bipartite matching • Maximum number of edge disjoint paths • Minimum cut (Max-Flow Min-Cut Theorem) • Machine learning applications • Structured Prediction, Dual Extragradient and Bregman Projections (Taskar, Lacoste-Julien, Jordan JMLR 2006) • Local Search for Balanced Submodular Clusterings (Narasimhan, Bilmes, IJCAI 2007)

4. Relation to Optimization • Special case of submodular function minimization • Special case of linear programming • Integer edge capacities permit integer maximum flows (constructive proof)

5. History of Algorithms • Augmenting Paths based algorithms • Ford-Fulkerson (1962) O(mU) • Edmonds-Karp (1969) O(nm3) • … O(n3) O(nmlog(n)) O(nmlog(U)) • Push-Relabel based algorithms • Goldberg (1985) O(n3) • Goldberg and Tarjan (1986) O(nmlog(n2/m)) • Ahuja and Orlin O(nm + n2log(U))

6. Outline • Background • Definitions • Push-Relabel Algorithm • Correctness / Termination Proofs • Implementation

7. Definitions • Graph G = (V, E) • |V| = n • |E| = m • G is a flow network if it has • source s and sink t • capacity c(v,w) for each edge (v,w) in E • c(v,w) = 0 for (v,w) not in E

8. Definitions (continued) • A flow f on G is a real value function on vertex pairs • f(v,w) <= c(v,w) for all (v,w) • f(v,w) = -f(w,v) • ∑uf(u,v) = 0 for all v in V - {s,t} • Value of a flow |f| is ∑vf(v,t) • Maximum flow is a flow of maximum value

9. Definitions (continued again) • A preflow f on G is a real value function on vertex pairs • f(v,w) <= c(v,w) for all (v,w) • f(v,w) = -f(w,v) • ∑uf(u,v) >= 0 for all v in V - {s} • Flow excess e(v) = ∑uf(u,v) • Intuition: flow into a vertex can exceed flow out

10. Outline • Background • Definitions • Push-Relabel Algorithm • Correctness / Termination Proofs • Implementation

11. Intuition • Starting with a preflow, push excess flow closer towards sink • If excess flow cannot reach sink, push it backwards to source • Eventually, preflow becomes a flow and in fact the maximum flow

12. Residual Graph • Residual capacity rf(v, w) of a vertex pair is c(v, w) – f(v, w) • If v has positive excess and (v,w) has residual capacity, can push δ = min(e(v), rf(v, w)) flow from v to w • Edge (v,w) is saturated if rf(v, w) = 0 • Residual graph Gf = (V, Ef) where Ef is the set of residual edges (v,w) with rf(v, w) > 0

13. Labeling • A valid labeling is a function d from vertices to nonnegative integers • d(s) = n • d(t) = 0 • d(v) <= d(w) + 1 for every residual edge • If d(v) < n, d(v) is a lower bound on distance to sink • If d(v) >= n, d(v) - n is a lower bound on distance to source

14. Push Operation Push(v,w) Precondition: v is active (e(v) > 0) and rf(v, w) > 0 and d(v) = d(w) + 1 Action: Push δ = min(e(v), rf(v, w)) from v to w f(v,w) = f(v,w) + δ; f(w,v) = f(w,v) – δ; e(v) = e(v) - δ; e(w) = e(w) + δ;

15. Relabel Operation Relabel(v) Precondition: v is active (e(v) > 0) and rf(v, w) > 0 implies d(v) <= d(w) Action: d(v) = min{d(w) + 1 | (v,w) in Ef}

16. Generic Push-Relabel Algorithm Starting from an initial preflow <<loop>> While there is an active vertex Chose an active vertex v Apply Push(v,w) for some w or Relabel(v)

17. Example 0/1 0/3 0/2 S T Flow Network

18. Example 0/1 0 0 3/3 0/2 4 0 S T Initial preflow / labeling

19. Example 0/1 0 0 3/3 0/2 4 0 S T Select an active vertex

20. Example 0/1 1 0 3/3 0/2 4 0 S T Relabel active vertex

21. Example 0/1 1 0 3/3 0/2 4 0 S T Select an active vertex

22. Example 1/1 1 0 3/3 0/2 4 0 S T Push excess from active vertex

23. Example 1/1 1 0 3/3 0/2 4 0 S T Select an active vertex

24. Example 1/1 1 1 3/3 0/2 4 0 S T Relabel active vertex

25. Example 1/1 1 1 3/3 0/2 4 0 S T Select an active vertex

26. Example 1/1 1 1 3/3 1/2 4 0 S T Push excess from active vertex

27. Example 1/1 1 1 3/3 1/2 4 0 S T Select an active vertex

28. Example 1/1 5 1 3/3 1/2 4 0 S T Relabel active vertex

29. Example 1/1 5 1 3/3 1/2 4 0 S T Select an active vertex

30. Example 1/1 5 1 1/3 1/2 4 0 S T Push excess from vertex

31. Example 1/1 5 1 1/3 1/2 4 0 S T Maximum flow

32. Outline • Background • Definitions • Push-Relabel Algorithm • Correctness / Termination Proofs • Implementation

33. Correctness • Lemma 2.1 If f is a preflow, d is a valid labeling, and v is active, either push or relabel is applicable to v • Lemma 3.1 The algorithm maintains a valid labeling d • Theorem 3.2 A flow is maximum iff there is no path from s to t in Gf (Ford and Fulkerson [7])

34. Correctness (continued) • Lemma 3.3 If f is a preflow and d is a valid labeling for f, there is no path from s to t in Gf • Proof by contradiction • Path s, v0, v1, …, vl, t implies that d(s) <= d(v0) + 1 <= d(v1) + 2 <= … <= d(t) + l < n • Which contradicts d(s) = n

35. Correctness (continued) • Theorem 3.4 If the algorithm terminates with a valid labeling, the preflow is a maximum flow • If the algorithm terminates, all vertices have zero excess (preflow is a flow) • By Lemma 3.3 the sink is not reachable from the source • By Theorem 3.2 the flow is maximum

36. Termination • Lemma 3.5 If f is a preflow and v is an active vertex then the source is reachable from v in Gf • Lemma 3.6 A vertex’s label never decreases

37. Termination (continued) • Lemma 3.7 At any time the label of any vertex is at most 2n – 1 • Only active vertex labels are changed • Active vertices can reach s • Path v, v0, v1, …, vl, s implies that d(v) <= d(v0) + 1 <= d(v1) + 2 <= … <= d(s) + l <= n + n - 1

38. Termination (continued) • Lemma 3.8 There are at most 2n2 labeling operations • Only the labels corresponding to V-{s,t} may be relabeled • Each of these n – 2 labels can only increase • At most (2n – 1) (n – 2) relabelings

39. Termination (continued) • Lemma 3.9 The number of saturating pushes is at most 2nm • For any pair (v,w) d(w) must increase by 2 between saturating pushes from v to w • Similarly d(v) must increase by 2 between pushes from w to v • d(v) + d(w) >= 1 on the first saturating push • d(v) + d(w) <= 4n - 3 on the last • At most 2n - 1 saturating pushes per edge

40. Termination (continued) • Lemma 3.10 The number of nonsaturating pushes is at most 4n2m • Φ = ∑v d(v) where v is active • Each nonsaturating push causes Φ to decrease by at least 1 • The total increase in Φ from saturating pushes is (2n – 1) 2nm • The total increase in Φ from relabeling is (2n – 1)(n – 2) • Φ is 0 initially and Φ at termination

41. Termination • Theorem 3.11 The algorithm terminates in O(n2m) Total time = # nonsaturating pushes + #saturating pushes + #relabeling operations 4n2m + 2nm + 2n2 =O(n2m)

42. Outline • Background • Definitions • Push-Relabel Algorithm • Correctness / Termination Proofs • Implementation

43. Implementation • At each step select an active vertex and apply either Push or Relabel • Problem: Determining which operation to perform and in the case of Push finding a residual edge • Solution: For each vertex maintain a list of edges which touch that vertex and a current edge

44. Push/Relabel Operation Push/Relabel(v) Precondition: v is active Action: If Push(v,w) is applicable to current edge (v,w) then Push(v,w) Else if (v,w) is not the last edge advance current edge Else reset the current edge and Relabel(v)

45. Push/Relabel Operation • Lemma 4.1 The push/relabel operation does a relabeling only when relabeling is applicable • Theorem 4.2 The push/relabel implementation runs in O(nm) time plus O(1) time per nonsaturating push operation

46. O(n3) bound • We can select vertices in arbitrary order • Certain vertex selection strategies give O(n3) bounds • First-in, first-out method (proved in paper) • Maximum distance method (proved here) • Wave method

47. Maximum distance method • At each step, select the active vertex with maximum distance d(v) • Theorem The maximum distance method performs at most 4n3 nonsaturating pushes • Corollary The maximum distance method runs in time O(n3) using the push/relabel implementation

48. Proof • Consider D = maxx d(x) where x is active • D only increases because of relabeling • D increases at most 2n2 times • D starts at 0 and ends nonnegative • D changes at most 4n2 times • There is at most one nonsaturating push per node per value of D