Supplementary: Feasible Labels and Linear Programming

1. Supplementary:Feasible Labels and Linear Programming • Consider the LP formulation of the shortest s-t path problem: (P) Minimize (i, j)A cijxij subject to {j: (i, j)A} xij - {j: (j, i)A} xji = +1, for i = s = -1, for i = t = 0, for i N\{s, t} xij  0 for all (i, j)A It is guaranteed that the optimal solution to (P), if exists, is integer-valued (proof later when we study MCF). Hence we can solve the LP instead of IP. • Multiply (-1) on both sides of the constraints, then get {j: (j, i)A} xji - {j: (i, j)A} xij = -1, for i = s = +1, for i = t = 0, for i N\{s, t} i.e consider (inflow to i) – (outflow from i)

2. (P) Minimize (i, j)A cijxij (S-1) subject to {j: (j, i)A} xji - {j: (i, j)A} xij = -1, for i = s = +1, for i = t = 0, for i N\{s, t} xij  0 for all (i, j)A • The dual of (P) is (D) maximize yt – ys (S-2) subject to yj - yi cij for all (i, j)  A • Note that, for a dual feasible solution y, we may subtract any constant  from each dual variable and we still have a dual feasible solution with the same objective value.  we may set ys = 0.

3. We assumed there exists a s-t path in G, hence (P) is feasible. If (P) has a finite optimal value (with directed s-t walk) , then by strong duality, (D) is also feasible and its optimal value is the same as (P) (shortest s-t distance = yt ) Conversely, if (D) has a feasible solution, then (P) and (D) have the same optimal value (since (P) is feasible too). (Compare with Thm 5.1) • If the network contains a negative cycle, (P) is unbounded, hence (D) is infeasible, i.e. no y (d()) satisfies (5.2)

4. The constraint matrix in (P) has special structure; each column has exactly one 1 and one –1 (except nonnegativity constraints). Adding the rows of the matrix A, we get 0 vector, which implies the rows are linearly dependent (rank is  n-1). • Prop : Let G be a connected directed graph and D = {ae : e  A} be its node-arc incidence matrix ( ae is the column vector for arc e in the constraint matrix) . A set {ae : e  T} is a column basis of D iff T is the arc-set of a spanning tree of G (not considering the directions of the arcs). ( G has n nodes, m arcs. D is n  m matrix. Spanning tree has n-1 arcs.)

5. So the rank of D is n-1. To solve the problem by the simplex method, we need nonsingular basis B. Two approaches: • Drop one of the constraints and solve the full row rank problem • Add an artificial variable to any one constraint (objective coefficient is 0). It makes D full row rank (the artificial variable always remains in the basis and has value 0 in any b.f.s). • b.f.s. and dual vector satisfies xe = 0 for all eT yj – yi = cij for all (i, j)T ( From y’B = cB’ for current basis B) This y vector gives the path length to each node j in the current tree T (If T is directed out-tree). Note that if we add artificial variable for node s (root), then we have ys = 0 in any b.f.s. ( from y’B = cB’)

6. A b.f.s. for (P) does not necessarily correspond to an directed out-tree. It may be just a spanning tree. However, the simplex iteration can be modified so that we always have directed out-tree, once we started the algorithm with a directed out-tree. • Each step of the simplex method may correspond to multiple Ford’s step; (1) Change of directed out-tree (2) update the node labels so that it gives the actual path length in the directed out-tree.

7. Suppose we solve s-t shortest path problem by LP (with simplex method) and have a feasible directed out-tree (bfs); • entering nonbasic arc selection: choose arc (i, j) with cij – (yj – yi) < 0, i.e. cij + yi < yj. ( from cj – y’Aj < 0 since we solve min problem ) • leaving basic variable: select the entering arc to node j in the current directed out-tree as leaving basic arc. (updating tree in Ford algorithm) • Update flow (x-value) so that we have value 1 for each arc on the s-t path. • Note that we may solve the shortest s-t path problem by LP. But, in an optimal basis (directed out-tree), we have shortest s-j path, for all jN. ( If you use the above modified simplex method) ( We have feasible potential (label) y and the s-j paths in the directed out-tree for all jN gives actual paths with length yjfor all jV.)

8. The identification of the basis of the LP as a spanning tree (not necessarily directed out-tree) is used in the network simplex algorithms for the minimum cost network flow problem. (Chapter 11)

9. Other Interpretation of Ford-Bellman Algorithm • Reference: Combinatorial Optimization: Networks and Matroids, E. Lawler • Bellman’s equations: u1 = 0 uj = min kj { uk +akj }, j=2, 3, …, n (necessary and sufficient conditions for shortest paths when G does not contain a nonpositive cycle.) • Solve Bellman’s eq. iteratively. Still effective if no negative cycle in G. • Initial values u1(1) = 0 uj(1) = a1j, j  1 Iteration uj(m+1) = min { uj(m) , min kj { uk(m) + akj }} • Interpretation uj(m) = the length of a shortest path from the origin to j, subject to the condition that the path contains no more than m arcs. (compare it with the presentation given in the text)

10. 5.6 All-Pairs Shortest Path Probelm • Assume G is strongly connected. (add (s, i) and (i, s) for iN\{s} with large costs if not exist) • Two approaches: • Repeated shortest path algorithm (suitable for sparse graph) • All-pairs label-correcting algorithm (suitable for dense graph) • Repeated Shortest Path Algorithm: Let S(n, m, C) denote the time needed to solve a shortest path problem with nonnegative arc lengths. Select a node s as a source node and use FIFO label-correcting algorithm. Then, have cijd = cij + d(i) – d(j)  0 for all arcs (i, j). Using cijd  0, solve single-source shortest path problem (n-1) times. From Property 5.2.(b) ((i, j)P cijd = (i, j)P cij + d(k) – d(l)), actual shortest path length from k to l is ((i, j)P cijd + (d(l) – d(k) ) Running time: O(nm + nS(n, m, C)) = O(nS(n, m, C)).

11. All-Pairs Shortest Path Optimality Conditions: • Thm 5.5. Let d[i, j] represent the length of some directed path from node i to node j. Then d[i, j] optimal  d[i, j]  d[i, k] + d[k, j] for all i, j, and k. (5.3) Pf) ) If d[i, k] + d[k, j] < d[i, j], use path from i → k → j walk. Contradiction to the optimality of d[i, j]. ) d[i, j] length of some path  upper bound on the shortest length Let i = i1 – i2 – i3 - … - ik = j be any i-j path P  d[i, j] = d[i1, ik]  d[i1, i2] + d[i2, ik]  ci(1)i(2) + d[i2, ik], d[i2, ik]  ci(2)i(3) + d[i3, ik], … d[ik-1, ik]  ci(k-1)i(k).  d[i, j]  ci(1)i(2) + ci(2)i(3) + … + ci(k-1)i(k) = (i, j)P cij. Hence d[i, j] is a lower bound on any i-j path. d[i, j] also upper bound, so optimal.

12. All-Pairs Generic Label Correcting Algorithm: algorithm all-pairs label-correcting; begin set d[i, j] :=  for all [i, j]NN; set d[i, i] := 0 for all iN; for each (i, j)A do d[i, j] := cij; while the network contains three nodes i, j, and k satisfying d[i, j] > d[i, k] + d[k, j] do d[i, j] := d[i, k] + d[k, j]; end;

13. Correctness: Whenever d[i, j] < ,  walk of length d[i, j] from i to j. The walk may decompose into a path P and some directed cycles. At termination of the algorithm, all of the directed cycles must have length 0, hence the path length is d[i, j]. Also labels d[i, j] satisfy the optimality condition (5.3). • Finiteness: Distance label is bounded from above by nC, from below by –nC. Each iteration decreases some d[i, j]. Hence, total O(n3C) iterations. • O(n3) implementation: Floyd-Warshall Algorithm

14. Floyd-Warshall Algorithm: dk[i, j]: i-j shortest path length using only the nodes 1, 2, … , k-1 as internal nodes. Then, dn+1[i, j] represents shortest path length from i to j. Starting from d1[i, j], update the labels recursively. • Property 5.6. dk+1[i, j] = min {dk[i, j], dk[i,k] + dk[k, i]}. Two cases: • Do not use node k. Then dk+1[i, j] = dk[i, j] • Use node k. Then dk+1[i, j] = dk[i,k] + dk[k, i]

15. Algorithm Floyd-Warshall; begin for all node pairs [i, j]NN d[i, j] :=  and pred[i, j] := 0; for all nodes iN do d[i, i] := 0; for each arc (i, j)A do d[i, j] := cij and pred[i, j] := i; for each k := 1 to n do for each [i, j]NN do if d[i, j] > d[i, k] + d[k, j] then begin d[i, j] := d[i, k] + d[k, j]; pred[i, j] := pred[k, j]; end; end;

16. k • View as operation on matrix: d[k, j] k d[i, j] d[i, k] • If there exists no negative cycle, dk[i, i] = 0 for all i, k • Hence row k, column k do not change. •  maintain only one copy of the matrix. • If there exists a negative cycle  d[i, i] < 0 for some i.

17. Retrieving actual paths: pred[i, j] denotes the last node prior to node j in the tentative shortest path from node i to node j. Consider a path P from node k to node l. Starting from node l, let g=pred[k, l]. Then g is the last node prior to node l in P. Similarly, h = pred[k, g] is the node prior to node g in P, and so on. Repeat until we reach node k. • Thm 5.7. The Floyd-Warshall algorithm runs in O(n3).

18. Detection of negative cycles: Two tests for negative cycle: • If i = j, check whether d[i, i] < 0. • If i  j, check whether d[i, j] < -nC. • Use predecessor graph to retrieve negative cycle.

19. Other algebraic view • Reference: Combinatorial Optimization: Networks and Matroids, E. Lawler • Let d[i, j] = length of a shortest path from i to j. dk[i, j] = length of a shortest path from i to j using  k arcs. d0[i, i] = 0, d0[i, j] = + (i  j) dk+1[i, j] = minl { dk[i, l] + clj } Have d(n-1)[i, j] = d[i, j] • Define the new matrix multiplication  as P = A  B, Pij = mink {aik + bkj } We have Dk+1 = Dk  C D(0) = is identity matrix D(0)C = C multiplication  is associative. Dn-1 = Dn-2  C = … = ((D0  C)  C) …  C) = Cn-1 Also Cm = Cn-1, m  n-1.

20. Can compute Cn-1 as C → C2 → C4 → … → C2(k), 2kn-1. log(n) steps  O(n3 log n) Floyd-Warshall is a clever implementation of matrix multiplcation.

21. 5.7 Minimum Cost-To-Time Ratio Cycle Problem • Tramp steamer problem: Identify a directed cycle W of G with the maximum ratio ((i, j)W pij)/((i, j)W ij) Let cost cij of arc (i, j) as cij = -pij, and identify W with minimum ratio. Assume data are integral and ij  0 for (i, j)A and (i, j)W ij > 0 for every directed cycle W in G. Let (W) = ((i, j)W cij)/((i, j)W ij) and * is optimal value . Starting from arbitrary value of , define the length of each arc as lij = cij - ij. Then use negative cycle detection algorithm.

22. Three cases: (1) G contains a negative length cycle W. have (i, j)W (cij - ij ) < 0. Alternatively,  > ((i, j)W cij)/((i, j)W ij)  *. Therefore,  is a strict upper bound on *. (2) G contains no negative cycle, but does contain a zero-length cycle W*. (see Exercise 5.19 on how to identify a zero-length cycle) no negative cycle implies (i, j)W (cij - ij )  0 for every directed cycle W. Alternatively,   ((i, j)W cij)/((i, j)W ij) for every directed cycle W. Also, existence of zero-length cycle W* implies  = ((i, j)W* cij)/((i, j)W* ij). Hence  = *, and W* is a minimum cost-to-time ratio cycle. (3) Every directed cycle W in G has a positive length. have (i, j)W (cij - ij) > 0 for every directed cycle W. Alternatively,  < ((i, j)W cij)/((i, j)W ij) for every directed cycle W. Hence,  is a strict lower bound on *.

23. Search procedures: Guess a value  and use shortest path algorithm If negative cycle,   *. Use smaller value of . If zero-length cycle, then optimal. If positive length cycle,  < *. Use larger value of . • Sequential search algorithm: Let 0 be a known upper bound on *. Solve the shortest path problem with (cij - 0ij ) as arc lengths. Either find a zero-length cycle W or a negative cycle W. In the former case, optimal found, stop. In the latter case, let 1 = ((i, j)W cij)/((i, j)W ij) as our next guess. Have 0 > 1  *. Obtain sequence of values 0 > 1 … > k = *.

24. Binary search algorithm: Let [L, U] be the interval that contains *. (e.g. [-C, C]) Use 0 = (L + U)/2, and check for negative cycle. If a negative cycle exists, 0 > *. Reset U = 0. Otherwise, 0  *. Reset L = 0. After sufficiently large number of iterations, the search interval has a unique solution. • Let c(W) and (W) denote the cost and travel time of any directed cycle W, and let 0 = max{ij : (i, j)A}. Claim: The interval of size at most 1/02 contains at most one value from the set {c(W)/(W): W is a directed cycle} Let W1 and W2 be two directed cycles with distinct ratios. Then |c(W1)/(W1) - c(W2)/(W2)|  0, or | {c(W1)/(W2) - c(W2)/(W1)} / {(W1)(W2)} |  0. Smallest value of the left-hand side is 1/02. Hence if the interval becomes smaller than 1/02, using U as the value of  identifies the optimal cycle W.

25. M Shortest Paths without Repeated Nodes • Handout: Combinatorial Optimization: Networks and Matroids, E. Lawler • Enumerate simple paths in nondecreasing order with respect to the weights. • Let P be the set of all 1-n paths. • Find shortest path P1 initially. Modify the lengths using cdij  0. • Partition P – {P1} into subsets P1, P2, … , Pk, where k  n-1, so that we can determine the shortest path in each subset. • Let P2 be the shortest of these shortest paths identified. Suppose P2 Pj. • We then partition Pj - {P2} into subsets in the same way, and so on. • The procedure can be described as a tree. • How to partition the subset so that we do not lose any path and we can find a shortest path in each subset easily?

26. Suppose we have shortest paths P1, P2, …, Pm . Let P be the set of all 1-n paths. We divide the set P – {P1, P2, …, Pm} into P1, P2, … , Pk, where k  n-1. Each Pj includes paths with the condition that (a) they include the arcs in a certain specified path from node 1 to another node p, and (b) certain arcs from node p are excluded. Then we can find a shortest path in Pj by finding a shortest path from node p to n after deleting nodes in the path from 1 to p, and deleting the arcs excluded from node p. • In general, suppose that Pm is the shortest path (1, 2), (2, 3), … , (q-1, q), (q, n) in Pj. Paths in Pj have restrictions that they should include arcs (1, 2), (2, 3), … , (p-1, p), where p  q, and that certain arcs from node p are excluded. Partition Pj – {Pm} as follows: • If p = q, apply Dijkstra’s to obtain 1-n shortest path, subject to the conditions that arcs (1, 2), (2, 3), … , (p-1, p) are included, and that (p, n) is excluded, in addition to the arcs from p excluded for Pm.

27. If p < q, apply Dijkstra’s to find the shortest path from 1 to n, subject to the following sets of conditions: (1) Arcs (1, 2), (2, 3), … , (p-1, p) are included and arc (p, p+1) is excluded, in addition to the arcs from p excluded for Pm. (2) Arcs (1, 2), (2, 3), … , (p, p+1) are included and arc (p+1, p+2) is excluded. … (q-p-2) Arcs (1, 2), (2, 3), … , (q-2, q-1) are included and arc (q-1, q) is excluded. (q-p-1) Arcs (1, 2), (2, 3), … , (q-1, q) are included and arc (q, n) is excluded.