MINIMUM COST FLOWS: NETWORK SIMPLEX ALGORITHM

1. MINIMUM COST FLOWS: NETWORKSIMPLEX ALGORITHM A talk by: Lior Teller

2. A short reminder (it’s been three weeks..) • G=(V,E) is a directed graph • Capacity (קיבולת) function uij> 0 for every • Cost functincijfor every • Balance function b(v) for every we will have a number • b(v) > 0 – supply • b(v) < 0 – demand • cost, capacity and balance are integers

3. Minimum Cost Flows - Definition • Flow is a function on the edges. • A feasible flow is a flow that satisfies: • Capacity constraint: • Non negativity constraint: • Mass balance constraint: • The cost of a feasible flow f: • The min cost flow problem is: find a maximum feasible flow f, with the minimum cost

4. Introduction • Cycle Canceling • Successive Shortest Paths • Primal-Dual • Out of Kilter • Polynomial Algorithms – Capacity Scaling • Today – The Simplex Method

5. Introduction - The Simplex Method A powerful method for: • solving linear programming • solving constrained optimization problems • Minimum cost flow problems define a special class of linear programs • Network flow problems have considerable special structure • When implementation does not exploit underlying network structure – not a competitive solution procedure for solving minimum cost flow problems. • Interpreting the concepts of the simplex method appropriately as network operations, we can produce an algorithm that is very efficient Our goal: developing a network-based implementation of the simplex method

6. We won’t describe the simplex method for general linear programming problems and then show how to adapt the method for minimum cost flow problems. • We will develop the network simplex method directly in the context of network flow problems as a particular type of augmenting cycle algorithm. • No need for any background in linear programming • Building more directly on concepts we have developed already

7. Algorithm Outline • The central concept - spanning tree solutions, obtained by fixing the flow of every arc not in a spanning tree either at zero or at the arc's flow capacity. • From this we retrieve uniquely the flow on all the arcs in the spanning tree • Minimum cost flow problem has at least one optimal spanning tree solution • We move from one solution to another to find an optimal spanning tree solution • At each step introducing one new non-tree arc into the spanning tree in place of one tree arc. Thismethod is known as the network simplex algorithm: • Spanning trees correspond to the so-called basic feasible solutions of linear programming • The movement from one spanning tree solution to another corresponds to a so-called pivot operation of the general simplex method

8. Cycle free and spanning tree solutions For any feasible solution X: • Arc (i, j) is a free arc if 0 < xij < uij • Arc (i, j)is a restricted arc if xij= 0 or xij= uij Note that we can both increase and decrease flow on a free arc. In a restricted arc we can only increase or decrease the flow.

9. Cycle Free Solution: • A solution X is a cycle free solution if the network contains no cycle composed only of free arcs. Note: in a cycle free solution, we can augment flow on any augmenting cycle in only a single direction (some arc in any cycle will restrict us from either increasing or decreasing that arc's flow). Spanning Tree Solution • A feasible solution X and an associated spanning tree of the network is a spanning tree solution if every non tree arc is a restricted arc. Note: in a spanning tree solution, the tree arcs can be free or restricted

10. Next we establish a fundamental result of network flows: • Minimum cost flow problems always have optimal cycle free and spanning tree solutions. Thenetwork simplex algorithm will exploit this result by restricting its search for an optimal solution to only spanning tree solutions First we will see some required background for this (a small reminder)

11. Reminder:negative cycles • First assume that all arcs are un capacitated • The network shown in the figure contains positive flow around a cycle. • We define the orientation of the cycle as the same as that of arc (4, 5). • Letus augment θ units of flow along the cycle in the direction of its orientation. • Increases flow on forward arcs by θ units and decreases the flow on backward arcs by θ units. The per unit incremental cost for this flow change is the sum of the costs of forward arcs minus the sum of the costs of backward arcs: Per unit change in cost Δ = 2 + 1 + 3 - 4 - 3 = - 1.

12. Augmenting flow in the cycle decreases the cost • We set θ as large as possible while keeping all arc flows non negative We must satisfythe inequalities: 3 - θ ≥ 0 and 4 - θ ≥ 0 We set θ = 3 • Some arc in the cycle has a flow at value zero • The objective function value is strictly less than the initial solution

13. We can extend this observation in several ways: • a positive per unit change in cost (Δ) means we set θ as small as possible • If Δ = 0, we are indifferent to all solutions in the interval • If we impose upper bounds on the flow the range of flow that preserves feasibility an interval

14. (Cycle Free Property). If the objective function of a minimum cost flow problem is bounded from below over the feasible region, the problem always has an optimal cycle free solution. Proof: (With respect to a given flow x): • Arc (i, j) is a free arc if 0 < xij < uijor restricted arc if xij = 0 or xij = uij • Solution x has the cycle free property - contains no cycles entirely consisting of free arcs. • If a solution x contains a cycle (i.e. it is not cycle free) calculate the cycle cost • If the cycle cost is negative we send θ flow around the cycle in the direction of traversal • If it is positive send θ flow around the cycle in the opposite direction • In either of these cases we make θ as large as possible • Cycle must be eliminating • If the cycle cost is 0 we can remove the cycle by increasing the flow in either direction Applying this argument repeatedly removes all cycles from the solution.

15. (Spanning Tree Property) If the objective function of a minimum cost flow problem is bounded from below over the feasible region, the problem always has an optimal spanning tree solution. Proof: (This is an extension of the cycle free property): • Let S be the set of free arcs • If the free arcs connect all the nodes of G, then the solution is connected and acyclic (from the cycle free property)  thus forming a spanning tree • Otherwise, we add to S some restricted arcs until S connects all the nodes. The set S now forms a spanning tree of G, and the feasible solution x form which S is derived is referred to as a spanning tree solution.

16. A spanning tree solution partitions the arc set A into three subsets: • T - thearcs in the spanning tree • L- the non-tree arcs whose flow is zero • U- the non-tree arcs whose flow is the arc’s flow capacities We refer to the triplet (T, L, U) as a spanning tree structure.  we can associate a spanning tree structure with a spanning tree solution

17. Also possible: spanning tree structure  spanning tree solution We can also obtain a unique spanning tree solution corresponding to a given spanning tree structure (T, L, U): • set xij= 0 for all arcs (i, j) L • set xij = uij for all arcs (i, j) U • Determine the flow values for arcs in T – we will see how later

18. A Spanning tree can be: • Non-degenerate:every arc in a spanning tree solution is a free arc • Degenerate (מנוון): otherwise A spanning tree structure as feasible if its associated spanning tree solution is a feasible solution of the minimum cost flow problem. A spanning tree structure as optimal if its associated spanning tree solution is an optimal solution of the minimum cost flow problem.

19. The optimality condition

20. Proof:Let X* be the solution associated with the spanning tree structure (T, L, U). • Some set of node potentials π together with the spanning tree structure (T, L, U), satisfies (11.1). • Need to show X* is an optimal solution of the min cost flow problem. • As we have already seen in class, minimizing is equivalent to minimizing

21. Proof (continued): • This isequavilent to minimizing: • By the definition of solution X*: • for all • for all • The expression impliesthat the objective function value of the solution X will be greater than or equal to x*.

22. Maintaining a spanning tree structure • Algorithm moves from one feasible spanning to another (pivot operation) • Stops when the spanning tree structure satisfies the optimality condition • At each iteration, the algorithm adds one arc to the spanning tree in place of one of its current arcs. • The entering arc is a non-tree arc violating its optimality condition. • The algorithm adds this arc to the spanning tree, creating a negative cycle • It sends the maximum possible flow in this cycle • Drops an arc whose flow has reached its lower or upper bound We need a data structure to support this..

23. Spanning tree data structure Predecessor index: Each node i has a unique path from the root. The index pred(i) stores the predecessor of i on this path. Depth index: The index depth(i) stores the number of arcs on the path from the root to i. Thread index: defines a traversal of a tree. set to be the node in the depth-first search encountered just after the node itself. Provide a convenient means for visiting all descendants of a node.

24. Computing node potentials the network simplex algorithm moves from one spanning tree to the next maintaining =0 for every arc (i, j) in the current spanning tree. We wish to calculate note potentials for this purpose: • Adding a constant to all node potential does not alter reduced costs • Therefore, one node potential can be arbitrary – we set (1) = 0 • Using we can calculate all other node potentials by using the thread indices (example?)

25. Computing flows Every spanning tree structure (T, L, U) defines a unique flow x • xij= 0 for all arcs (i, j) L • xij= uij for all arcs (i, j) U To ease our discussion, for the moment let us first consider the uncapacitatedversion of the minimum cost flow problem • Deleting a tree arc (i,j) from the spanning tree decomposes the tree into two subtrees • Let T1be the subtree containing node iand let T2be the subtree containing node j • The cumulative balance of nodes in T1must be equal to that of T2 • In the spanning tree, arc (i, j) is the only arc that connects the subtreeT1 toT2  must units of flow

26. Working up from the leaves it is possible to calculate all arc flows (won’t go into details.. Need some intuition?) For the capacitated version of the problem, we add this initialization: (won’t go into details.. Need some intuition?)

27. Network Simple Algorithm • The network simplex algorithm maintains a feasible spanning tree structure at each iteration and successively transforms it into an improved spanning tree structure until it becomes optimal.

28. Obtaining an Initial Spanning Tree Structure • Solve the maximum flow problem  feasible solution • Construct an initial spanning tree structure by converting this solution into a spanning tree solution

29. Optimality Testing and the Entering Arc To determine weather a spanning tree structure is optimal we check: If the spanning tree structure satisfies the conditions it is optimal  we terminate Else, the algorithm selects a non-tree violating arc to enter the tree:

30. Pivot Rules For any eligible arc (i, j) we refer to IcijI as its violation. The network simplex algorithm can select any eligible arc to enter the tree and still would terminate finitely (with some provisions for dealing with degeneracy) Pivot rules for selecting entering arcs: • Dantzig's pivot rule • First eligible arc pivot rule • Candidate list pivot rule

31. Dantzig's pivot rule At each iteration - select an arc with the maximum violation to enter the tree. Motivation - the arc with the maximum violation causes the maximum decrease in the objective function per unit change in the value of flow on the selected arc  maximum decrease per pivot The algorithm performs fewer iterations than other choices for the pivot rule. However, The algorithm must consider every non tree  very time consuming.  the algorithm is not attractive.

32. First eligible arc pivot rule • Scan the arc list sequentially and select the first eligible arc to enter the tree • In a popular version of this rule, we examine the arc list in a wraparound fashion • Quickly identifies the entering arc the algorithm generally performs more iterations than it would with other pivot rules because each pivot operation produces a relatively small decrease in the objective function value

33. Candidate list pivot rule Selects the entering arc using a two-phase procedure consisting of major iterations and minor iterations: • major iteration: construct a candidate list of eligible arcs. • Examine arcs emanating from a node and add eligible arcs to the candidate list. • We repeat this process for next nodes until the list has reached max size or we have examined all the nodes. • Next major iteration begins with the node of the previous major iteration and examines nodes in wraparound • minor iterations: in each iteration selects an eligible arc from the candidate list with the maximum violation • Scans arcs in candidate list and selects maximum violation arc to enter the tree • Removes arcs that are no longer eligible from candidate list (due to changes in the node potentials) • Once the candidate list becomes empty or a reached limit on the number of minor iterations per major iteration  we rebuild the candidate list by performing another major iteration

34. Candidate list pivot rule (continued) By setting the: • maximum allowable size of the candidate list • number of minor iterations per major iteration we can obtain numerous different pivot rules. Example: • Dantzig'spivot rule: number of minor iterations per major iteration = 1 • First eligible arc pivot rule: maximum allowable size of the candidate list = 1

35. Leaving Arc • Suppose (k, l) is the entering arc • Creates exactly one pivot cycle W • Orientation of the cycle: • Same as that of (k, l) if (k, I) L • opposite the orientation of (k, l) if (k, I) U • Letand denote the sets of forward arcs and backward arcs in the pivot cycle • Maximum flow change on an arc (i, j) W: • We can augment flow along W feasibly

36. We refer to any arc (i, j) W that defines δas a blocking arc • We then augment δunits of flow and select an arc (p, q) with δpq =δ • Apivot iteration is non-degenerateif δ> 0 and is degenerateif δ= O • A degenerate iteration occurs only if T is a degenerate spanning tree. Observe: two arcs tie while determining the value of δ  the next spanning tree will be degenerate.

37. Identifying the pivot cycle • P(i) denotes the unique path from any node ito the root node • Pivot cycle consists of the arcs {(k, I)} U P(k) U P(l) - (P(k) ∩P(l)) • Find first common ancestor if k and l in O(n) using pred and depth • (example?) Note: is the net decrease in the cost per unit flow sent around the pivot cycle W (example? Remember that and π(1) = 0 imply is the cost from the root to u)

38. Updating the Tree • When leaving arc (p, q) is determinedfor a given entering arc (k, I), we updates the tree structure • If the leaving arc is the same as the entering arc, (when δ= δpq=upq), the tree does not change. • the arc (k, I) merely moves from the set L to the set U, or vice versa. • Else, the algorithm must perform more extensive changes. • The arc (p, q) becomes a non-tree arc at its lower or upper bound, depending on whether (in the updated flow) X pq= 0 or X pq= u pq • Adding arc (k, I) to the current spanning tree and deleting arc (p, q) creates a new spanning tree  the node potentials also change

39. Updating node potentials • Deleting (p, q) from the previous tree partitions the set of nodes into two sub-trees • TI – containing the root node • T2 – not containing the root node. This sub-treehangs from node p or node q • The arc (k, I) has one endpoint in TIand the other in T2 • The conditions π(1) = 0 and Cij- π(i) + π(j) = 0 imply: • Node potentials inTI remain unchanged • If lT2the node potentials in T2 decrease by (homework) • If k T2 the node potentials in T2increase by (which is negative) • Using thread and depth indices we update the node potentials quickly

40. Termination • Algorithm moves from one feasible spanning tree solution to another • Stops when the spanning tree structure satisfies the optimality condition • If each pivot operation is non-degenerate, the algorithm terminates finitely: • Recall – is the net decrease in the cost per unit flow sent around the pivot cycle W. After a non-degeneratepivot the cost of the new spanning tree structure is δ*units less • Every spanning tree structure has a unique associated cost The network simplex algorithm will encounter any spanning tree structure at most once • The network has a finite number of spanning tree structures • The algorithmwill terminate finitely. Degenerate pivots, pose a difficulty: The algorithm might not terminate finitely

41. Strongly feasible spanning trees • A poor choice of entering and leaving arcsleads to cycling(i.e., an infinite repetitive sequence of degenerate pivots) • Degeneracy is both a theoretical and practical issue • We willmaintaining a special type of spanning tree, a strongly feasible spanning tree We will show the network simplex algorithm terminates finitely

42. Terminology and definitions The tree arcs are either: • upward pointing (toward the root) • downward pointing (away from the root) Strongly feasible spanning tree: • A spanning tree T is strongly feasible if every tree arc with zero flow is upward pointing and every tree arc whose flow equals its capacity is downward pointing • A spanning tree T is strongly feasible if we can send a positive amount of flow from any node to the root along the tree path without violating any flow bound

43. We wish implement the network simplex algorithm so that it always maintains a strongly feasible spanning tree • must first find an initial strongly feasible spanning tree We append to the problem an artificial node, with index n + 1 referred to as the root, and an artificial arc for each node of the original problem. Each artificial arc will have lower bound zero, upper bound infinity, and a high costC For each node iV, we add the arc (i, r) and assign flow xir= b(i) if b(i) > 0,and arc (r, i) with flow xir= -b(i) otherwise • satisfies all supply and demand constraints • Strongly feasible

44. Note that a non-degenerate spanning tree is always strongly feasible • Adegenerate spanning tree might or might not be strongly feasible • Algorithm creates a degenerate spanning tree from a non-degenerate spanning tree whenever two or more arcs are qualified as leaving arcs and we drop only one of these  Therefore, the algorithm needs to select the leaving arc carefully so that the next spanning tree is strongly feasible.

45. Leaving Arc Rule: select the leaving arc as the last blocking arc encountered in traversing the pivot cycle W along its orientation starting at the apex w. • We show that the leaving arc rule guarantees that in the next spanning tree every node in the cycle W can send a positive amount of flow to the root node.

46. Property: Each node in the segment W2 can send a positive amount of flow to the root in the next spanning tree • Arc (p, q) is the last blocking arc in W • No arc in W2is blocking • Every node in this segment can send a positive amount of flow to the root via node w (the apex) along the orientation of W2

47. Property: Each node in the segment W1can send a positive amount of flow to the root in the next spanning tree We prove this observation by considering two cases: • If the previous pivot was a non-degenerate pivot, the pivot augmented a positive amount of flow δalong the arcs in W1. After the augmentation, every node in W1can send a positive amount of flow back to the root opposite to the orientation of W1via the apex node w (each node can send at least δunits to the apex and then at least some of this flow to the root since the previous spanning tree was strongly feasible) • If the previous pivot was a degenerate pivot, W1must be contained in the segment of W between node w and node k because the property of strong feasibility implies that every node on the path from node I to node w can send a positive amount of flow to the root before the pivot, and thus no arc on this path can be a blocking arc in a degenerate pivot. Now observe that before the pivot, every node in W1could send a positive amount of flow to the root, and since the pivot does not change flow values, every node in W1must be able to send a positive amount of flow to the root after the pivot as well.

48. We next show that in the next spanning tree, nodes not belonging to the cycle W can also send a positive amount of flow to the root • In the previous spanning tree (before the augmentation), every node j could send a positive amount of flow to the root and if the tree path from node j does not pass through the cycle W, the same path is available to carry a positive amount of flow in the next spanning tree • If the tree path from node j does pass through the cycle W, the segment of this tree path to the cycle W is available to carry a positive amount of flow in the next spanning tree and once a positive amount of flow reaches the cycle W, then, as shown earlier, we can send it (or some of it) to the root node This completes the proof that the next spanning tree is strongly feasible.

49. We now establish the correctness of the network simplex algorithm: • We have previously shown that each non-degenerate pivot strictly decreases the objective function value • The number of non-degenerate pivots is finite • The algorithm can, however, also perform degenerate pivots • We will show that the number of successive degenerate pivots between any two non-degenerate pivots is finitely bounded.

50. Suppose that arc (k, l) enters the spanning tree at its lower bound and in doing so it defines a degenerate pivot • In this case, the leaving arc belongs to the tree path from node k to the apex w • Just as seen before, node k lies in the tree for which the potentials of all nodes change by an amount . Since < 0, this degenerate pivot strictly decreases the sum of all node potentials • Since no node potential can fall below – n*C, the number of successive degenerate pivots is finite.