UMass Lowell Computer Science 91.503 Analysis of Algorithms Prof. Karen Daniels Fall, 2002

1. UMass Lowell Computer Science 91.503Analysis of AlgorithmsProf. Karen DanielsFall, 2002 Lecture 8 Tuesday, 11/19/02 Linear Programming

2. Overview • Motivation & Basics • Standard & Slack Forms • Formulating Problems as Linear Programs • Simplex Algorithm • Duality • Initial Basic Feasible Solution • Literature Case Study

3. Motivation & Basics

4. Motivation: A Political Problem Goal: Win election by winning majority of votes in each region. 100,000 voters 200,000 voters 50,000 voters Thousands of voters who could be won with $1,000 of ads Subgoal: Win majority of votes in each region while minimizing advertising cost. source: 91.503 textbook Cormen et al.

5. Motivation: A Political Problem (continued) Thousands of voters representing majority. urban suburban rural source: 91.503 textbook Cormen et al.

6. General Linear Programs real numbers Linear function variables Linear constraints Linear inequalities source: 91.503 textbook Cormen et al.

7. Overview of Linear Programming Objective function Convex feasible region Objective value source: 91.503 textbook Cormen et al.

8. Standard & Slack Forms

9. Standard Form objective function constraints source: 91.503 textbook Cormen et al.

10. Standard Form (compact) n-dimensional vectors m-dimensional vector mxn matrix Can specify linear program in standard form by (A,b,c). source: 91.503 textbook Cormen et al.

11. Converting to Standard Form source: 91.503 textbook Cormen et al.

12. Converting to Standard Form (continued) Transforming minimization to maximization Negate coefficients source: 91.503 textbook Cormen et al.

13. Converting to Standard Form (continued) Giving each variable a non-negativity constraint New non-negativity constraints If xj has no non-negativity constraint, replace each occurrence of xj with xj’ – xj”. source: 91.503 textbook Cormen et al.

14. Converting to Standard Form (continued) Transforming equality constraints to inequality constraints source: 91.503 textbook Cormen et al.

15. Converting to Standard Form (continued) Changing sense of an inequality constraint Rationale: source: 91.503 textbook Cormen et al.

16. Converting Linear Programs into Slack Form for algorithmic ease, transform all constraints except non-negativity ones into equalities slack variable for inequality constraint: define slack instead of s non-basic variables basic variables source: 91.503 textbook Cormen et al.

17. Converting Linear Programs into Slack Form (continued) objective function source: 91.503 textbook Cormen et al.

18. Converting Linear Programs into Slack Form (continued) set of indices of basic variables set of indices of non-basic variables Compact Form: (N, B, A, b, c, v) Slack Form Example Compact Form negative of slack form coefficients source: 91.503 textbook Cormen et al.

19. Formulating Problems as Linear Programs

20. Shortest Paths Single-pair shortest path: minimize “distance” from source s to sink t. Can we replace maximize with minimize here? Why or why not? source: 91.503 textbook Cormen et al.

21. Maximum Flow source: 91.503 textbook Cormen et al.

22. Minimum Flow source: 91.503 textbook Cormen et al.

23. Multicommodity Flow should be si source: 91.503 textbook Cormen et al.

24. Simplex Method

25. Solving a Linear Program • Simplex algorithm • Geometric interpretation • Visit vertices on the boundary of the simplex representing the convex feasible region • Transforms set of inequalities using process similar to Gaussian elimination • Run-time • not polynomial in worst-case • often very fast in practice • Ellipsoid method • Run-time • polynomial • slow in practice • Interior-Point methods • Run-time • polynomial • for large inputs, performance can be competitive with simplex method • Moves through interior of feasible region source: 91.503 textbook Cormen et al.

26. Simplex Algorithm: ExampleBasic Solution Standard Form Slack Form Basic Solution: Basic Solution: set each nonbasic variable to 0. source: 91.503 textbook Cormen et al.

27. Simplex Algorithm: Example Reformulating the LP Model Main Idea: In each iteration, reformulate the LP model so basic solution has larger objective value Select a nonbasic variable whose objective coefficient is positive: x1 Increase its value as much as possible. Identify tightest constraint on increase. For basic variable x6 of that constraint, swap role with x1. Rewrite other equations with x6on RHS. new objective value entering variable PIVOT source: 91.503 textbook Cormen et al. leaving variable

28. Simplex Algorithm: Example Reformulating the LP Model Next Iteration: select x3 as entering variable. new objective value entering variable PIVOT leaving variable New Basic Solution: source: 91.503 textbook Cormen et al.

29. Simplex Algorithm: Example Reformulating the LP Model Next Iteration: select x2 as entering variable. new objective value entering variable PIVOT leaving variable New Basic Solution: source: 91.503 textbook Cormen et al.

30. Simplex Algorithm: Issues to be Addressed… source: 91.503 textbook Cormen et al.

31. Simplex Algorithm: Pivoting entering variable leaving variable Rewrite the equation that has xl on LHS to have xe on LHS Update remaining equations by substituting RHS of new equation for each occurrence of xe. Do the same for objective function. Update sets of nonbasic, basic variables. source: 91.503 textbook Cormen et al.

32. Simplex Algorithm: Pivoting (continued) source: 91.503 textbook Cormen et al.

33. Simplex Algorithm: Pseudocode Feasibility initial basic solution to be defined later (detects infeasibility) optimal solution source: 91.503 textbook Cormen et al.

34. Simplex Algorithm: Correctness Proof Strategy • If SIMPLEX has initial feasible solution and eventually terminates, then it either • returns a feasible solution or • determines that the LP is unbounded. • SIMPLEX terminates. • SIMPLEX’s solution is optimal. source: 91.503 textbook Cormen et al.

35. Simplex Algorithm: Correctness Proof Approach, Part I • If SIMPLEX has initial feasible solution and eventually terminates, then it either • returns a feasible solution or • determines that the LP is unbounded. source: 91.503 textbook Cormen et al.

36. Simplex Algorithm: Correctness Proof Approach, Part II source: 91.503 textbook Cormen et al.

37. Duality Simplex Algorithm: Correctness Proof Approach, Part III

38. Linear Programming Duality max becomes min x variables go away RHS coefficients swap places with objective function coefficients y variables appear sense changes source: 91.503 textbook Cormen et al.

39. Duality Example source: 91.503 textbook Cormen et al.

40. Weak Linear Programming Duality Any feasible solution to primal LP has value no greater than that of any feasible solution to the dual LP. source: 91.503 textbook Cormen et al.

41. Weak Linear Programming Duality (continued) source: 91.503 textbook Cormen et al.

42. Finding a Dual Solution Finding a dual solution whose value is equal to that of an optimal primal solution… source: 91.503 textbook Cormen et al.

43. Optimality source: 91.503 textbook Cormen et al.

44. Initial Basic Feasible Solution

45. Finding an Initial Solution An LP model whose initial basic solution is not feasible source: 91.503 textbook Cormen et al.

46. Finding an Initial Solution(continued) Auxiliary LP model Laux: source: 91.503 textbook Cormen et al.

47. Finding an Initial Solution(continued) source: 91.503 textbook Cormen et al.

48. Finding an Initial Solution(continued) Original LP model Laux Laux in slack form source: 91.503 textbook Cormen et al.

49. Finding an Initial Solution(continued) PIVOT PIVOT source: 91.503 textbook Cormen et al.

50. Finding an Initial Solution(continued) source: 91.503 textbook Cormen et al.