Column Generation

1. Column Generation By Soumitra Pal Under the guidance of Prof. A. G. Ranade

2. Agenda • Introduction • Basics of Simplex algorithm • Formulations for the CSP • (Delayed) Column Generation • Branch-and-price • Flow formulation of CSP & solution • Conclusions

3. 10 5 3 Cutting Stock Problem • Given larger raw paper rolls • Get final rolls of smaller widths

4. Cutting Stock Problem (2) • Raw width (W) = 10 • No of finals given below • Minimize total no of raws reqd. to be cut

5. Agenda • Introduction • Basics of Simplex algorithm • Formulations for the CSP • (Delayed) Column Generation • Branch-and-price • Flow formulation of CSP & solution • Conclusions

6. Objective/ Cost fn Min -x1 -x2 5x1 - 8x2≥ -80 Constraints -5x1 - 4x2≥ -100 -5x1 - 2x2≥ -80 -5x1 + 2x2≥ -50

7. (8,15) (0,10) (12,10) (13,5) (10,0) 5x1 - 8x2≥ -80 -5x1 - 4x2≥ -100 -5x1 - 2x2≥ -80 -5x1 + 2x2≥ -50

8. Cost decreases in this direction -x1-x2 = -10 -x1-x2 = -20 -x1-x2 = -30

9. (8,15) (0,10) (12,10) (13,5) (10,0) Simplex method

13. Basic & non-basic 5x1 - 8x2≥ -80 -5x1 - 4x2≥ -100 -5x1 - 2x2≥ -80 -5x1 + 2x2≥ -50 (8,15,0,0,10,40) (0,10,0, 60,60,70) (12,10,60,0,0,10) (13,5,110,10,0,0) (10,0,130,50,30,0) (0,0,80, 100,80,50)

15. If all elements of it are non-negative, it is already optimal • Otherwise choose the non-basic variable corresponding to most negative value to enter the basic

16. How to find out outgoing basic variable?

17. Compute reduced cost If all >= 0, optimal stop. Otherwise choose the most negative component Corresponding variable enters basis Compute the ratios for finding out leaving basis Update basic variables and B for next step Few steps of simplex algorithm

18. Smart way of doing

19. d y Start

20. d y Reduced cost vector

21. d y Selection of non-basic variable

22. d y y vector

23. d y Selection of leaving basic variable

24. d y Preparation for next step

25. d y Entering variable in 2nd iteration

26. d y Leaving variable in 2nd iteration

27. d y Preparation for 3rd iteration

28. d y Final iteration

29. Done so far • Basics of (revised) Simplex algorithm • Formulation • Corners, basic variables, non-basic variables • How to move from corner to corners • Optimal value

30. Agenda • Introduction • Basics of Simplex algorithm • Formulations for the CSP • (Delayed) Column Generation • Branch-and-price • Flow formulation of CSP & solution • Conclusions

31. First step to solution - Formulation Kantorovich formulation

32. Kantorovich formulation example 1 2 3 4

33. 0 ≤ yk≤ 1 LP relaxation • Integer programming is hard • Convert it to a linear program

34. LP relaxation (2) • LP relaxation is poor • For our example, optimal LP objective is 120.5 • The optimal integer objective solution value is 157 • Gap is 36.5 • It can be as bad as ½ of the integer solution • Can we get a better LP relaxation?

35. Gilmore-Gomory Formulation Another Formulation

36. Example formulation 3*1 + 5*0 + 6*1 + 9*0 = 9 ≤ 10

37. LP relaxation of Gilmore-Gomory • LP relaxation is better • For our example, 156.7 • Integer objective solution, 157 • Gap is 0.3 • Conjecture: Gap is less than 2 for practical cutting stock problems

38. Issues • The number of columns are exponential • Optimal value is still not integer • Gilmore-Gomory proposed an ingenuous way of working with less columns • Branch-and-price to solve the other problem

39. Done so far • Basics of (revised) Simplex algorithm • Formulation • Corners, basic variables, non-basic variables • How to move from corner to corners • Optimal value • Formulation of the CSP • LP relaxation • Bounds

40. Agenda • Introduction • Basics of Simplex algorithm • Formulations for the CSP • (Delayed) Column Generation • Branch-and-price • Flow formulation of CSP & solution • Conclusions

41. Column Generation • In pricing step of simplex algorithm one basic variable leaves and one non-basic variable enters the basis • This decision is made from the reduced cost vector • The component with most negative value determines the entering variable

42. Column Generation (2) • In terms of columns, we need to find • That is equivalent to finding a such that • For cutting stock problem, cj=1, hence it is equivalent to • With implicit constraint • Pricing sub-problem

43. Solve Restricted Master Problem (RMP) Update RMP with Solve New Columns Pricing Problem Any Yes New Columns? No STOP (LP Optimal) Column Generation (3)

44. Example formulation

48. Done so far • Basics of (revised) Simplex algorithm • Formulation • Corners, basic variables, non-basic variables • How to move from corner to corners • Optimal value • Formulation of the CSP • LP relaxation • Bounds • Delayed column generation • Restricted master problem • Sub-problem to generate column • Repeated till optimal

49. Agenda • Introduction • Basics of Simplex algorithm • Formulations for the CSP • (Delayed) Column Generation • Branch-and-price • Flow formulation of CSP & solution • Conclusions

50. Integer solution • A formulation of the problem which gives ‘good’ LP relaxation • Solved the LP relaxation using column generation • But, the LP relaxation is not the ultimate goal, we need integer solution • In the example x2 is 39.5 • One way is to rounding