Introduction to Multistage Stochastic Programming

1. Introduction to Multistage Stochastic Programming Ryan Goodfellow – COSMO Laboratory

2. Outline • Overview of multistage optimization • Modelling example – investment portfolio • Application – multistage optimization of long-term production schedules for open pit mines

3. References • Books: • Shapiro, Dentcheva, Ruszyynski (2009) Lectures on Stochastic Programming: Modeling and Theory. Chapter 3. E-book available through McGill library (i.e. FREE for students). • King & Wallace (2012) Modeling with Stochastic Programming. • Online lecture notes: • Linderoth(2003) Multistage Stochastic Programming. http://homepages.cae.wisc.edu/~linderot/classes/ie495/lecture21.pdf • Application: • Boland, Dumitrescu, Froyland (2008) A Multistage Stochastic Programming Approach to Open Pit Mine Production Scheduling with Uncertain Geology http://www.optimization-online.org/DB_FILE/2008/10/2123.pdf

4. Introduction • Two-stage stochastic optimization • Make a set of decisions (first-stage) • Profit from outcome or clean up the mess (recourse) • Multistage stochastic optimization • Information is slowly revealed over time: • Make a decision for today (t) based on what I know today. • Observe the outcome from stochastic process in time t. • Set t = t + 1. • Go to step 1.

5. Introduction - Multistage • Notations used: • Let ξt, t=1,…,T, represent a sequence of random variables with a specified probability distribution (stochastic process). • Let ξ[t]:=(ξ1,…, ξt) denote the history of the process up to time t. • Let xt represent the decision vector, chosen at stage t.

6. Introduction - Multistage • Non-anticipativity: the values of xt may depend on the information ξ[t], the data available up to time t, but may not be influenced by the result of future observations (ξt+1,…, ξT). • xt is a stochastic process because it depends on ξ[t]– our decisions made today are influenced by previous decisions and outcomes.

7. Introduction - Multistage • Nested formulation for T-stage stochastic program: where:

8. Introduction - Multistage • Nested formulation for T-stage stochastic program: where: ξ1 :=(c1 ,A1 ,b1) are known first-stage information (non-random)

9. Introduction - Multistage • Nested formulation for T-stage stochastic program: where: ξt :=(ct,Bt,At ,bt) are data vectors where some or all elements may be random

10. Introduction - Multistage • Linear formulation: How can we formally introduce “history” into our models?

11. Introduction – Scenario Trees • Scenario trees are used to represent history of decision making. Period 1 Root node – f1(x1) is deterministic Period 2 Period 3 Period 4

12. Introduction – Scenario Trees • Scenario trees are used to represent history of decision making. Period 1 Period 2 Period 3 Period 4 S1 S2 S3 S4 S5 S7 S10 S6 S8 S9 A path from the root to a leaf is called a “scenario”.

13. Introduction – Scenario Trees • Scenario trees are used to represent history of decision making. Period 1 0.4 0.6 Period 2 0.25 0.25 0.5 Period 3 0.9 0.1 1 Period 4 Probabilities can be assigned to branches – defines the conditional probability of moving from one node to the next.

14. Introduction – Scenario Trees • Scenario trees are used to represent history of decision making. Period 1 0.6 Period 2 0.25 Period 3 0.1 Period 4 Probability of scenario #1: (0.6)*(0.25)*(0.1) = 0.015

15. Introduction – Scenario Trees • Scenario trees are used to represent history of decision making. Period 1 Period 2 Period 3 Period 4 Constraint matrices (A4, B4, b4) may be equal, however decision history from (x1, x2, x3) is different.

16. Introduction – Scenario Trees • Scenario trees are used to define non-anticipativity constraints. Period 1 Period 2 Period 3 Period 4 x1 x2 x3 x4 x5 x7 x10 x6 x8 x9 For the moment, let xkt represent the decisions made for scenario k and time t. Letxk represent the set of decisions (xk1,…,xkT)

17. Introduction – Multistage Formulation • Let k={1,…,K} denote the index of a given scenario. • Let pk denote the probability of scenario k={1,…,K}. • Let (Atk, Btk, xtk, btk) denote the decision variables and LHS/RHS coefficients for scenario k in period t={2,…,T}. • Let ctk denote the objective function coefficient for decision vector xtk for scenario k in period t.

18. Introduction – Multistage Formulation • Multistage linear formulation:

19. Introduction – Scenario Trees • Scenario trees are used to define non-anticipativity constraints. Period 1 Period 2 Period 3 Period 4 x1 x2 x3 x4 x5 x7 x10 x6 x8 x9 Without scenario trees or histories, solving x1-x10 would be solving each scenario independently.

20. Introduction – Scenario Trees • Scenario trees are used to define non-anticipativity constraints. Period 1 x11=x21=x31=x41=x51=x61=x71=x81=x91=x101 Period 2 x12=x22=x32=x42=x52=x62 x72=x82=x92=x102 Period 3 x13=x23 x43=x43=x53 x83=x93=x103 Period 4 x1 x2 x3 x4 x5 x7 x10 x6 x8 x9 We can group xit variables according to the scenario tree.

21. Introduction – Multistage Formulation • Multistage linear formulation: Non-anticipativity constraints

22. Introduction – Aggregation • Multistage stochastic programming is useful to integrate flexibility in models. • However, as the number of periods or branches increases, the scenario tree grows exponentially, making it very challenging to optimize. • Many types of multistage problems are meant to be re-solved in each period. • Solve multistage problem to get good answer for today. • Tomorrow, observe outcome of random variables, re-optimize decisions. • To control computational explosion, we can aggregate decisions.

23. Introduction – Aggregation • Scenario trees may be simplified through aggregation Period 1 0.6 Period 2 0.25 0.25 0.5 Period 3 0.9 0.1 1 Period 4 x14 x24 x34 x44 x54 x74 x104 x64 x84 x94 As depth of scenario tree increases, total probability for a deep node decreases, so it may be useful to simplify the problem.

24. Introduction – Aggregation • Scenario trees may be simplified through aggregation Period 1 Period 2 Period 3 Period 4 x14 x24 x34 x44 x54 x74 x104 x64 x84 x94 Probability: 0.3 0.015 0.135 As depth of scenario tree increases, total probability for a deep node decreases, so it may be useful to simplify the problem.

25. Introduction – Aggregation • Scenario trees may be simplified through aggregation Period 1 Period 2 Period 3 Period 4 x14=x24 x3 x44=x54=x64 x7 x84=x94=x104 Probability: 0.15 0.3 0.15

26. Introduction – Aggregation • Scenario trees may be simplified through aggregation Period 1 Period 2 Period 3 Period 4 x14=x24 x3 x44=x54=x64 x7 x84=x94=x104 Most models that include the time value of money are not heavily influenced by decisions made very far in the future – impact of aggregation may be negligible.

27. Example – Ryan & Veronica’s Retirement Jobs • Ryan and Veronica wish to have a large nest-egg for their retirement job. + + Secondary goal: Goats! Primary goal: Gastro-pub

28. Example – Ryan & Veronica’s Retirement Jobs • We have a set N of stocks that we can invest in. • We have T={1,…,40} investment periods before retirement. • Let ωit, iN, tT be the return on stock i in period t. • If we exceed goal G, we get $y to invest in the goat farm. • If we don’t meet G, we will need to win $r from the lottery to make up for the loss. • We can invest $bt in period t (hopefully not stochastic).

29. Example – Ryan & Veronica’s Retirement Jobs • Variables: • xit: amount of money to invest in stock i during period t. • y: money that we can spend on goats after retiring. • w: money needed from lottery winnings to open the gastro-pub.

30. Example – Ryan & Veronica’s Retirement Jobs • Deterministic formulation: Deterministic decision made today. Amount of money available for re-investment next period. Determine how much $ we have or need. Non-negativity constraints.

31. Example – Ryan & Veronica’s Retirement Jobs • Multistage formulation: • I may be naïve, but not enough to think that investing in the stock market is a deterministic problem. • I can simulate prices for stocks and combine them into scenarios. • Scenarios do not have to be dependent on the performance of a single stock, but the general performance over all stocks in a given year. • Create a binomial scenario tree based on “High/Low” returns for a given period. • Let ps define the probability of scenario s occurring. • Let define the set of scenarios that are indistinguishable to scenario s in period t.

32. Example – Ryan & Veronica’s Retirement Jobs Investment Portfolio Binomial Tree Period 1 Period 2 Low High Low High High Period 3 Low High High High Low Low Low High Period 4 … … … … … … … …

33. Example – Ryan & Veronica’s Retirement Jobs • Multistage formulation: Non-anticipativity constraints

34. A Multistage Stochastic Programming Approach to Open Pit Mine Production Scheduling with Uncertain Geology Authors: Natashia Boland, Irina Dumitrescu, Gary Froyland Written: October, 2008

35. Outline • Introduction • Deterministic case: MIP production schedule • Stochastic case: • Scenario-dependent mining & processing decisions • Conclusions • Discussion

36. IntroductionFrom geological estimation to generating cash flows • The first step to start a mine is to drill and interpolate the remaining volume. • Traditional geostatistical interpolation (kriging) produces a single “image” of what the deposit looks like. Drillholes & Orebody Estimated (Kriged) Deposit Cross-Section Plan View

37. IntroductionFrom geological estimation to generating cash flows • Open pit mine production scheduling problem: • Schedule the extraction sequence of blocks such that the net present value (NPV) of the mine is maximized. • Cash flows generated directly related to when a block is mined. • The catch: we aren’t certain about the value of a block until we have mined it. How can we decide which block to mine first?

38. IntroductionDefinitions • Volume of earth is discretized into K blocks: Empty volume of rock Discretized volume of rock Set of blocks K={1,2,…,K}

39. IntroductionDefinitions • In order to reduce the number of variables, the blocks are combined into N “aggregates”. Aggregates:

40. IntroductionDefinitions • In order to reduce the number of variables, the blocks are combined into N “aggregates”.

41. IntroductionVariables

42. Deterministic CaseMIP Production Schedule (D-MIP) Objective function Mine/mill extraction relationship Mill production capacity Mine production capacity Precedence/time constraints Variable definitions

43. Stochastic Case • Motivation: • Deterministic MIP ignores geological uncertainty • Use sequential simulation to produce many equiprobable “images”/scenarios of what the deposit looks like. • Goal: create a schedule that can adapt to changes in information using stochastic geological simulations. Simulated Deposits

44. Stochastic Case • Mining has endogenous uncertainty. • More information is revealed as the operation proceeds. • Uncertainty is generally reduced as more information is revealed. • The information that we reveal is based on the decisions that we made.

45. Stochastic Case • Multistage stochastic programming accommodates endogenous uncertainty through “adaptive” scheduling. • Rather than producing a schedule to accommodate all scenarios well, produce a series of schedules telling you how to adapt under certain conditions. • Two proposed models for stochastic scheduling: • Scenario-dependent processing decisions • Scenario-dependent mining & processing decisions • We are allowed to dynamically change mining sequence and processing decisions based on new information.

46. Stochastic Case • Non-anticipativity constraints: • If multiple scenarios have a common history, then we must make the same set of decisions for each of those scenarios. • In the mining context: we have mined the exact same areas of the mine, thus we have revealed the same geological information. We must make the same decisions for all of those scenarios for future mining/processing.

47. Stochastic Case • More definitions • Set of scenarios (geostatistical simulations): • α-{r,s}-differentiator:

48. Stochastic CaseScenario-Dependent Mining & Processing Decisions • Premise: • Mines are slow to react to changes. • We can change our mining sequence, however there is a (1 year) delay between when we get information and when we can make changes for the sequence. • Processing decisions are made ad-hoc. • We need to redefine our variables for each scenario:

49. Stochastic CaseScenario-Dependent Processing Decisions (SMP-MIP) Objective function Mine/mill relationship Mill production capacity Mine production capacity Non-anticipativity constraints Precedence/time constraints Variable definitions

50. Stochastic CaseScenario-Dependent Mining & Processing Decisions (SMP-MIP) • Non-anticipativity constraints: