Robust Decision Making

1. Robust Decision Making Robert Lempert RAND HDGC Seminar February 13, 2004

2. How Should Climate-Change Uncertainties Be Characterized for Decisionmakers? • Key climate change uncertainties include • “Basecase” emissions • Behavior of perturbed climate system • Value our descendants place on impacts of climate change • Costs of abatement with future technology • Climate-change decisionmakers must understand uncertainties to make effective choices

3. Predict Act Analytic Tools Often Vital to Clarify Thinking, But Can Illuminate Trees Rather Than Forest • Analytic tools often vital in improving complicated decisions: • Can successfully summarize vast quantities of information • Help address flaws in human reasoning • Traditional analytic methods assume well-characterized risks and policy choices based on predictions • But strategic decisions can go awry if decision-makers assume risks are well-characterized when they are not • Uncertainties are underestimated • Strategies can be brittle • Misplaced concreteness helps blind decision-makers to surprise

4. Scenarios Capture and Communicate Information About Future, But Hard to Link to Actions • Global Scenario Group offers three families of sustainability scenarios • Conventional worlds • Barbarization • Great Transformations • These scenarios • Capture a wide range of factors which may affect the future • Attempt to make an argument for a particular risk-management strategy

5. Pros Necessary to make policy Others will provide likelihood estimates if experts don’t Cons Little evidence to support judgments about probabilities Arguing over likelihoods distracts from reaching consensus on near-term actions Should Analysts Put Probabilities on Scenarios Such as Those Developed by SRES? Desire for concreteness driving IPCC towards placing probabilities on scenarios

6. Outline • Robust decision making • Example of robust decisionmaking as a means of characterizing uncertainty • Conclusions

7. Climate Change is a Problem of Decisionmaking Under Deep Uncertainty • Deep uncertainty is: • When we do not know, and/or key parties to the decision do not agree on, the system model, prior probabilities, and/or “cost” function • Under conditions of deep uncertainty, decision-makers: • Often seek robust strategies, ones which perform reasonably well compared to the alternatives across a wide range of plausible futures, evaluated with a range of values • Robust strategies are often (but not always) adaptive, that is they evolve over time in response to new information Often use choice of strategy, not additional information, to reduce uncertainty

8. Robust Decisionmaking (RDM) • Robust decisionmaking • Is an iterative, analytic process that identifies • Strategies that perform well over a wide range of futures • Remaining vulnerabilities of these strategies • Made possible by advances in computational capabilities • Characterizes uncertainties most important to the choice among strategies

9. Four Key Elements of Robust Decision Making • Consider large ensembles (hundreds to millions) of scenarios • Seek robust, not optimal strategies • Achieve robustness with adaptivity • Design analysis for interactive exploration of a multiplicity of plausible futures

10. Consider Ensembles of Many Scenarios On the occasion of the 1893 World Columbian Exposition, 74 experts wrote essays predicting what the United States would look like in 1993 • Most were wrong • But some were strangely close to the truth

11. Use Robustness Criteria to Judge Alternative Strategies • Under deep uncertainty, decision makers often seek robust strategies that work reasonably well over a wide range of plausible futures • We measure robustness according to degree of “regret,” which is defined as the difference between • the performance of a strategy in a given future, and • the performance of the best strategy in that future

12. Landscape of plausible futures Alternative strategies X Ensemble of scenarios Robust strategies Combine Human and Machine Capabilities

13. Tools to draw meaning from information Tools to represent information Users Exploratory Modeling Software Supports This Process • Exploratory modeling software enables users to • navigate through large numbers of scenarios and • Formulate rigorous policy arguments based on these explorations • CARsTM is java-based exploratory modeling software that: • Links to virtually any type of model and/or data • Supports interactive use of search and visualization to create, explore, compare, and understand large scenario ensembles

14. Outline • Robust decision making • Example of robust decisionmaking as a means of characterizing uncertainty • Conclusions

15. Example Application of Robust Decisionmaking Example: What choice of near-term actions will help ensure strong economic growth and a healthy environment over the course of the 21st century? • The RDM approach employed a simple method of representing information • “Toy” systems-dynamics model with 41 input parameters representing uncertainties about • future economic, demographic, and environmental trends • values and capabilities of future decisionmakers • Simple agent-based model of future decisionmakers • Four value functions based loosely on UN Human Development Index, which reflects interests of a range of stakeholders • Near-term strategies affect “decoupling” rate

16. Conventional World scenario Great Transition scenario Barbarization scenario Visualizations Capture Key Relationships Among Plausible Futures Landscape of plausible futures helps illuminate key challenges to ensuring strong economic growth and a healthy environment over the course of the 21st century. 5.0 China since 1960 Decoupling rate 4.0 3.0 U.S. since 1950 2.0 Russia since 1993 U.S. in 20th century 1.0 Brazil since 1980 0 U.S. 1890-1930 –1.0 India since 1960 0 1.0 2.0 3.0 4.0 Economic growth rate

17. Compare “Fixed” Near-Term Strategies Across Scenarios Assume near-term policy continues until changed by future generations Near Term Future Future decision-makers recognize and correct our mistakes Choose policies

18. Look for Robust Strategies Landscape of plausible futures Alternative strategies X Ensemble of scenarios Robust strategies

19. Strategies Should Be RobustAcross Multiple Measures of “Goodness” • Use measures inspired by UN’s Human Development Index (HDI) • Discounted, average rate of improvement in GDP/capita, longevity, and environmental quality (but no education level) time series • Four different weightings N$: North GDP/capita and longevity W$: Global GDP/capita and longevity NG: North GDP/capita, longevity, and environmental quality WG: Global GDP/capita, longevity, and environmental quality

20. N$ W$ NG WG No regret Mild A lot Overwhelming Speeding Decoupling Performs Well in Many Futures Using North HDI Measure Slight speed-up 5.0 4.0 Decoupling Rate U.S. since 1950 Conventional world scenario 3.0 2.0 U.S. in 20th century U.S. in 19th century 1.0 0 –1.0 0 1.0 2.0 3.0 4.0 Economic growth rate

21. N$ W$ NG WG No regret Mild A lot Overwhelming But Often Fails for Global Green Measure Slight speed-up 5.0 4.0 Decoupling rate Conventional world scenario 3.0 2.0 1.0 0 –1.0 0 1.0 2.0 3.0 4.0 Economic growth rate

22. 5.0 4.0 5.0 Conventional worldscenario 3.0 Conventional worldscenario 4.0 2.0 3.0 1.0 2.0 0 1.0 N$ W$ –1.0 0 NG WG 0 1.0 2.0 3.0 4.0 –1.0 0 1.0 2.0 3.0 4.0 No regret Mild A lot Overwhelming Exploration DemonstratesNo “Fixed” Strategy Is Robust Stay the Course Crash Effort Decouplingrate 5.0 5.0 Conventional worldscenario 4.0 4.0 Conventional worldscenario 3.0 3.0 2.0 2.0 1.0 1.0 0 0 –1.0 –1.0 0 1.0 2.0 3.0 4.0 0 1.0 2.0 3.0 4.0 Economic growth rate

23. X Design and Examine Additional Strategies Landscape of plausible futures Alternative strategies Ensemble of scenarios Robust strategies

24. Start with a Milestone, but Evaluate Progress Early and Modify Milestone If Necessary (Safety Valve) Present Future Select near-term milestone Does the carrying capacity change? NO Determine best policy to meet milestone YES Implement policy Is milestone achievable with current approach? Choose policies to maximize utility YES NO Relax milestone

25. N$ W$ NG WG N$ W$ NG WG No regret Mild A lot Overwhelming “Safety Valve” Strategy Appears Highly Robust Safety valve 5.0 5.0 4.0 4.0 U.S. since 1950 U.S. since 1950 3.0 3.0 Decoupling rate (%) + U.S. in 19th century U.S. in 19th century 2.0 2.0 U.S. in 20th century U.S. in 20th century 1.0 1.0 Worst Case 0 0 –1.0 –1.0 0 1.0 2.0 3.0 4.0 0 1.0 2.0 3.0 4.0 Economic growth rate (%) Economic growth rate (%)

26. Even Simple Scenario Generator Implies a High Dimensional Uncertainty Space

27. RDM Employs an Iterative Process Suggest candidate robust strategy • Initial choice is contingent on probability weighting across futures Characterize breaking scenarios • i.e., clusters of futures where strategy performs poorly independent of assumed weightings Identify tradeoffs among well-hedged strategies

28. Regret North & some South No increase All North Mostly North All South Stay the course Policy Scanning Across All Scenarios Suggests a Candidate Robust Strategy

29. Analytic Tools Generate “Narrative” Scenarios • RDM identifies low-dimensional, easy-to-interpret regions where candidate strategy performs poorly • Used Friedman and Fisher’s (1999) Patient Rule Induction Method (PRIM) • “Low Global Decoupling” scenario is defined by 3 of 41 parameters • Scenario suggests important data for consideration by decisionmakers

30. Regret in low-global- decoupling futures Regret in SV01.005.002 “Satisficing” Futures RDM Analysis Helps Policymakers Focus on a Small Number of Key Tradeoffs Assessment of adaptive “milestone” sustainability strategies over two computer-generated scenarios

31. } Stringent/stringent SV01-0.5%-0.2% Robust Regions Stringent/lax SV01-1%-1.5% Lax/lax SV02-1%-1.5% Stringent milestones and stringent cost constraints (SV01-0.5%-0.2%) SVab-x%y% a = N milestone b = S milestone x% = N cost threshold y% = S cost threshold Expected Regret Lax milestones and lax cost constraints (SV02-1%-1.5%) Stringent milestones and lax cost constraints (SV01-1%-1.5%) 1:100 1:10 1:1 10:1 100:1 Relative Odds of A Low Decoupling Future Analysis Ends by Characterizing Uncertainties which Drive Policy Choices

32. Outline • Robust decision making • Example of robust decisionmaking as a means of characterizing uncertainty • Conclusions

33. Predict-Then-Act Robust Decisions High Complexity Scenario Planning Few Low Hedging Opportunities Many Well-characterized Deep Uncertainty Different Methods Appropriate in Different Circumstances

34. Robust Decision Making Adds Another Means to Characterize Uncertainty for Decisionmakers • Information about future characterized by identifying robust strategies and their vulnerabilities • Complicated technology supports simple operational concept • Focus on alternative policies may require • Closer coordination between analyst and decisionmakers • Changes in process in organizations that use analysis

36. What About Surprises? Landscape of plausible futures Alternative strategies X Ensemble of scenarios Robust strategies

37. The Advisory Panel Suggested Several Potentially Stressing Surprises • Rapid technological advance that eliminates emissions • Plague that decimates population for twenty years • Future generations whose values (utility) are completely disconnected from concern about the environment

38. 5.0 5.0 Technological surprise 4.0 4.0 3.0 3.0 2.0 2.0 1.0 1.0 0 0 –1.0 –1.0 5.0 Population surprise Value surprise 4.0 3.0 N$ W$ 2.0 1.0 NG WG 0 –1.0 0 1.0 2.0 3.0 4.0 0 1.0 2.0 3.0 4.0 “Safety Valve” Strategy Is Still Robust, Even with Surprises Rate of change in emissions intensity 5.0 No surprise 4.0 3.0 2.0 1.0 0 –1.0 Economic growth rate