Defining and Expressing Uncertainty Or, how not to do the wrong thing more precisely

1. Robert S. Webb, Roger S. Pulwarty, Kristen Averyt Defining and Expressing UncertaintyOr, how not to do the wrong thing more precisely "In this world nothing can be said to be certain, except death and taxes." - Benjamin Franklin

2. Desire for scientific certainty and unanimity among "experts"a barrier decisions to mitigate or adapt to climate change

3. Uncertainty in climate predictions • Uncertainty - a fundamental characteristic of weather, seasonal climate, and hydrological prediction, and no forecast is complete without a description of its uncertainty • Uncertainty - an overarching term that refers to the condition whereby the state of a system cannot be known unambiguously. Probability is one way of expressing uncertainty. • Probabilistic forecasts convey uncertainty in the prediction. The converse are deterministic forecasts, which provide single predictions of the future state of a system, with no information regarding uncertainty

4. Uncertainty, managing risk, hedging strategies • “….The most important questions of life are, for the most part, really only problems of probability”(Laplace, Théorie Analytique des Probabilités, 1812) • "The research community should be focused on delivering a "coherent" product without major disagreements, to the user communities if it seeks to have the information acceptable and used wisely" (Changnon, 1994) • "Incorporating" or "building in" forecast information into a decision-making process does not automatically result from an explicit demonstration of “value” (Stewart, 1994) • Value is a complex, user-dependent function (Murphy, 1994)

5. Types of uncertainty • Epistemology: Are we asking the right question? • Risk: odds known • Technical uncertainty: odds unknown, but main parameters may be known • Ignorance: don’t know what we don’t know • Indeterminacy: causal chains/networks are open (physical outcomes depend on social and institutional behavior) • Disagreement: divergence over observation, framing and interpretation of issues.

6. Different types and sources of uncertainty characterizations Morgan and Henrion Funtowicz and Ravetz (1990) (1985) 1. Empirical quantities 1. Technical uncertainties 2. The functional form of models 2. Methodological 3. Disagreements among experts 3. Epistemological Decision analysis vs Decision-making Decision Event vs Decision Process

7. Sources of uncertainty • Observations • Instrumental error • Spatial/temporal sampling • Methodology/Technical • Variable quantities are uncertain because their value varies, seemingly stochastically • Uncertainty in model parameters based on expert judgment or on empirical estimates of a parameter's value • Uncertainty in model functional forms • Uncertainty in reduced model forms are based on comparisons of full-form with reduced-form model results • Behavior/Environmental • Personal/societal interactions and feedbacks

8. Minimizing uncertainty: Where is the uncertainty? -Problem Domain -science -organizational -community -political - adequate theory -multiple hypotheses & congruent management actions. -tractability (complexity) -confronting models w/data -independence/ rigor -novelty Gunderson and others

9. Managing Uncertainty: Where is it? -Problem Domain -science -organizational -community -cognitive -political • - expressions of power • - multiple equilibria • paths not taken • - NONE are scale invariant • - stability of institutions • novelty of approaches • role of epistemic groups • perception of risk • -multiple discourses • -juggling domains

10. Uncertainty in Weather vs. Climate “The entire [weather and climate] enterprise should take responsibility for providing products that effectively communicate forecast uncertainty information” NRC, 2006 Challenge.... What works for weather may not work for climate

11. Climate & Uncertainty IPCC AR4 WGI Terminology Virtually certain Extremely likely Very likely Likely More likely than not About as likely as not Unlikely Very unlikely Extremely unlikely Exceptionally unlikely >99% >95% >90% >66% >50% 33–66% <33% <10% <5% <1%

12. Climate & Uncertainty FAR: IPCC (1990) Broad overview of climate change science, discussion of uncertainties and evidence for warming. SAR: IPCC (1995) “The balance of evidence suggests a discernible human influence on global climate.” TAR: IPCC (2001) “Most of the warming of the past 50 years is likely (>66%) to be attributable to human activities.” AR4: IPCC (2007) “Warming is unequivocal, and most of the warming of the past 50 years is very likely (90%) due to increases in anthropogenic greenhouse gases.” Did it work?

13. Climate & Uncertainty Global Warming ‘very-likely’ Man-Made AP, Feb 1 2007 Science Panel calls global warming ‘unequivocal’ NY Times, Feb 2 2007 World urged to act on definitive report Financial Times, Feb 2 2007 New tack on global warming: Report’s dire forecast seen as call to action Chicago Tribune, Feb 4 2007

14. Climate & Uncertainty IPCC AR4 WGI Terminology IPCC AR4 WGII Terminology Virtually certain Extremely likely Very likely Likely More likely than not About as likely as not Unlikely Very unlikely Extremely unlikely Exceptionally unlikely >99% >95% >90% >66% >50% 33–66% <33% <10% <5% <1% Very high confidence High Confidence Medium Confidence Low confidence Very low confidence At least 9 out of 10 About 8 out of 10 About 5 out of 10 About 2 out of 10 Less than 1 out of 10 CCSP SAP 4.3 One size does not fit all!

15. Regional Climate Assessments: Uncertainty Language Quote & explain IPCC likelihood statements “Modeling projections...indicate that by the 2080s the most probable amounts of change in NYC and its Watershed Region will be 7.5ºF to 8.0ºF....” Uses and explains IPCC terminology but scientists make own judgments “Climate models project Colorado will warm 2.5ºF [1.5ºF to +3.5ºF by 2025....” “...annual average temperatures across Pennsylvania are projected to increase by 2.5ºF....” No uncertainty framework

16. Regional Climate Assessments: Uncertainty Language Quotes & explains IPCC likelihood statements “It is extremely likely(>95%) that global temperatures and temperatures over Chicago are expected to warm further over coming decades....” Uses and explains IPCC terminology but scientists make own judgments No uncertainty framework

17. Regional Climate Assessments: Uncertainty Language Quotes & explains IPCC likelihood statements “also likely, though less certain, are increases in precipitation quantity (particularly in winter and spring), precipitation intensity, intensity of tropical and extratropical cyclones (though their frequency may decrease), and sea-level variability....” Uses and explains IPCC terminology but scientists make own judgments No uncertainty framework

18. Regional Climate Assessments: Education All had detailed explanations of models, scenarios, downscaling, and climate projections

19. Moving Forward The need for climate information to inform decisions is immediate…and everyone is trying to fill that gap. Do we need a best practices for regional climate assessments, including how best to develop the data (model results) that go into the them? Can an uncertainty framework be developed that can be used in all regions by all sectors for all products? Can we take the IPCC model and develop a model framework? Can a typology approach work with different frameworks for different users? Or should regions be left to their own devices? Different uncertainty frameworks are being used depending on the type of climate information, the intended use for planning, and the scale.

20. Traditional Science and Policy Interactions Historical model of communicating uncertainty Climate Science Society: Impact and response

21. Pathways for Communicating Uncertainty Participatory assessments: Is the research relevant for decisions? Analysis of usefulness for policy/decision making arena Is the research compatible with existing decision models? Are the sources/providers of information credible to the decision maker? Are policymakers receptive to the problem and to research? Is the research accessible to policy/decision maker? Goals, Criticality, time frame, basis for decisions, usability, entry points, experience

22. Process for empowering decision making under uncertainty • Bring the delivery persons, research community etc, to an understanding of what has to be done to translate current information into usable information: assess this process • Interacting with actual and potential users to better understand the true informational needs, desired formats of information, timeliness of delivery etc.: How do people know what they’re missing? • Identify areas of agreement and divergence between scientists engaged in prediction/forecasting, the community that carries out research in impacts and human resources, and the community at large

23. Decision Support Planning Methods: Incorporating Climate Change Uncertainties into Water Planning 1. Classic decision analysis, 2. Traditional scenario planning, 3. Robust decision making, 4. Real options, and 5. Portfolio planning.

24. Decision Support Planning Methods: Incorporating Climate Change Uncertainties into Water PlanningClassic decision analysis • probability-based • systematically catalogs information and mathematically evaluating and ranking decision alternatives against multiple, potentially conflicting, decision objectives • illustrates the process with a decision tree or influence diagram • handles uncertainty through the use of probabilities. • used to find a preferred plan with the best value

25. Decision Support Planning Methods: Incorporating Climate Change Uncertainties into Water PlanningTraditional scenario planning • scenario-based • objective is development of a plan that best prepares for a plausible range of uncertain circumstances • scenarios are developed through the identification of critical uncertainties and driving forces • goal is to develop a range of future conditions that go beyond extrapolation of current trends and represent surprising but plausible conditions • scenarios are treated as equally likely to occur

26. Decision Support Planning Methods: Incorporating Climate Change Uncertainties into Water PlanningRobust decision making • combines features of both classic decision analysis and traditional scenario planning • systematic way of developing a strategy to best adapt to a wide range of plausible future conditions • uses existing or modified decision models to evaluate candidate strategies against large sets of quantitative scenarios that reflect future uncertainty. • used to identify major vulnerabilities within these strategies (unacceptable consequences). • hedging options and alternative strategies are develop to address vulnerabilities • Successive iterations lead to robust strategies

27. Decision Support Planning Methods: Incorporating Climate Change Uncertainties into Water PlanningReal options • helps identify strategies that adjust over time and balance risks. • determines sets of strategies that maximize value by using traditional discounted cash flow approaches. • flexible investment strategies are sought that can be • risk-adjusted with time and deferred into the future • Uncertainties are handled through the use of probabilities • Results are flexible in that they may incorporate delaying and phasing of decision implementation

28. Decision Support Planning Methods: Incorporating Climate Change Uncertainties into Water PlanningPortfolio planning • used in the financial world to select a portfolio containing a mix of assets or strategies that minimize exposure due to future scenarios • uncertainty is handled through the use of probabilities and Monte Carlo simulations • exposure to uncertainty is minimized through hedging • used extensively in the electric utility area

29. Approaches to risk communication and associated assumptions • ___________________________________________________________________________ • ApproachAssumptions and actions • Development andFrom the risk expert to the public--finite and • delivery of a risk message uni-directional • Aimed at bringing public views into line with expert views • Assumes expert view has more validity for decision-making • Dialogue about risk Interactive exchange of risk information--continuous • Aimed at balancing the content of risk message • Assumes both views contribute to decision-making • Social processesEngage in a process that addresses concerns about risk • of risk communication Aimed at enhancing understanding among stakeholders (DECISIVE AND NON-DECISIVE). Assumes the process is as important as the product • _____________________________________________________________________________ • Problem-definition and Framing: Mapping decision processes and information needs across temporal and spatial scales

30. Decisions are made every day under the cloud of uncertainty

31. Backup Slides

32. Managing an Uncertain Future: Climate Change Adaptation Strategies for California’s Water • array of adaptive water management strategies must be implemented to better address the risk and uncertainty of changing climate patterns • precipitation and runoff patterns are changing, increasing the uncertainty for water supply and quality, flood management, and ecosystem functions • existing systems has some capacity to cope with climate variability, but extreme weather events resulting in increased droughts and floods will strain that capacity to meet future needs • since some uncertainty will always exist, agencies need to perform sensitivity analyses of preliminary planning studies, and risk-based analyses for more advanced planning studies.

33. 2012 Central Valley Flood Protection PlanClimate Change Scope DefinitionWork Group Summary Report • planning frameworks need to be flexible enough to incorporate uncertainties related to climate change in managing risks. • planning approaches incorporating climate change probabilities, robust decision making, and adaptive management allow decisions to be more flexible and consider future advances in scientific understanding • planning starts by identifying key system vulnerabilities and/or critical risk thresholds • regional downscaled climate projections can be used to assess the likelihood of projected climate crossing system vulnerabilities or environmental thresholds • as climate science, global modeling, and downscaling methods improve, better information can be used to assess if probability of occurrence of critical climate condition thresholds have change.

34. NOAA Near-Term Opportunities Team: Assessing Assessments R. Webb, C. McNutt, D. Easterling, N. Cyr, K. Averyt, D. Goodrich, D. Kluck, K. Broughton Regional Climate Reports & Assessments • Provide guidance on the extent for downscaling as supported by the science • Provide datasets in easy to use formats for the non-climate scientist • Accurate portrayal of uncertainty (known unknowns) • Create an online climate assessment and climate impact assessment toolkit