Uncertainty Subgroup ARB Expert Workgroup

1. Uncertainty SubgroupARB Expert Workgroup October 15, 2010

2. Michael O’Hare, chair • Wes Ingram, ARB staff • Paul Hodson • Stephen Kaffka • Keith Kline • Michelle Manion • Richard Nelson • Mark Stowers • With assistance from Richard Plevin, UCB

3. Uncertainty and ILUC • How has uncertainty about the ‘real value’ of ILUC resulting from substitution of biofuel B for (fossil or other) fuel F been described? • Given a characterization (PDF function, range, table of estimate values, etc.) ‘how big’ is the uncertainty in a given estimate? Across all estimates? • How can it be reduced? • How should LCFS implementation accommodate it? • Cost of error • Objective function

4. Implementation Estimate physical GWI for MJ/MJ substitution of fuels Analyze optimal operational GWI and choose (implement) Fuel system responds with blends and driving behavior; nature “responds” with real GWI GHG emissions Social cost (DT etc.)

5. Three distinct variables for a single fuel • g*ij = estimate of “physical” GWIof fuel i from model j • gi = real “physical” GWI: if fuel i is substituted for fuel k on a MJ/MJ basis, additional GHG release is (gi -gk). gi is a random variable. • gi = operational GWI of fuel i used in LCFS implementation

6. Sources of uncertainty • Errors in assumed conceptual frameworks: relationships between biofuel policy and land use change • Errors and variety in model structure • Errors in model parameters • Intrinsic variation in real-world variables Are new model results converging on a narrow range of values?

7. Uncertainty in existing models • Modelers’ uncertainty reporting • Variation among model results • Random error vs. bias • Can a probability distribution be constructed?

8. Estimates of ILUC emissions from corn ethanol

9. Estimates of ILUC emissions from corn ethanol

10. Notes about prior table • Calculated from reported sensitivity results. • Analysis was performed using the GTAP-6 database, based on 2001 data, but the results were adjusted post facto to account for the 10% greater average corn yield in 2010. • Range is based on a combination of high and low values for various uncertain economic model parameters. • Range is based on evaluating alternative model assumptions. • Range is 95% CI around mean considering only the uncertainty in satellite data analysis and carbon accounting. • Analysis was performed using the GTAP-7 database, based on 2004 data, using the model to project out to 2020. • Effect of additional 106 GJ after meeting 5.6% mandate. Higher value is for greater trade liberalization.

11. Economic model uncertainty dominates Source: Plevin, O’Hare et al. 2010

12. EPA Uncertainty Analysis Soybean biodiesel N.B. Does not include uncertainties related to economic modeling or production period.

13. Large, but nebulous uncertainty “The lesson for policymakers is that results from economic models depend heavily on assumptions, and because we are trying to predict long-run human behavior, there can be legitimate differences in these assumptions.” – Dumortier et al. 2009 “[T]his modeling project has demonstrated how the current limits to data availability create significant uncertainty regarding outcomes predicted by these policy simulations.” – Al-Riffai et al. (IFPRI) 2010 [O]ur experience with modeling, data, and parameter estimation and assumptions leads us to conclude that one cannot escape the conclusion that modeling land use change is quite uncertain. Of course, all economic modeling is uncertain, but it is important to point out that we are dealing with a relatively wide range of estimation differences. – Tyner et al. 2010

14. Uncertainty characterization is inconsistent and incomplete • Searchinger et al.: Halving and doubling average emissions per hectare, other local SA • Dumortier et al.: different FAPRI versions; crop yields; GREET vs BESS; w/wo forest conversion • Hertel et al.: Systematic Sensitivity Analysis on few parameters, with notable limitations. • USEPA: Uncertainty analysis of remote sensing and emission factors. • CARB: OAT sensitivity analysis

15. Reducing uncertainty • Better models • More models • Fewer models • Better data for models • Selection of “best” models Most of this work is being done in other subgroups Uncertainty will not be reduced to zero

16. Policy and implementation challenge • The LCFS requires publishing a GWI with infinite precision for each fuel, but our knowledge of the ‘real’ GWI is imperfect • What is the best GWI to use within that framework?

17. Recall: three distinct variables for a single fuel • g*ij = estimate of “physical” GWIof fuel i from model j • gi = real “physical” GWI: if fuel i is substituted for fuel k on a MJ/MJ basis, additional GHG release is (gi -gk). gi is a random variable. • gi = operational GWI of fuel i used in LCFS implementation

18. Optimal value for GWI Elements of choice Action: Implement a set of values gi for fuel GWI’s Choose objective Factors: Distribution of gi(real values): data is {g*ij} System response R({gi}, {gi}) Cost of ‘errors’ gi – gi, R ≠Ρ(real response) The most likely value of giis not necessarily the optimal value of gi (safety factor principle).

19. Optimal gi’s Classical rule: Max{g} Expectation of value where f and h are distributions of g and R

21. What is the goal of the LCFS? • Climate • Minimize difference gi – gi ? • Minimize DT at a date (end of LCFS?) • Minimize additional GHG in atmosphere at a date (end of LCFS?) • Minimize total forcing up to a date? • Minimize probability that a condition will fail (LCFS increases one of the above)? • Other • Minimize social cost (including cost of DT, food effects, socioeconomic/welfare effects…?) • As of when (discounted how?)

22. Why might the cost of error be asymmetric? • Could it be better/worse for gi – gi to be +10 than -10? (Using too much/too little biofuel) • ILUC is irreversible even if biofeedstock production stops. • Encourage/obstruct advanced biofuels industry. • Food effects from overuse of biofuel

23. Why might the cost of error be non-linear? • Catastrophic events (Greenland ice cap, Gulf stream, peat feedback, etc.) • Socio-economic effects • Other costs may be non-linear even if symmetric: • Encourage/obstruct advanced biofuels industry. • Food effects from overuse of biofuel

24. Recognition of uncertainty entails attention to cost of error • Asymmetric distribution of gi means E(gi) is not necessarily the optimal gi . • This means the cost of error (functional form and asymmetry) also affects choice of gi

26. Uncertainty-explicit policy • Few examples to learn from in fuel policy • More theory, few examples in climate policy • Many examples in health and safety regulation • Safety factor concept • Risk aversity for large costs

28. How much investment in reducing uncertainty? • --LUC is a complex social and biophysical phenomenon with significant variation across space and time. • --Validating the role of any one factor/driver of LUC is difficult, at best, and very information-intensive. • --So, even with substantial additional investments in GTAP, some uncertainties surrounding current iLUC estimates are irreducible.

29. “...in many situations, limitations of data, scientific understanding, and the predictive capacity of models will make (uncertainty) estimates unavailable, with the result that they must be supplemented with other sources of information." • --Morgan et al. (2010), "Best Practices for Characterizing, Communicating, and Incorporating Scientific Uncertainty in Climate Decisionmaking." • "Expert elicitation," i.e., a formal assessment by individual experts, based on the full range of scientific evidence, of their best judgement of a subjective probability distribution for the value in question. • --Expert elicitations have been used to address many questions of climate science (e.g., role of aerosols). • --While not a substitute for further research, expert elicitation can allow for a formal expression of diversity of opinion not fully reflected in the literature

30. Extending policy scope • The greatest uncertainty and largest GHG effects are associated with a few landscape types/areas • A separate policy restricting use of such landscapes for economic purposes may reduce uncertainty and the risk of policy failure more than improved analysis and modeling.

31. Expert elicitation • --Experts could review full suite of information defining the influence of biofuels on iLUC, not only from GTAP, but including other models (FASOM/FAPRI), databases, and studies from other disciplines • --In lieu of quantitative values for iLUC, experts could recommend adaptive decision rules/guidance for categorizing feedstocks and fuel pathways according to relative iLUC risk, based on an assessment of all available information

32. Recommendations • Model estimates should present complete, systematic uncertainty characterization • Expert elicitation may help infer distribution of gi from {g*ij} • CARB should establish objectives • Cost of error should be examined and estimated • CARB should implement giwith explicit attention to cost of error and distribution of gi • CARB should explore mechanisms to incorporate estimates from multiple models in determining gi.