Download
1 / 22
220 likes | 304 Views
Barriers to a More Even-Handed Treatment of Uncertainties in Projected Economic Costs and Health Benefits of Environmental Health Regulations Dale Hattis Clark University. Approach to the Problem.
E N D
Barriers to a More Even-Handed Treatment of Uncertainties in Projected Economic Costs and Health Benefits of Environmental Health Regulations Dale HattisClark University
Approach to the Problem • Adam has introduced the issues and prior observations of imbalance in the quantification of uncertainties in costs vs benefits. • I chose to first build additional observations from the most recent available EPA RIAs. • This gave me a basis to interview participants in creating these RIAs and discuss ideas about why they were crafted as the were. • Today I will first build some historical context for “impact analysis” before moving to the observations from recent RIAs.
Historical Background of “Impact Analysis”-- • Which “Facts” need to be assembled for value choices to be made? • Corps of Engineers tradition of traditional cost benefit analysis (promoting societal aggregate net benefit as the goal) • Environmental movement of Late 60s-early 70s aimed to right distributional wrongs • NEPA (1969) • Required analysis of environmental impacts not previously assessed. • Allowed enforcement by citizen suits. • Did not change the underlying decision-making mandate of affected agencies.
Political Opponents of Environmentalists Borrow a Page • 1974—Ford “Whip Inflation Now (WIN)” program institutes requirement for “Inflation Impact Statements” • Requirement is based on an Executive Order, not a congressional law, so enforcement is not via the courts but solely by administrative appeal to the Office of Management and Budget • For both kinds of impact statements, advocates were not powerful enough to change underlying legislative mandates, but sought impact statements as a way of less formally changing the mix of considerations influencing administrative choices. • Statement initially required to say whether the costs are “justified” by the benefits, not the stronger statement (later used under Regan) that benefits “outweigh” costs.
Development of Mandates for Uncertainty Analysis • Risk analysis emerges as a discipline beginning in the mid-late 1970s and early 1980s. • Agencies develop “conservative” procedures for estimating risks and associated risk reduction benefits—simplifying assumptions thought likely to overstate risks and therefore lead to protective standards—placing the burden of uncertainty on protected parties. • Such “conservative” estimation procedures, however did not correspond to the “risk neutral” arithmetic mean “expected value” estimates called for under traditional cost benefit criteria. • (They may also not have been as health-protective as the participants may have imagined or sold to managers).
Take-away from this background • This discussion sets the policy context for understanding the role of impact analyses in decision making. The impact analysis advocates did not have the power to change the basic legislative mandates, but did have enough influence to at least get their considerations analyzed and disclosed. This helps to understand why the mandated analyses were never placed at the very center of the decision-making processes and made highly rigorous, but tended to be conducted more with a goal to satisfice (be sufficient to appease) potential opponents after important policy choices had already been made
Bush 1991 “Regulatory Program of the United States” • Unfortunately, risk-assessment practices continue to rely on conservative models and assumptions that effectively intermingle important policy judgments within the scientific assessment of risk. Policymakers must make decisions based on risk assessments in which scientific findings cannot be readily differentiated from embedded policy judgments. This policy environment makes it difficult to discern serious hazards from trivial ones, and distorts the ordering of the Government’s regulatory priorities. In some cases, the distortion of priorities may actually increase health and safety risks.
OMB Circular A-4 (2003) • “When benefit and cost estimates are uncertain (for more on this see "Treatment of Uncertainty" below), you should report benefit and cost estimates (including benefits of risk reductions) that reflect the full probability distribution of potential consequences. Where possible, present probability distributions of benefits and costs and include the upper and lower bound estimates as complements to central tendency and other estimates.”
Obama (2011) • “The goal of the agency’s uncertainty analysis is to present both a central “best estimate,” which reflects the expected value of the benefits and costs of the rule, as well as a description of the ranges of plausible values for benefits, costs, and net benefits, which informs decision-makers and the public of the degree of uncertainty associated with the regulatory decision.”
Summary Results from New Observations • Overall 9 of the 14 new cases had clearly greater depth of analysis of uncertainties in benefits than in costs—in accord with the prior findings. • Of the 4 cases where we judge the treatment of uncertainties in costs and benefits to be reasonably comparable, three represent cases where there was little or no quantitative treatment of uncertainties at all—impact estimates, when given, were single points, often derived using “conservative” thought likely to overstate health/biological benefits. • In the remaining case of comparable uncertainty treatment, the treatment of cost uncertainties seems to reflect effects of controversy arising from challenges by both environmentalists and industry to a baseline estimate originally derived by/for the agency. • A final case of arguably greater analysis of cost uncertainty was the adoption of general drinking water standards for use in air planes—where there was some analysis of uncertainty in costs but quantitative estimation of benefits (and their uncertainty) was not attempted
Results of Interviews • Perceptions of a key EPA analyst • engineering cost estimates don’t come naturally attached to uncertainty statement, unlike the epidemiologically-based confidence limits for the mortality estimates and other effects associated with fine particle exposures. • overall the most important source of uncertainty is probably the changing baseline where effects both positive and negative occur far into the future (e.g. 2020 for the NAAQS particulate standard).
Another Interview • Perceptions of 2001-6 OMB/OIRA head John Graham: • EPA economists, while willing to use statistical confidence ranges as measures of uncertainty, were very reluctant to go beyond this to potentially more extensive quantitative characterizations of the uncertainties in analyses of costs and benefits. • EPA economists were comfortable with sensitivity analyses, and happy to illustrate results of different scenarios, but did not recognize the appropriateness of non-statistical analyses and necessarily subjective weightings of relative likelihood of different scenarios. • Economists’ position on this was reinforced by the general unwillingness of EPA lawyers to sanction the explicit incorporation of numerical uncertainties in official documents such as RIAs.
Speculation on Lawyers’ Reasons • Adam’s perception: I think laypeople (lawyers, judges) think they understand economics, but are more willing to admit they aren't scientists-- and hence they see an error bar for science and see it as reasonable, whereas an error bar for a cost figure connotes to them "go back and come up with the right answer; you are being lazy to stop with the range." • Additional possibility: lawyers want to preserve the traditional legal doctrine that courts should show “deference” to the judgments of expert executive agencies within the agencies’ field of expertise. Stating uncertainties in scientifically-quantified health benefits may be judged to disturb this tradition less than more accessible conclusions with respect to costs.
Interpretation • It is clear that the EPA analysts are very aware and cite in their documents the record of ex-ante vs ex post predictions of costs, and the empirical data on learning curve effects from the innovation/business literature. Learning curve effects resulting in reduced cost estimates have been incorporated into regulatory impact analyses for some mobile source emission rules. However the EPA analysis team seems to have been generally reluctant to draw on non-case-specific information to quantify Bayesian “prior” expectations for the bias and uncertainty in the engineering estimates of compliance costs that they receive.
Positivism vs Critical Realism—Role of Theory/”Metaphysics” • Positivism—Theory is to be avoided wherever possible in favor of simple analysis/summarization of the data. (Role of statistical analysis is to describe/summarize data.) • Critical Realism—building a picture of underlying causal relationships—some of which may not be directly observable--is the essence of the goal for scientific research.
Frequentistvs Bayesian Concepts of Probability • Frequentist—Probability statements can only be made by sampling from some data set; there is no information relevant to an analysis beyond the data. • Bayesian—Probability is a degree of belief or perception of the likelihood of an event or circumstance, considering all available information. • Contrast—implications of finding an alligator in the Charles River (near Boston) vs a cockroach in Aunt Suzie’s kitchen.
Conclusion • The most likely principal interpretation for the economists’ behavior is a frequentist orientation to statistical analysis—case specific information is required for quantitative uncertainty analysis. • The record of ex-ante vs ex-post comparisons of cost estimates could provide a reasonable starting point for understanding the direction and magnitude of likely errors in cost estimation. Moreover, distributional uncertainty estimates specific to particular cases could be constructed by quantitatively analyzing the past record for the effects of known likely causal determinants of systematic and random errors.
Likely Gains From Improved Uncertainty Analysis of Both Costs and Benefits • Use distributions to replace the single point “uncertainty” factors now used to adjust estimates of “acceptable daily intakes” and “reference doses” from primary observations for both cancer and non-cancer effects. However this requires setting explicit quantitative goals for RfDs in terms of residual incidence and severity of effects with a particular degree of confidence/uncertainty • Provides a long term incentive for improvement of the estimates and accountability for the estimators.
