Presentation Transcript

1. Improving Weight of Evidence Approaches to Chemical EvaluationsCropLife America & RISE2014 Spring ConferenceArlington, VA

   Randall Lutter Frank Batten School of Leadership and Public Policy, UVa
2. Why focus on WOE?
3. Judging Weight of Evidence Approaches Focus on Chemical Evaluation May 2012
4. Review of the EPA’s Draft IRIS Assessment of Formaldehyde, NRC, 2011 One major, overarching issue is the use of weight of evidence in hazard identification The determination of weight of evidence relies heavily on expert judgment. As called for by others, EPA might direct effort at better understanding how weight-of-evidence determinations are made with a goal of improving the process (White et al. 2009).
5. Review of the EPA’s Draft IRIS Assessment of Formaldehyde, NRC, 2011 Strengthened, more integrative, and more transparent discussions of weight of evidence are needed. The discussions would benefit from more rigorous and systematic coverage of the various determinants of weight of evidence, such as consistency. (p.113)
6. WOE: Background “Weight of evidence is a measure of evidence on one side of an issue as compared with the evidence on the other side of the issue, or to measure the evidence on multiple issues.” WOE can be used in math, statistics and info theory, psychology, law, business, other uses. Source: Wikipedia But WoE phrase has many meanings and uses within regulatory chemical evaluations.
7. Questions for the 2012 Panel How should one judge whether a risk assessment of a chemical properly reflects the weight of the evidence? Weight? Evidence? Process? Hypothesis?
8. Questions for the 2012 Panel In determining weight of evidence, what is the importance of the following? Consistency with research from other labs Biological plausibility Mechanism / mode of toxicity Statistical power to identify an effect if one exists Consistency across species Appropriateness of endpoint selection Range of dose selection Site concordance
9. Hypothesis-based WoE Approaches Participants all supported an approach outlined and advocated by Rhomberg and colleagues (2010, 2011), Bailey et al (2002), and Prueitt (2011), and by Borgert et al. (2011) Like any WoE approach identified by Linkov et al., a hypothesis-based approach should include a systematic review of all relevant individual studies, including an evaluation of data quality and study reliability; a systematic evaluation of consistency, specificity, and reproducibility of specific outcomes 
10. Hypothesis-based WoE Approaches A critical aspect of a hypothesis-based WoE (HbWoE) approach is specifying explicitly the hypothesized basis for using the information at hand to infer the ability of an agent to cause human health impacts, or more broadly, other endpoints of concern. An “hypothesis” thus consists of the proposed basis for using the cited study results as evidence of human risk, and each line of evidence should have such an articulation of why it is deemed to bear on the larger question of human risk.
11. A Hypothesis-based WoE ApproachRequires Systematically review all studies that are potentially relevant to the causal question at hand (i.e., epidemiology, mode of action, pharmacokinetic, toxicology) and summarize the results. Within a realm of investigation (e.g., epidemiology, animal toxicology studies), examine the data for particular relevant endpoints, and/or modes of action, across studies. Identify and articulate hypotheses by which results from available studies could be used to infer the existence, nature, or magnitude of risks of concern, e.g., to human health.
12. A Hypothesis-based WoE ApproachRequires Trace through the logic within each line of evidence, thinking about how all of the relevant studies within each line of evidence support each other, considering consistencies and inconsistencies across studies. Similarly, trace through the logic across all lines of evidence as a whole and how they inform interpretation of each other.
13. A Hypothesis-based WoE ApproachRequires Formulate alternative accounts, composed of a set of proposals, hypotheses, assertions, and assumptions that together should provide tentative competing explanations for why all of the relevant observations came out as they did. Evaluate alternative, competing explanations based on how well each hypothesis is strengthened by supporting data versus how much each is weakened by requisite ad hoc assumptions, rejecting weaker explanations.
14. Hypothesis-based WoE Approaches Broader use of HbWoE approaches to evaluation of chemicals could improve the use of science in regulatory decision-making. It would increase the role of science, by leading to the explicit formulation and testing of hypothesized propositions. It would provide a structure that accommodates but confines the role of professional judgment in chemical evaluation. But HbWoE approaches are still evolving.