1 / 43

Steps 3 & 4: Evaluating types of evidence for the Truckee River case study

Steps 3 & 4: Evaluating types of evidence for the Truckee River case study. Detect or Suspect Biological Impairment. Stressor Identification. Define the Case. List Candidate Causes. Decision-maker and Stakeholder Involvement. As Necessary: Acquire Data and Iterate Process.

harper
Download Presentation

Steps 3 & 4: Evaluating types of evidence for the Truckee River case study

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Steps 3 & 4: Evaluating types of evidence for the Truckee River case study

  2. Detect or Suspect Biological Impairment Stressor Identification Define the Case List Candidate Causes Decision-maker and Stakeholder Involvement As Necessary: Acquire Data and Iterate Process Step 3: Evaluate Data from the Case Step 4: Evaluate Data from Elsewhere Evaluate Data from Elsewhere Identify Probable Cause Identify and Apportion Sources Management Action: Eliminate or Control Sources, Monitor Results Biological Condition Restored or Protected

  3. Use all available types of evidence to make an inferential assessment • Types of evidence using data from the case • Spatial/temporal co-occurrence • Evidence of exposure or biological mechanism • Causal pathway • Stressor-response relationships from the field • Manipulation of exposure • Laboratory tests of site media • Temporal sequence • Verified predictions • Symptoms • Types of evidence using data from elsewhere • Stressor-response relationships from other field studies • Stressor-response relationships from laboratory studies • Stressor-response relationships from ecological simulation models • Mechanistically plausible cause • Manipulation of exposure at other sites • Analogous stressors italics indicates commonly available types of evidence

  4. Basic analysis strategy • Develop as many types of evidence, for as many candidate causes, as you can • you won’t have all types of evidence, for all candidate causes • most effective when you can compare results across candidate causes • Work through one type of evidence, then set it aside • avoid cognitive overload • Show your work • make your process transparent & reproducible • make use of appendices

  5. Let’s begin by figuring out what types of evidence we have for the Truckee…

  6. General vs. specific causation • General – Does C cause E? • Does smoking cause lung cancer? • Does increased water temperature reduce bull trout abundance in rivers? • Specific – Did C cause E? • Did smoking cause lung cancer in Ronald Fisher? • Did increased water temperature reduce bull trout abundance in my stream?

  7. Specific causation: using data from the case

  8. SUPPORTS Want paired measurements of proximate stressors & biological impairments, at locations where impairments are & are not observed. WEAKENS Spatial/temporal co-occurrence

  9. Want paired measurements of other steps in causal pathway & biological impairments, at locations where impairments are & are not observed. Causal pathway

  10. Want paired measurements of proximate stressors (or other steps in causal pathway) & biological impairments, at varying levels of exposure. Stressor-response relationships from the field

  11. Other types of evidence using data from case

  12. What types of evidence do we have, using data from the case?

  13. General causation: using data from elsewhere?

  14. Stressor-response relationships from other field studies

  15. Stressor-response relationships from the lab

  16. Other types of evidence using data from elsewhere

  17. What types of evidence do we have, using data from elsewhere?

  18. Now that we know what data we have, how do we analyze it?

  19. Spatial co-occurrence Do your impairment and your stressor co-occur in space? To Do: • Load relevant data file • Merge files • Make boxplots for each candidate cause Select ‘reference’ and impaired sites • Fill in worksheet

  20. Causal Pathway Does your data support the steps in the causal path between the stressor and the impairment? To Do: • Return to the conceptual diagram • Identify the steps in the causal pathway • Construct table to show whether data supports the steps between the stressor and the impairment • Fill in worksheet

  21. Verified Prediction - Traits Do data support predictions based on stressor’s mode of action? To Do: • Load relevant data file • Merge files • Make boxplot Select ‘reference’ and impaired sites • Fill in worksheet

  22. Verified Prediction - PECBO Do data support predictions based on stressor’s mode of action? To Do: • Load relevant data file • Merge files • Run PECBO • Load PECBO results file into CADStat • Merge files • Make boxplot • Sed • STRMTEMP • Fill in worksheet

  23. Stressor-response from elsewhere Does impairment decrease as exposure to the stressor decreases (or increases as exposure increases)? To Do: • Listen and ask lots of questions • Fill in the worksheet

  24. Randomized, controlled experiments The scientific standard for establishing cause and effect Key elements: • Replication: use of multiple test units (e.g. tanks, sites) • Controls: differ only by absence of the treatment • Randomization: random assignment of test units to “control” or “treated” status • Statisticalanalysis: estimate treatment effect (causal)

  25. Observational studies Often the only option for large-scale field studies Key elements: • Replication: collect data from multiple test units • Controls: ? • Randomization: ? • Statisticalanalysis: identify associations among variables of interest (non-causal) None None

  26. control realism, scale Trade offs: control vs. realism, scale Lab Experiment Field Experiment Observational Study

  27. Biomonitoring = Observational Issues for causal analysis: • Estimates of stressor effects are confounded by covarying factors • Analyst can’t randomly assign treatments (stressors) to sites * Reference sites are not experimental controls

  28. Analogous to clinical trials Does smoking cause lung cancer? • Estimates of stressor effects are confounded by covarying factors • Analyst can’t randomly assign treatments (stressors) to subjects * Non-smokers without lung cancer are not experimental controls

  29. Example using western EMAP* Using propensity scores to infer cause-effect relationships in observational data • Analysis and slides by Lester Yuan (USEPA), Amina Pollard (USEPA), and Daren Carlisle (USGS) • Original presentation given at North American Benthological Society conference, May 2008 *EPA Environmental Monitoring and Assessment Program (EMAP)

  30. EMAP-West Study Area • Measurements Collected: • Macroinvertebrates • Substrate composition • (SED) • Stream temperature • (STRMTEMP) • N = 838 Data collected by the EPA Environmental Monitoring and Assessment Program (EMAP)

  31. Total N vs. total taxon richness SLOPE = -16.5 Data from EMAP Western Pilot

  32. Total N covaries with many other factors

  33. Include covariates in the regression model to control for their effect. SLOPE = -16.5 Multiple linear regression Correlation of Total Richness and Total N (ug/L) Regression model includes: %agriculture, %urban, grazing intensity, %sands/fines, stream temperature, and log conductivity. SLOPE = -9.4

  34. Must assume that linear relationships are appropriate for all covariates. Regression model may extrapolate. Inclusion of certain variables may “mask” true effect: e.g., part of the effect of agriculture may be attributed to total N Potential issues with multiple regression

  35. Alternate approach: Stratify dataset r = 0.64 r = -0.01 r = 0.15 r = 0.27

  36. Model richness vs. total N within strata SLOPE = -10.7 SLOPE = -12.3 SLOPE = -9.7 How do we simultaneously stratify on many different covariates?

  37. Method developed in epidemiology to retroactively control for confounding effects in observational studies Sometimes called a quasi-experiment Intuitively: Model the magnitude of treatment (e.g. nutrient concentration) as a function of the covariates. The predicted magnitude of treatment at each site is its propensity score. Stratify the total set of observations by the propensity scores (i.e., group sites with similar scores). Six strata are typically used. Within each stratum, sites having different treatment levels (e.g. high vs. low nutrients) may be considered to have been “randomly assigned” to those treatment levels, because covariates have effectively been controlled by propensity score matching of “treated” and “control” sites. Propensity Score Matching

  38. Propensity Score Model Total N = f(percent agriculture, percent urban, grazing intensity, percent sand/fines, stream temperature, log conductivity,elevation, log catchment area, canopy cover, sampling day)

  39. Define 6 strata based on propensity scores

  40. Covariate values within strata Percent agriculture Grazing intensity

  41. Stratification by propensity score controls covariance of all modeled variables After stratification

  42. Total N vs. total taxon richness SLOPE = -16.5 Data from EMAP Western Pilot

  43. Total N vs. total richness: Stratified SLOPE = -3.3 (n.s.) SLOPE = -7.1* SLOPE = -7.1* SLOPE = -10.0*** SLOPE = -10.5*** SLOPE = -8.1***

More Related