Two Sources of Error

1. Module 6:UncertaintyDon’t just calculate—first think about sources of error, and don’t double-count errors module 6

2. Two Sources of Error • Sampling • How/where/when/who makes the measurements • Population • Actual variability in what you are measuring module 6

3. Measurement error specific to… • Operator • Instrument • Lab • Procedure • Standard • Time (day of week, year, season) • Measurement level (harder to measure at low concentrations) module 6

4. Population Error • Ideally, estimate some aspect of homogenous “clump” of air, water, people • If population is totally homogenous, only one measurement is necessary • The more variability in the population, the more measurements you need module 6

5. Minimizing the effect of population uncertainty • Careful sampling plan, designed to include measurements from all “over” the distribution • Sampling plan to measure smallest “homogenous” parts of environment as possible • Careful adherence to identical procedures module 6

6. QC measurements designed to… • Identify where errors occur • Quantify errors (difference from “reality”) • Save $ by improving program • Produce estimates of how certain your conclusions can be… • …therefore allowing decisions based on what you really “can” know module 6

7. PM QC Results • Collocated • Flow rate checks done with routine standard • Flow rate checks done with an external standard • PEP intercomparisons of external instrument and lab • What to do with each? module 6

8. How EPA Summarizes QC • First, estimate uncertainty for each site • Use collocated results to calculate confidence interval for precision (CV) • Start with RPD (diff/mean) • Always use same pair and order module 6

9. See P&B DASC with PM Data module 6

10. PM2.5 Precision Estimate (40 CFR 58 App. A eq’n 11) module 6

11. 90% confidence limit for precision = 7.7% • Average over quarter = x microg/m3 +- 8% (with 90% confidence, from precision error alone) • Can use this as part of overall uncertainty estimate • Combine with bias estimates from flow rate and PEP audits module 6

12. To estimate bias… • Use PEP audit results, if available • Use any comparisons that are independent as possible • Use DASC PM2.5 Bias (Current PEP) tab • Calculates upper and lower 90% confidence intervals module 6

13. UCL and LCL(upper and lower confidence level) module 6

14. What does this mean? • UCL is ~ 10% • LCL is ~ -10% • Uncertainty of bias about 10% • Average bias of 7% could really be 7.7, or about 8% module 6

15. Combining precision and bias? • For rough estimate: square root of sum of squares • Start with d=diff/mean for all QC checks • Calc STDEV of each set of d’s • Square each STDEV • Add squares • Take square root, see if it makes sense! module 6

16. Precision for qrtr 1 of 2003 Collocated pairs, so PRECISION estimate (if A is not consistently higher/lower than B) module 6

17. Bias for 2003, based on independent checks module 6

18. Square Root of Sum of Squares module 6

19. Presenting Uncertainty • Use error bars or upper, lower lines in graph module 6

20. Uncertainty for Gaseous Methods • Simpler than PM • RPD between known and measured for automated and manual checks • Estimates validated with results of independent audits • QC checks produce estimates that include both precision and bias error module 6

21. Summarizing Uncertainty:COMMON SENSE first! • “Highest” estimate or worst-case calculation from results of independent audits (that encompass both precision and bias) • Uncertainty estimate should encompass (already include the error from) your internal assessments, so do not double-count results module 6