1 / 66

Deana M. Crumbling, M.S. Technology Innovation Office U.S. Environmental Protection Agency and

Modernizing Site Cleanup: Managing Decision Uncertainties Using the Triad Approach. Deana M. Crumbling, M.S. Technology Innovation Office U.S. Environmental Protection Agency and Kira P. Lynch, M.S. Innovative Technology Advocate Seattle District, US Army Corps of Engineers.

Download Presentation

Deana M. Crumbling, M.S. Technology Innovation Office U.S. Environmental Protection Agency and

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Modernizing Site Cleanup: Managing Decision Uncertainties Using the Triad Approach Deana M. Crumbling, M.S. Technology Innovation Office U.S. Environmental Protection Agency and Kira P. Lynch, M.S. Innovative Technology Advocate Seattle District, US Army Corps of Engineers

  2. Take-Home Message • Triad Approach= Integration of systematic planning, dynamic work plans, and real-time analysis as applied to wastes and contaminated sites  time & costs;  decision certainty • Theme for the Triad Approach = Managing the largest sources of decision error, especially the samplingrepresentativeness of data

  3. Talk Outline • “Modernizing” from what to what? • Broadening the “data quality” concept • Intuitive terminology for data quality • Case Study

  4. Characterization & Cleanup Strategy:Where We’ve Been

  5. Start here The Historical Process-Driven Approach • Easy to get lost in a maze, especially if no clear exit strategy • Sample and resample, hoping the data will tell us what to do • Reasonable for the 1970/80s • Limited experience, knowledge • Few tools available for either • data generation or cleanup • A rote one-size-fits-all process can get you through a maze; • BUT it will not be a • resource -efficient trip.

  6. START: “define the nature and extent of contamination.” 1 2 One-size-fits-all lab analyses 3 2 3 2 1 We need more information REPORT 2 1 6

  7. Characterization & Cleanup Strategy:Where We Are Heading

  8. EXIT START Experience, Knowledge as a Foundation Key Features: • Project planning (vs. process) • Multidisciplinary team • Stakeholders involved • Create opportunities for real-time decision-making to save time and $$ • Real-time decisions need real-time data • Project-specific CSM • identifies data gaps • evolve in real-time

  9. Systematic Planning Dynamic Work Plans Real-time Measurement Technologies A Systems-Approach Framework The Triad Approach

  10. Unifying Concept for Triad: Managing Uncertainty Systematic planning means… • Managing uncertainty about project goals • Includes stakeholders to set priorities & select ultimate goals • Identifies decision goals w/ tolerable overall uncertainty • Identifies major uncertainties (cause decision error) • Identifies the strategy to manage each major uncertainty • Managing uncertainty in data • Sampling uncertainty: Use field methods and a dynamic work plan to effectively manage sample representativeness • Analytical uncertainty: Various strategies available to manage any residual analytical uncertainties from field methods

  11. Technical Team Development • Assemble the project team by getting the right people involved • May include: statistician, chemist, hydrologist, legal or regulatory advisor, biologist, geologist, etc.

  12. Prevailing wind direction Transport Medium (air) Releasemechanism(volatilization) Exposure point Inhalation Inhalation Ingestion TransportMedium(soil) Waste(Source) Water table Ground water flow Transport medium (ground water) Conceptual Site Model (CSM)

  13. } Mix And Match Dynamic Work Plans • Real-time decision-making “in the field” • Evolve CSM in real-time • Implement pre-approved decision tree using senior staff • Contingency planning: most seamless activity flow possible to reach project goals in fewest mobilizations • Real-time decisions need real-time data • Use off-site lab w/ short turnaround? • Use on-site analysis? • Use mobile lab with conventional equipment? • Use portable kits & instruments? In all cases, must generate data of known quality

  14. Generating Real-time Data Using Field MethodsManage Uncertainty through Systematic Planning • Need clearly defined data uses—tie to project goals • Understand dynamic work plan—branch points & work flow • Project-specific QA/QC protocols matched to intended data use • Select field analytical technologies to • Support the dynamic work plan (greatest source of $$ savings) • Manage sampling uncertainty (improves decision quality) • Select fixed lab methods (as needed) to • Manage uncertainties in field data (as ONE aspect of QC) • Supply analyte-specific data and/or lower quantitation limits (as needed for regulatory compliance, risk assessment, etc.)

  15. Data Quality as a Tool to Achieve Decision Quality

  16. Perfect Analytical Chemistry Non- Representative Sample + “BAD” DATA Data is Generated on Samples Distinguish: Analytical Quality from Data Quality

  17. What is “Data Quality”? Data Quality = The ability of data to provide information that meets user needs • Users need to make correct decisions • Data quality is a function of data’s… • ability to represent the “true state” in the context of the decision to be made • The decision defines the scale for the “true state” • information content (including its uncertainty)

  18. D E C I S I O N D E C I S I O N Goal Making The Data Quality “Chain” Sampling Analysis Sample Support

  19. #2 #3 #1 Sample Support: Critical to Representativeness Sample Volume & Orientation The Nugget Effect Sample Prep Same Contaminant Mass in Nugget,but Different Sample Volumes Produce Different Concentrations The decision driving sample collection: Assess contamination resulting from atmospheric deposition 19

  20. D E C I S I O N D E C I S I O N e.g., Method 8270 Goal Making The Data Quality “Chain” Sampling Analysis Extract Cleanup Method(s) Result Reporting Sub- Sampling Sampling Design Sample Support Sample Preservation Determinative Method(s) Sample Preparation Method(s) All links in the Data Quality chain must be intact for Decision Quality to be supported !

  21. Partitioning Variability: Sample Location vs. Analytical Method Analytical (between methods) ~ 5% 500 On-site 416 Lab 331 On-site 286 Lab 2 7 39,800 On-site 41,400 Lab Sample Location ~ 95% 1,280 On-site 1,220 Lab 164 On-site 136 Lab 6 1 3 4 5 24,400 On-site 27,700 Lab 27,800 On-site 42,800 Lab

  22. Total Uncertainty Analytical Uncertainty Ex. 1 Sampling Uncertainty 3 X Ex. 3 Ex. 2 1/3 X Ex. 1 Ex. 3 Ex. 2 Summing Uncertainties Uncertainties add according to (a2 + b2 = c2)

  23. Less likely More likely Decision Quality vs. Analytical Quality ¢ ¢ ¢ ¢ ¢ ¢ ¢ $ $ $ $ $ $ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ Manylower analytical quality data points Higher information value of the data set Fewerhigh analytical quality data pointsLower information value of the data set Goal: A defensible site decision that reflects the “true” site condition

  24. Less likely More likely Decision Quality vs. Analytical Quality ¢ ¢ ¢ ¢ ¢ ¢ ¢ $ $ $ $ $ $ $ $ $ $ $ $ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ NOW, few high analytical quality data pointswill Highly informative data set Many “lower analytical quality” data points Higher information value of the data set Fewerhigh analytical quality data pointsLower information value of the data set Nearly Certain Goal: A defensible site decision that reflects the “true” site condition

  25. $ $ $ $ $ $ Fixed Lab Analytical Uncertainty Ex 1 Sampling Uncertainty ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ $ $ $ $ $ $ Remove hot spots Decreased Sampling Variability after Removal of Hotspots Ex 3 Fixed Lab Data Field Analytical Data Ex 2 Ex 1 Ex 2 Sampling Uncertainty Controlled through Increased Density Ex 3 Improve Decision Quality--Manage Uncertainties From This To This

  26. Sample Representativeness: Key to Environmental Data Quality Finally able to address this issue defensibly and affordably! • Cheaper analyses increase sample density • Real-time analyses support real-time decision-making • Rapid feedback permits course correction and smart sampling • Focus on overall data uncertainty • Remember that analytical uncertainty is but a fraction

  27. High spatial density Low DL + analyte specificity Manages sampling uncertainty Manages analytical uncertainty Definitive sampling quality Screening analytical quality Definitive analytical quality Screening sampling quality Marrying Analytical Methods to Make Sound Decisions Involving Heterogeneous Matrices Cheap screening analytical methods Costly definitive analytical methods

  28. High spatial density Low DL + analyte specificity Manages sampling uncertainty Manages analytical uncertainty Collaborative Data Sets Marrying Analytical Methods to Make Sound Decisions Involving Heterogeneous Matrices Cheap screening analytical methods Costly definitive analytical methods

  29. Managed sampling uncertainty: achieved very high confidence that all excessive contamination located and removed Managed analytical uncertainty as additional QC on critical samples; confirm & perfect decision levels used with field kits Case Study: Wenatchee Tree Fruit Site • Pesticide IA kits guide dynamic work plan: 56 tons soil segregated for incineration; 334 tons landfilled 230 IA analyses (2 different kits + QC) + 29 fixed-lab samples for 33 analytes • Clean closure data set • 33 fixed lab samples (floor & partial sidewall) +16 IA (sidewalls) • Demonstrate full compliance with all regulatory requirements for all • 33 pesticide analytes with >95% statistical confidence • Projected cost: ~$1.2M; Actual: $589K (Save ~ 50%) • Field work completed: <4 months; single mobilization

  30. Terminology to Integrate Data Quality into Decision Quality

  31. “Data Quality” Terminology Current terminology usage does not focus on the goal of decision quality • Irony: Great focus on the quality of data points; but overall quality of decisions easily unknown. • Current usage does not distinguish • Methods vs. data vs. decisions • The factors that impact each step in the process • Relationships between different aspects of quality

  32. = = Methods Data Decisions Screening Methods Screening Data Uncertain Decisions “Definitive” Methods “Definitive” Data Certain Decisions The SYSTEM functions as if it believes that… Distinguish: Analytical Methods from Data from Decisions

  33. Misleading Terminology Field Screening • False Implications: • All methods run in the field are screening methods. • All data produced in the field are of screening quality. • Fixed labs using definitive analytical methods don’t produce screening quality data. • Fixed labs don’t use screening methods.

  34. “Effective Data” “Decision Quality Data” Data of known quality that can logically be demonstrated to be effective for making the specified decision because both the sampling and analytical uncertainties are managed to the degree necessary to meet clearlydefined (and stated) decision confidence goals

  35. Wenatchee Tree Fruit Case Study: Soil Removal Using Field Analytical and a Dynamic Work Plan 35

  36. Wenatchee Tree Fruit Project Overview • Action required to achieve clean closure • 390 tons of soil removed (56 tons incinerated; 334 tons landfilled) • Total cost • Projected: ~$1.2M; Actual: $589K • Savings: ~50% • Total field time • Single mobilization: <4 months from start of field work until project completion • Outcome: Happy client, regulator, stakeholders

  37. Systematic Planning

  38. Coordinate/Assemble Teams • Who’s Who?: Coordinate with client, regulators and stakeholders • Planning Team: client, State, stakeholder, and USACE staff • Technical/Field Team: USACE staff, prime contractor staff, and subcontractor staff • Community outreach found little additional interest

  39. First Step: Identify Decisions • Problem: Pesticide contamination of vadose soil • Decisions to be made: • Locate and remove contamination • Remaining soil meet WA state cleanup stds • Manage excavated material for disposal • incineration • landfilling

  40. Desired Decision Confidence • Detect contamination • Grid size set to detect a 5 ft. x 10 ft. elliptical hotspot • Remove contamination so that remaining soil meets stringent WA state regulatory cleanup standards: • for 33 individual pesticide analytes • to a 95% statistical confidence

  41. State the Decision Goals (the Data Quality Objectives) • Provide results of sufficient analytical quality to • guide soil removal, • segregate and classify wastes for final disposal, and • confirm compliance with the required regulatory closure decision confidence. • Provide turnaround times for data that can support real-time decision-making in the field. • Provide sufficient sampling density to detect a 5X10 ft. hotspot. Field Analytical Field Analytical Fixed Lab Fixed Lab Field Analytical Field Analytical

  42. Managing Sampling Uncertainty Understanding how contamination occurred

  43. Site Grid with Probable Locations of Buried Bags Row A FR2/3 FR4/5 Row B Row C Col 1 Col 2 Col 3 Col 4 Col 5 Col 6 Col 7 Col 8 Col 9 Original Remediation Boundary X-Y Coordinate Origin Final Remediation Boundary NorthDrawing not to scale

  44. Optimize the Data Collection Design • Use a Dynamic Work Plan • Use immunoassay (IA) field kits for on-site analysis to guide DWP • Perform pre-field work pilot study to • Assess IA kit suitability • Estimate field/IA kit decision/action levels • Evaluate Geoprobe performance • Prepare SOPs and contingency plans • Use fixed lab analyses to generate site closure confirmation data sets

  45. Systematic Planning: Analytical Optimize On-Site Methods Pre-field work pilot study: • Compared IA to analyte-specific analyses • Understand cross-reactivity behavior of IA kits • Establish initial field decision/action levels: • 5 ppm for sum DDT; 0.086 ppm for sum cyclodienes • Project-specific SOPs established (PBMS) to improve project performance and save labor costs • Adjusted range of calibration standards • Increased the volume of the extraction solvent • Used a different solvent for the cyclodiene kit

  46. Immunoassay Kits One kit for Cyclodiene Family One kit for DDT Family 46

  47. Field Lab DDT,ppm Field Lab Endrin Regression Analysis (R2 = .60) Field Lab Endrin, ppm Data Comparability 47

  48. QA/QC for IA Kits • 3-point calibration & CCV w/ each batch (12 samples) • Reagent blank • Matrix duplicate (intra-laboratory sample split) • LCS [prepared from a purchased (soil) PE sample] • Split sample confirmation analysis (by fixed lab analysis) for samples representing critical decision points • Excavation boundaries • Clean closure data set for regulatory compliance

  49. Systematic Planning: Analytical Optimize Off-site Methods Certain fixed lab methods for pesticides were optimized using PBMS principles: • Organophosphorus (OP) pesticides: • SW-846 Method 8141 (GC/NPD) was changed to SW-846 Method 8270 (GC/MS) • Carbamates by GC: • A blend of EPA Water Method 632 and SW-846 Method 8141 (GC/NPD) was used • Paraquat in soil by spectrophotometry: • An industry developed method was used

  50. Original Remediation Boundary X-Y Coordinate Origin NorthDrawing not to scale Site Characterization Sample Site Grid Showing DP Core Sampling Locations Row A FR2/3 FR4/5 Row B Row C Col 1 Col 2 Col 3 Col 4 Col 5 Col 6 Col 7 Col 8 Col 9 Final Remediation Boundary

More Related