Uniform Federal Policy for Quality Assurance Project Plans

1. Uniform Federal Policy for Quality Assurance Project Plans Slides presented at Tyndall AFB 10 March 2010

2. 2 Outline Introduction Systematic Planning Conducting a Scoping Session Documentation Using Worksheets Checklist Best Practices Basewide QAPPs

3. Transition to UFP-QAPP 9 July 09 UFP-QAPP mandated for AF Restoration Program Applicable to MMRP Suggested for other programs UFP-QAPP more than lab Project management and objectives Measurements and data acquisition Assessment and oversight Data review 3 The Navy has used the UFP-QAPP format very successfully for MMRP projects. Not all worksheets will be applicable, but the systematic planning process associated with the UFP-QAPP certainly can be applied. Details contained in a UFP-QAPP cover much more than just analytical chemistry. However, chemists own UFP-QAPP through IDQTF; everything driven by EPA, Army, Navy, and AF chemists. Project management and data quality objectives are laid out at the beginning of a project. Action limits and corresponding analytical methods are chosen based on the project objectives. The review process and contingency plans are established before data are collected. Results are reviewed systematically to ensure all objectives are met and if they aren’t, corrective actions have already been established. Letter directs each contract service center to develop standard UFP-QAPP language requiring its use. AFCEE and USACE contracts now require use of the UFP-QAPP format.The Navy has used the UFP-QAPP format very successfully for MMRP projects. Not all worksheets will be applicable, but the systematic planning process associated with the UFP-QAPP certainly can be applied. Details contained in a UFP-QAPP cover much more than just analytical chemistry. However, chemists own UFP-QAPP through IDQTF; everything driven by EPA, Army, Navy, and AF chemists. Project management and data quality objectives are laid out at the beginning of a project. Action limits and corresponding analytical methods are chosen based on the project objectives. The review process and contingency plans are established before data are collected. Results are reviewed systematically to ensure all objectives are met and if they aren’t, corrective actions have already been established. Letter directs each contract service center to develop standard UFP-QAPP language requiring its use. AFCEE and USACE contracts now require use of the UFP-QAPP format.

4. UFP-QAPP Background UFP-QAPP Manual created by the Intergovernmental Data Quality Task Force (IDQTF) and signed by EPA (2004) and DoD (2005) Based on ANSI/ASQ E-4 Section 6 (Part B) Complies with EPA QA/R-5 and QA/G-5 Office of the Under Secretary of Defense Memorandum, 11 Apr 06 DoD Instruction 4715.15, 11 Dec 06 - Mandatory Procurement Policy and Procedures, November 2007 4 DoD has chosen the UFP format as its quality plan format and now the Air Force is falling in line. EPA has not gone through congress to make use of the UFP-QAPP format mandatory, so non-use is not subject to an NOV, but many EPA regions are moving towards only accepting quality plans in the UFP-QAPP format. IDQTF spent around 2 years building the manual and accompanying worksheets. Based on ANSI/ASQ E4 Section 6 (Part B) and EPA QA/R-5 and QA/G-5 Assistant Undersecretary of Defense Alex Beehler requested immediate implementation back in 2006. DoD Instruction 4715.15 authorizes the publication of DoD 4715.RR-M-1 “Department of Defense Quality Systems Manual for Environmental Laboratories,” DoD 4715.RR-M-2, “Uniform Federal Policy for Implementing Environmental Quality Systems,” DoD 4715.RR-M-3 “Uniform Federal Policy for Quality Assurance Project Plans” Procurement policy and guidelines performance standard is that documentation of project-specific quality assurance (QA) and quality control (QC) activities is in accordance with the UFP-QAPP manual.DoD has chosen the UFP format as its quality plan format and now the Air Force is falling in line. EPA has not gone through congress to make use of the UFP-QAPP format mandatory, so non-use is not subject to an NOV, but many EPA regions are moving towards only accepting quality plans in the UFP-QAPP format. IDQTF spent around 2 years building the manual and accompanying worksheets. Based on ANSI/ASQ E4 Section 6 (Part B) and EPA QA/R-5 and QA/G-5 Assistant Undersecretary of Defense Alex Beehler requested immediate implementation back in 2006. DoD Instruction 4715.15 authorizes the publication of DoD 4715.RR-M-1 “Department of Defense Quality Systems Manual for Environmental Laboratories,” DoD 4715.RR-M-2, “Uniform Federal Policy for Implementing Environmental Quality Systems,” DoD 4715.RR-M-3 “Uniform Federal Policy for Quality Assurance Project Plans” Procurement policy and guidelines performance standard is that documentation of project-specific quality assurance (QA) and quality control (QC) activities is in accordance with the UFP-QAPP manual.

5. Transition Impact 5 UFP-QAPP section 1.2: “The QAPP document may be referred to by another name, including field sampling plan (FSP), workplan, sampling and analysis plan (SAP). The UFP-QAPP integrates ALL technical and quality aspects for the life cycle of the project.” This diagram shows both content and document deliverables (both to AFCEE and to regulators). Can also look at this diagram as a process. Model plans were formats as well as content; UFP-QAPP is a format that requires specific content. UFP-QAPP may not contain all elements needed for a work plan. Model QAPP is being replaced with QA/QC compendium and QSM requirements. Illustration shows that all aspects of a project are thought about as a whole. UFP-QAPP section 1.2: “The QAPP document may be referred to by another name, including field sampling plan (FSP), workplan, sampling and analysis plan (SAP). The UFP-QAPP integrates ALL technical and quality aspects for the life cycle of the project.” This diagram shows both content and document deliverables (both to AFCEE and to regulators). Can also look at this diagram as a process. Model plans were formats as well as content; UFP-QAPP is a format that requires specific content. UFP-QAPP may not contain all elements needed for a work plan. Model QAPP is being replaced with QA/QC compendium and QSM requirements. Illustration shows that all aspects of a project are thought about as a whole.

6. Contract Deliverables Work Plans, Health & Safety, Sampling and Analysis Plans Can be ‘stand–alone’ documents or incorporated into the UFP-QAPP using the graded approach e.g., NAVFAC Sampling and Analysis Plan is the UFP-QAPP Work Plan refers to QAPP Munitions and Chemical Templates available At a minimum, other documents, such as basewide QAPP must be REFERENCED in the UFP-QAPP All information related to data gathering/project management should be contained and available 6 Depending on the scope and history of the project, some documents may be submitted separately. As the UFP-QAPP is phased in, Quality Plan deliverables should become more and more similar. The UFP-QAPP format facilitates consideration of all aspects in a project that relates to quality as a whole. Depending on the scope and history of the project, some documents may be submitted separately. As the UFP-QAPP is phased in, Quality Plan deliverables should become more and more similar. The UFP-QAPP format facilitates consideration of all aspects in a project that relates to quality as a whole.

7. Applicability UFP-QAPP Format should be applied to: New projects that do not already have a QPP New or existing projects that require changes to the existing QPP components where: There are new investigation requirements, or There are changes in project requirements that need to be documented, or There are issues that are not already addressed in the existing QPP. Flexibility Complete transition may not be appropriate in all cases Minor revisions documented through worksheets; appended 7 New projects can be defined as new data collection or new data use efforts. New projects can be defined as new data collection or new data use efforts.

8. UFP-QAPP Documents 8 Additional example QAPPs will be available through the IDQTF.Additional example QAPPs will be available through the IDQTF.

9. Elements of UFP-QAPP 9 The four basic element groups of a QAPP present a framework consistent with EPA Requirements for Quality Assurance Project Plans (EPA QA/R-5), which requires use of a systematic planning process. The project management and objectives elements of a QAPP ensure that the project has a defined purpose by documenting the environmental problem, the environmental questions being asked, and the environmental decisions that need to be made. The Measurement/Data Acquisition elements cover how project data will be collected, measured, and documented, and help ensure that resulting data are scientifically sound, of known and documented quality, and suitable for their intended use. Assessment/Oversight elements help to ensure that the resultant data quality is adequate for its intended use, and that appropriate responses are in place to address nonconformances and deviations from the QAPP. The Data Review element group encompasses the data review activities used to ensure that only scientifically sound data that are of known and documented quality and meet project quality objectives are used in making environmental decisions. The four basic element groups of a QAPP present a framework consistent with EPA Requirements for Quality Assurance Project Plans (EPA QA/R-5), which requires use of a systematic planning process. The project management and objectives elements of a QAPP ensure that the project has a defined purpose by documenting the environmental problem, the environmental questions being asked, and the environmental decisions that need to be made. The Measurement/Data Acquisition elements cover how project data will be collected, measured, and documented, and help ensure that resulting data are scientifically sound, of known and documented quality, and suitable for their intended use. Assessment/Oversight elements help to ensure that the resultant data quality is adequate for its intended use, and that appropriate responses are in place to address nonconformances and deviations from the QAPP. The Data Review element group encompasses the data review activities used to ensure that only scientifically sound data that are of known and documented quality and meet project quality objectives are used in making environmental decisions.

10. Graded Approach Intended use (complexity and scope of project) defines: Degree of documentation Level of effort Detail required UFP-QAPP Manual Text boxes contain examples 10

11. UFP-QAPP Worksheets The 37 worksheets contain the required elements of a UFP-QAPP Focus of critical elements Clear Data Quality Objectives Better Conceptual Site Model development Exit Strategy Documentation of decision path Sample design meets project needs Management of uncertainty/error Achievable goals Methods match the project 11 Purpose is to complete project, not complete a paperwork exercise.Purpose is to complete project, not complete a paperwork exercise.

12. UFP-QAPP Worksheets 12 There are many worksheets and this may look overwhelming. Worksheets are designed to capture needed elements of a quality plan. There are many worksheets and this may look overwhelming. Worksheets are designed to capture needed elements of a quality plan.

13. Worksheet Breakdown 13 There are some redundancies in the worksheets. Originally designed with this breakdown so sections of QAPP could be distributed, but in practice that is not really happening. Currently, an IDQTF streamlining team is looking at updating the UFP-QAPP guidance and worksheets.There are some redundancies in the worksheets. Originally designed with this breakdown so sections of QAPP could be distributed, but in practice that is not really happening. Currently, an IDQTF streamlining team is looking at updating the UFP-QAPP guidance and worksheets.

14. 14 Focus on Systematic Planning, not Worksheets Worksheets do not drive planning Worksheets document decision logic Worksheets not filled out in order Focus on the “important” worksheets Today’s training will describe and help optimize the documentation of your systematic planning process

15. Scoping Session Worksheets

16. Scoping Session Worksheets

17. 17 Systematic Planning Process SPP is based on the scientific method Logical organization of steps to acquire knowledge Ensures Appropriate amount and type of data for decision Data collected addresses physical, environmental, and chemical characteristics of the site Promotes communication between all organizations and individuals involved Scientific Method- Ask a Question, Do Background Research, Construct a Hypothesis, Test Your Hypothesis by Doing an Experiment, Analyze Your Data and Draw a Conclusion, Communicate Your Results Scientific Method- Ask a Question, Do Background Research, Construct a Hypothesis, Test Your Hypothesis by Doing an Experiment, Analyze Your Data and Draw a Conclusion, Communicate Your Results

18. SPP Elements Team-based approach Project goal, objectives, questions, and issues Project schedule, resources, milestones, and applicable requirements Data collection and analysis process matched to project objectives Collection and analysis requirements Process for generation, evaluation, and assessment of collected data 18 The systematic planning process is based on the scientific method and includes concepts such as objectivity of approach and acceptability of results. It uses a commonsense graded approach to ensure that the level of detail in planning is commensurate with the importance and intended purpose of the work and the use of available resources. This framework promotes communication between all organizations and individuals involved in an environmental project. Team should include program managers, regulators, chemists, hyrogeologists, risk assessors, statistician, and anyone else involved in project. The full team may not be needed at subsequent meetings after the initial scoping session. The worksheets will help guide and document the process. The systematic planning process is based on the scientific method and includes concepts such as objectivity of approach and acceptability of results. It uses a commonsense graded approach to ensure that the level of detail in planning is commensurate with the importance and intended purpose of the work and the use of available resources. This framework promotes communication between all organizations and individuals involved in an environmental project. Team should include program managers, regulators, chemists, hyrogeologists, risk assessors, statistician, and anyone else involved in project. The full team may not be needed at subsequent meetings after the initial scoping session. The worksheets will help guide and document the process.

19. Team Planning 19 The emphasis on scoping sessions may be different from previous QAPP preparations where the contractor worked in relative isolation to prepare the QAPP and then send it in for review.The emphasis on scoping sessions may be different from previous QAPP preparations where the contractor worked in relative isolation to prepare the QAPP and then send it in for review.

20. Scoping ensures all needs are defined adequately The penalty for ineffective planning often is greater conflict and extensive rework Prior Planning (Pre-scoping) Installation PM has overall project responsibility Contractor performs majority of work Identify items that need consensus Identify participants Data Users Data Producers Decision Makers Set preliminary agenda 20 Scoping Session Provide background information to scoping session participants ahead of time.Provide background information to scoping session participants ahead of time.

21. Participants and Documentation Key decisions and action items reached during each scoping session should be documented, the decisions reached should be consistently reflected in the rest of the QAPP All scoping sessions should be documented, including dates, participants, and purpose, using a separate Worksheet #9 21 Completing worksheets 5,6, and 7 may also help identify essential personnel and their roles and responsibilities. Everyone should understand the level of commitment (buy-in) from the participants at the scoping session. In other words, if the regulator attends, do they have authority to make decisions at the meeting? Or will they defer to their experts back at the office? Will they require things to be submitted in writing prior to agreeing to any aspect of the QAPP? This will be important to establish prior to the scoping session.Completing worksheets 5,6, and 7 may also help identify essential personnel and their roles and responsibilities. Everyone should understand the level of commitment (buy-in) from the participants at the scoping session. In other words, if the regulator attends, do they have authority to make decisions at the meeting? Or will they defer to their experts back at the office? Will they require things to be submitted in writing prior to agreeing to any aspect of the QAPP? This will be important to establish prior to the scoping session.

22. Worksheet #9 22 Extra text to document decision logic.Extra text to document decision logic.

23. 23 What’s Next?Define the Problem Review the reason for project initiation Identify chemicals of potential concern Identify matrices of interest Identify potential end use and exposure population Identify location and areal extent “Soils within the 2 acre site area of Building 3220 are historically known to have elevated levels of chromium and lead due to past site activities. Soils within the site must be determined to meet 2002 Ohio remediation requirements for metals (based on past land use). It is not clear if the previous site remediation has provided for sufficient clean-up levels to allow light manufacturing operations at the site.” What do we want to accomplish in the scoping session? Things to accomplish during the scoping session.What do we want to accomplish in the scoping session? Things to accomplish during the scoping session.

24. 24 Environmental Question Reflect project goals Identify action levels Define extent of concern Develop decision statement “Determine whether the designated 2 acres area defined as Building 3220 has been adequately remediated to light manufacturing risk based action levels for metals (OHREC 56784, 2002) and requires no further action or additional cleanup.”

25. 25 Site Information Reference previous site investigations Include site maps, history, current and future use, geology, hydrology, endangered species, natural resources List secondary data Note data quality for appropriate usability List pertinent site conditions and physical objects present Develop/clarify conceptual site model Come to a common understanding about site conditions Figure 18 of UFP-QAPP manual provides a flowchart for evaluating secondary data. A summary of secondary data and any limitations on its use will be presented on Worksheet #13.Figure 18 of UFP-QAPP manual provides a flowchart for evaluating secondary data. A summary of secondary data and any limitations on its use will be presented on Worksheet #13.

26. 26 Conceptual Site Model Problem statement should address lack of understanding in the CSM. Project work should refine CSM.Problem statement should address lack of understanding in the CSM. Project work should refine CSM.

27. 27 Decision Conditions Examine all possible outcomes Determine action for each Develop a series of “if…then” statements Link data results and possible actions These are the necessary decision logic steps to support an exit strategy One “if…then” results in No Further Action If [Principal Study Question #1] is above or equal to the contamination level then proceed with [Alternative Action A] or If [Principal Study Question #1] is below the contamination then proceed with [Alternative Action B]

28. 28 If…Then Statement Example If the total metal concentration for each metal in each sample within the two-acre area defined as Bldg 3220 is below the risk based levels (dry weight action limits) as defined by the State of Ohio Remediation Regulations for light manufacturing use, then the site has been successfully remediated with respect to metals, and continue with transfer of the site to the City. Decision tree may be useful to identify all outcomes and corresponding actions.Decision tree may be useful to identify all outcomes and corresponding actions.

29. 29 Worksheet #10 The QAPP should frame, for the reader or reviewer, the reasons for conducting the project, including historical information, current site conditions, and other existing data applicable to the project.The QAPP should frame, for the reader or reviewer, the reasons for conducting the project, including historical information, current site conditions, and other existing data applicable to the project.

30. 30 Project Quality Objectives Restate what the data will be used for, and identify who will use it e.g., risk assessment, plume definition, compliance Determine the type of data needed e.g., screening or definitive, target analyte or analytical group Analytical method Determine project action limits for each target analyte that will support project decision statements Identify measurement performance criteria, including precision, accuracy, quantitation, and completeness limits that ensure the level of quality needed to sufficiently address the problem and make defensible decisions Once the problem has been defined and the decisions to be made are discussed, the type, quantity, and quality of data that are needed to support those decisions are identified.Once the problem has been defined and the decisions to be made are discussed, the type, quantity, and quality of data that are needed to support those decisions are identified.

31. 7-Step DQO Process State the problem Identify the goals of the study Identify information input Define the boundaries of the study Develop the analytic approach Specify performance or acceptance criteria Develop the plan for obtaining data 31 For those familiar with the 7-step DQO process, this is a formalized approach to systematic planning. From EPA QA/G-4 February 2006; revised from earlier version of 7-step DQO process. The DQO Process is a series of logical steps that facilitate the resource-effective acquisition of environmental data. The DQO Process is used to establish performance and acceptance criteria, which serve as the basis for designing a plan for collecting data of sufficient quality and quantity to support the goals of the study. Use of the DQO Process leads to efficient and effective expenditure of resources; consensus on the type, quality, and quantity of data needed to meet the project goal; and the full documentation of actions taken during the development of the project. Project teams do not have to complete each of these during initial scoping. Concentrate on the CSM, Problem Statement, and Objectives. For those familiar with the 7-step DQO process, this is a formalized approach to systematic planning. From EPA QA/G-4 February 2006; revised from earlier version of 7-step DQO process. The DQO Process is a series of logical steps that facilitate the resource-effective acquisition of environmental data. The DQO Process is used to establish performance and acceptance criteria, which serve as the basis for designing a plan for collecting data of sufficient quality and quantity to support the goals of the study. Use of the DQO Process leads to efficient and effective expenditure of resources; consensus on the type, quality, and quantity of data needed to meet the project goal; and the full documentation of actions taken during the development of the project. Project teams do not have to complete each of these during initial scoping. Concentrate on the CSM, Problem Statement, and Objectives.

32. Data Needs Table 32 This worksheet is adapted from USACE Manual No. 200-1-2, Technical Project Planning Process, August 31, 1998, Appendix F. It is NOT a UFP-QAPP worksheet.This worksheet is adapted from USACE Manual No. 200-1-2, Technical Project Planning Process, August 31, 1998, Appendix F. It is NOT a UFP-QAPP worksheet.

33. 33 Worksheet #11 PQO Statement should identify what is being measured and where, what confidence level we can have in the data. Worksheet 11 has redundancies with 10. 11 is summary in nature; it could form executive summary for project. Specifics, like action levels and data quality indicators, will be used to populate later worksheets.PQO Statement should identify what is being measured and where, what confidence level we can have in the data. Worksheet 11 has redundancies with 10. 11 is summary in nature; it could form executive summary for project. Specifics, like action levels and data quality indicators, will be used to populate later worksheets.

34. Analysis Requirements Ensure target analyte list includes those most likely to be found on the site considering site history and other information Avoid including analytes that are unrelated to the project List project action limits and project quantitation limits for every target analyte Ensure project quantitation limits are at least 3 to 10 times lower than the action limits Ensure laboratory quantitation limits can meet project quantitation limits 34 Becoming more specific, we are going to compare/document analytical methodology with project action levels. Should be discussions between chemist and risk assessor to come to consensus on risk based action levels and available analyses. Tailor analytical requirements to meet risk-based action levels.Becoming more specific, we are going to compare/document analytical methodology with project action levels. Should be discussions between chemist and risk assessor to come to consensus on risk based action levels and available analyses. Tailor analytical requirements to meet risk-based action levels.

35. Limits 35 DoD Quality Systems Manual uses terms DL, LOD, and LOQ DL is usually statistically determined using 40 CFR 136 Appendix B method. LOD is lab spike at 2-3 times the DL, and must demonstrate detection. LOQ is lab spike that produces a quantitative result within specified limits of accuracy (and precision).DoD Quality Systems Manual uses terms DL, LOD, and LOQ DL is usually statistically determined using 40 CFR 136 Appendix B method. LOD is lab spike at 2-3 times the DL, and must demonstrate detection. LOQ is lab spike that produces a quantitative result within specified limits of accuracy (and precision).

36. Project Action Limits 36 This is an example of an issue that may be discussed in scoping sessionThis is an example of an issue that may be discussed in scoping session

37. Reference Limits and Evaluation TableWorksheet #15 37 Project quantitation limit should be 3-10 times greater than project action limit. Laboratory quantitation limits should be less than or equal to project action limit. If lab has not been hired, analytical method gives baseline performance. Complete a worksheet for each matrix/analytical group/concentration level.Project quantitation limit should be 3-10 times greater than project action limit. Laboratory quantitation limits should be less than or equal to project action limit. If lab has not been hired, analytical method gives baseline performance. Complete a worksheet for each matrix/analytical group/concentration level.

38. QSM and ERPIMS If Non-Detects, report parval=0 and mdl = LOD If Trace, report estimated value and detection limit for MDL Report LOQ for RL AFCEE flagging convention will not change. 38

39. 39 Measurement and Data Acquisition Problem Definition and Project Quality Objectives describe what is being done. Sampling design describes how it will be done. Worksheets 17-30 Moving from Project Management to Data AcquisitionMoving from Project Management to Data Acquisition

40. Sampling Design Purpose (decision data to support) Sample locations & numbers Grab or multi-increment samples Sample matrices Time frequency of sample collection Sample collection procedures Analytical parameters & concentration levels Complicated by contaminant variability 40 Things to think about.Things to think about.

41. Representative Sample Statistical collection? Professional judgment? Composite (multi-increment sampling) Mixing and homogenization Representative of the decision to be made on: Site Area Drum Individual sample container 41 Document decision logicDocument decision logic

42. Worksheet #17 Describe the project sampling approach Rationale for selecting the approach Rationale for selecting the number of samples and matrices Describe or refer to Worksheet #18 for analytical groups, concentration levels, number of samples and rationale for each Describe sampling frequency if applicable 42 Worksheet #18 will list each sample location and assign it a distinct ID number, and provide rationale for sampling location (e.g., background, grid, hot spot, downgradient, etc.). Worksheet #18 will list each sample location and assign it a distinct ID number, and provide rationale for sampling location (e.g., background, grid, hot spot, downgradient, etc.).

43. 43 Using Visual Sample Plan Can visualize spatial arrangement of locations Graphically displays statistical approach to sample design Will allow for consensus at scoping session May need “statistician” to communicate approach Sampling points are geographically referenced for sampler use

44. H0 = Site Is Less Than Action Limit 44 Using 1,4-dioxane in soil with an action limit of 160 ppm as a limited example. Alpha of 5% means 95% “certainty” or probability that site will not be unnecessarily cleaned up. (Falsely reject H0) Beta of 10% means 90% “certainty” or probability that site will not be called “clean” when it actually is “dirty.” (Falsely accept H0) Using 1,4-dioxane in soil with an action limit of 160 ppm as a limited example. Alpha of 5% means 95% “certainty” or probability that site will not be unnecessarily cleaned up. (Falsely reject H0) Beta of 10% means 90% “certainty” or probability that site will not be called “clean” when it actually is “dirty.” (Falsely accept H0)

45. Less Certainty 45 Change to 85% certainty and the number of samples is reduced to 10.Change to 85% certainty and the number of samples is reduced to 10.

46. Increasing Heterogeneity 46 If site is thought to be more heterogeneous, the number of samples for the same certainty is increased. Standard deviation is representative of site heterogeneity. Previous testing may give indication of what standard deviation to use.If site is thought to be more heterogeneous, the number of samples for the same certainty is increased. Standard deviation is representative of site heterogeneity. Previous testing may give indication of what standard deviation to use.

47. Sample Grid 47 A site map with geographically identified sampling points is produced. (Example: random grid sampling)A site map with geographically identified sampling points is produced. (Example: random grid sampling)

48. Summary Sheet 48 Text from the software can be inserted into worksheet #17.Text from the software can be inserted into worksheet #17.

49. 49 Worksheet #17 Worksheet 17 is a summary sheet that documents decision logic. Worksheet 18 will document specifics; each sample will be numbered and its purpose identified.Worksheet 17 is a summary sheet that documents decision logic. Worksheet 18 will document specifics; each sample will be numbered and its purpose identified.

50. Data ReviewWorksheets #34-#37 Objective is to assess usability for project decisions Use statistical assessments where applicable Relate to problem statement and project quality objectives Define the process at the start of the project Identify who will perform each data review task, and when it will be performed Document when project quality objectives are met and not met Plan corrective actions that address instances when quality objectives are not met 50 Data review is the process which data are examined and evaluated to varying levels of detail and specificity by a variety of personnel who have different responsibilities within the data management process. This QAPP element group encompasses the data review activities used to ensure that only scientifically sound data, that are of known and documented quality, and that meet project quality objectives, are used in making environmental decisions. The approach used for data review of a project must be appropriate to the project requirements.Data review is the process which data are examined and evaluated to varying levels of detail and specificity by a variety of personnel who have different responsibilities within the data management process. This QAPP element group encompasses the data review activities used to ensure that only scientifically sound data, that are of known and documented quality, and that meet project quality objectives, are used in making environmental decisions. The approach used for data review of a project must be appropriate to the project requirements.

51. 51 Data Review Elements Determine what verification, validation, and usability assessments need to be accomplished Lists of example assessments can be found in Tables 9-12 of the UFP-QAPP manual Consensus must be reached on assessments Populate the table before scoping session and have project team reach consensus Project data should be assessed to the measurement performance criteria identified in the QAPP Not all of the activities described here will occur in the scoping session, but thinking about these review steps, and the input that is required may help identify key elements that need to be identified in the planning stage.Not all of the activities described here will occur in the scoping session, but thinking about these review steps, and the input that is required may help identify key elements that need to be identified in the planning stage.

52. Data Review Overview 52 The UFP-QAPP Manual defines three distinct evaluative steps that are used to ensure that project data quality needs are met. These data review steps are required for all data collected and used in environmental projects. Step I, IIa, and IIb activities are organized separately, but they may be performed at the same time and/or by the same people. Step I: Verification evaluates that the specified requirements have been completed. Step II: Validation includes evaluating compliance with method, procedure, or contract requirements and extends to evaluating against criteria based on the quality objectives developed in the QAPP. (Step IIa assesses and documents compliance with methods, procedures, and contracts.) (Step IIb assesses and documents a comparison with MPC in the QAPP.) Step III: Usability Assessment determines the adequacy of data, based on the results of validation and verification, for the decisions being made.The UFP-QAPP Manual defines three distinct evaluative steps that are used to ensure that project data quality needs are met. These data review steps are required for all data collected and used in environmental projects. Step I, IIa, and IIb activities are organized separately, but they may be performed at the same time and/or by the same people. Step I: Verification evaluates that the specified requirements have been completed. Step II: Validation includes evaluating compliance with method, procedure, or contract requirements and extends to evaluating against criteria based on the quality objectives developed in the QAPP. (Step IIa assesses and documents compliance with methods, procedures, and contracts.) (Step IIb assesses and documents a comparison with MPC in the QAPP.) Step III: Usability Assessment determines the adequacy of data, based on the results of validation and verification, for the decisions being made.

53. Data Review Process 53 *Verification, validation, and usability reports do not have to be separate reports. Although steps are shown in a sequential manner, certain steps in the data review process may be performed simultaneously.*Verification, validation, and usability reports do not have to be separate reports. Although steps are shown in a sequential manner, certain steps in the data review process may be performed simultaneously.

54. Example InputsPlanning Documents 54 The example inputs come from Table 9 of the UFP-QAPP manual.The example inputs come from Table 9 of the UFP-QAPP manual.

55. Example Inputs Analytical Data Package 55

56. Example Inputs Sampling Documents 56

57. Example Inputs External Reports 57

58. ExampleWorksheet #34 58 Verification is a completeness check that is performed before the data review process continues in order to determine whether the required information is available for further review. It is essential for ensuring the availability of sufficient information for subsequent steps of the data review process. Internal or external is in relation to the data generator.Verification is a completeness check that is performed before the data review process continues in order to determine whether the required information is available for further review. It is essential for ensuring the availability of sufficient information for subsequent steps of the data review process. Internal or external is in relation to the data generator.

59. Data ValidationWorksheet #35 Identify processes that will be used to validate sample collection, handling, field analysis, and analytical laboratory project data Describe: How each item will be validated When the activity will occur What documentation is necessary Tables 10 and 11 of the UFP-QAPP manual provide suggested validation activities for Step IIa and IIb IIa validates to method/SOP criteria, IIb validates to project-specific requirements 59 Project-specific validation procedures are developed to identify and qualify data that do not meet the measurement performance criteria. The individual, identified by title and organizational affiliation, who is ultimately responsible for data validation is the person (lead chemist, project chemist, etc.) who will sign the project validation reports. Some of the activities listed for step IIa may have a QAPP-specific review element and would also be listed as activities under step IIb. Required documentation includes documentation necessary for validation as well as how the validation process is recorded.Project-specific validation procedures are developed to identify and qualify data that do not meet the measurement performance criteria. The individual, identified by title and organizational affiliation, who is ultimately responsible for data validation is the person (lead chemist, project chemist, etc.) who will sign the project validation reports. Some of the activities listed for step IIa may have a QAPP-specific review element and would also be listed as activities under step IIb. Required documentation includes documentation necessary for validation as well as how the validation process is recorded.

60. ExampleWorksheet #36 60 The title and affiliation of the person who will perform the validation should be included for each entry, since this may be different from the person ultimately responsible for the entire validation. Notice that IIa activities use method and procedure criteria for validation , and that IIb activities use QAPP requirements. Sources of sampling and analytical error should be identified and corrected as soon as possible after sample collection activities have begun. Incorporating an ongoing usability assessment process throughout the project, rather than just as a final step, will facilitate the early detection and correction of problems, thereby ensuring that PQOs are met.The title and affiliation of the person who will perform the validation should be included for each entry, since this may be different from the person ultimately responsible for the entire validation. Notice that IIa activities use method and procedure criteria for validation , and that IIb activities use QAPP requirements. Sources of sampling and analytical error should be identified and corrected as soon as possible after sample collection activities have begun. Incorporating an ongoing usability assessment process throughout the project, rather than just as a final step, will facilitate the early detection and correction of problems, thereby ensuring that PQOs are met.

61. Data Usability (Step III) Determine how quality of the data affects ability to support decision making Things to consider: Does the data tend to confirm or refute your CSM? What do the quality system results tell you about how the project was implemented? What effect do deviations have on data usability? Table 12 of the UFP-QAPP manual provides suggested usability assessment activities 61 A usability assessment considers whether data meet project quality objectives as they relate to the decision to be made, and evaluates whether data are suitable for making that decision. All types of data (e.g., sampling, on-site analytical, off-site laboratory) are relevant to the usability assessment. The usability assessment is the final step of data review and can be performed only on data of known and documented quality (i.e., verified and validated data).A usability assessment considers whether data meet project quality objectives as they relate to the decision to be made, and evaluates whether data are suitable for making that decision. All types of data (e.g., sampling, on-site analytical, off-site laboratory) are relevant to the usability assessment. The usability assessment is the final step of data review and can be performed only on data of known and documented quality (i.e., verified and validated data).

62. Summarize the usability assessment process Describe evaluative procedures used to assess measurement error Identify personnel responsible for performing the usability assessment Describe the documentation that will be generated during the usability process and how the results of the process will be presented Discuss how the project team will come to consensus on the usability of the data Data Usability Elements 62

63. ExampleData Usability Elements 63

64. ExampleData Usability Elements (cont’d) 64

65. Worksheet #37 65 To simplify some entries, can create a table (adding column to Table 12) and insert in Worksheet #37, but text needs to explain and rationalize processes.To simplify some entries, can create a table (adding column to Table 12) and insert in Worksheet #37, but text needs to explain and rationalize processes.

66. Data Review – How much?Streamlining Criteria Level of risk associated with the target analytes Cost and schedule Specific decisions for which the data will be used Complexity of analysis Political attention to project Recurring sampling events Proximity of results to action levels Outcome of project-specific audits 66 Streamlining data review is meant to reduce time and costs while still confirming the quality of the data. The data review process may be streamlined in order to meet schedules or cost constraints. The risk must be weighed by the project team. Planning the data review process ensures that budget dollars are available for critical data review identified by the project team. The QAPP should also contain decision criteria that allow for revision of the initial streamlining plan. Verification is not subject to streamlining. It may be possible to identify critical (most significant) samples and focus data review on those samples. Streamlining data review is meant to reduce time and costs while still confirming the quality of the data. The data review process may be streamlined in order to meet schedules or cost constraints. The risk must be weighed by the project team. Planning the data review process ensures that budget dollars are available for critical data review identified by the project team. The QAPP should also contain decision criteria that allow for revision of the initial streamlining plan. Verification is not subject to streamlining. It may be possible to identify critical (most significant) samples and focus data review on those samples.

67. 67 End of Scoping Session Before people go home Assign action items Review consensus decisions Delineate reporting relationships Assign Roles and Responsibilities Identify Communication Pathways Record outcomes in worksheet#9.Record outcomes in worksheet#9.

68. Post-Scoping Session Contractor fills in worksheets using results from the scoping session Action items are resolved Complete rest of worksheets Hold additional scoping sessions (iterative process) 68

69. Other Worksheets 69 This is only one suggested order; actual order will probably be different and have more overlap.This is only one suggested order; actual order will probably be different and have more overlap.

70. MeasurementPerformance Criteria Determine measurement performance criteria for each matrix, analytical group, concentration level, and analyte, if applicable Assign QC samples that measure appropriate Data Quality Indicators (precision, accuracy, and sensitivity; representativeness, comparability, and completeness) UFP-QAPP QA/QC Compendium provides listing of QA/QC activities/samples DoD Quality Systems Manual (QSM) provides required (minimum) measurement performance criteria for SW-846 methods DoD ELAP requirement 70

71. ExampleWorksheet #12 71

72. 72 Best Practices Include appropriate subject matter experts and decision makers during project scoping (and get buy-in!) Clearly justify why the project is being completed Provide sufficient written discussion to facilitate understanding of the project’s objectives and to document decision logic Ensure that information remains consistent throughout the QAPP Include all worksheets; if a worksheet will not be filled out, label as N/A and justify its exclusion Develop the sampling design within the context of the CSM Select analytical measurement performance criteria that meet project quality objectives

73. 73 Review Checklist Created based on EPA R5 requirements document, UFP-QAPP Manual requirements, and common errors/best practices seen in previous DoD UFP-QAPPs. This checklist is presented in the order of the worksheets; EPA R5 checklist has similar requirements, but is not ordered by worksheet. The checklist should not be used as a cookbook for creating a UFP-QAPP.Created based on EPA R5 requirements document, UFP-QAPP Manual requirements, and common errors/best practices seen in previous DoD UFP-QAPPs. This checklist is presented in the order of the worksheets; EPA R5 checklist has similar requirements, but is not ordered by worksheet. The checklist should not be used as a cookbook for creating a UFP-QAPP.

74. Basewide QAPPs Should you write basewide QAPPs in the UFP format? Template vs Problem Statement/PQO for entire base UST, Small-Arms Range Worksheets that can be pre-populated Worksheets that shouldn’t be pre-populated Worksheets that pre-population may or may not be a benefit 74 Include background information that does not change (background metals study), verification and validation procedures, data review processes, etc. Keep references together in Administrative Record. Must pay attention to document control and version maintenance in order to recreate QAPP as time goes on. Basewide QAPP must be included with project QAPP for review.Include background information that does not change (background metals study), verification and validation procedures, data review processes, etc. Keep references together in Administrative Record. Must pay attention to document control and version maintenance in order to recreate QAPP as time goes on. Basewide QAPP must be included with project QAPP for review.

75. Questions afcee.chemistry.support@us.af.mil 75

76. Bonus Slides Other worksheets 76

77. Table of Contents A Table of Contents is a required element of UFP-QAPP Table 2 of the UFP-QAPP manual lists all requirements 77

78. Title and Approval PageWorksheet #1 Provide signature spaces for contractor and government project managers, and all regulatory stakeholders Use a consistent document control format that includes: The title of the document (abbreviations may be used) The original version number or revision number, whichever is applicable, including document status (i.e., draft, interim draft, interim final, final) The date of the original version or current revision, whichever is applicable The page number in relation to the total number of pages Apply the document control procedures beginning on the Title and Approval Page and including the Table of contents and all figures, tables, and diagrams 78 A document control numbering system accounts for all copies of the QAPP provided to project personnel and helps to ensure that the most current version is in use.A document control numbering system accounts for all copies of the QAPP provided to project personnel and helps to ensure that the most current version is in use.

79. QAPP Identifying InformationWorksheet #2 List pertinent information, identify references, identify UFP QAPP requirements that are not applicable to the project Provide crosswalk to referenced documents, and identify the location of required information within that document 79 Worksheet 2 will: Identify guidance used to prepare QAPP Identify regulatory program Identify approval entity Indicate whether the QAPP is a generic or a project-specific QAPP List dates of scoping sessions that were held List dates and titles of QAPP documents written for previous site work, if applicable List organizational partners (stakeholders) and connection with lead organization List data users Provide an explanation for the exclusion of any required elementsWorksheet 2 will: Identify guidance used to prepare QAPP Identify regulatory program Identify approval entity Indicate whether the QAPP is a generic or a project-specific QAPP List dates of scoping sessions that were held List dates and titles of QAPP documents written for previous site work, if applicable List organizational partners (stakeholders) and connection with lead organization List data users Provide an explanation for the exclusion of any required elements

80. Distribution ListWorksheet #3 Include all key project personnel, including project manager, lead field sampler, regulatory stakeholders, laboratory QA manager, and a 3rd party data reviewer 80 The Distribution List documents those entities to whom copies of the approved QAPP and any subsequent revisions will be sent.The Distribution List documents those entities to whom copies of the approved QAPP and any subsequent revisions will be sent.

81. Project Personnel Sign-Off SheetWorksheet #4 List all key project personnel If a specific person is TBD, identify by function 81 The Project Personnel Sign-Off Sheet documents that all key project personnel performing work have read the applicable sections of the QAPP and will perform the tasks as described. Although it is not always possible to identify people by name early in the planning stages, the project team should identify by function (e.g., laboratory QC manager) all personnel who are to read and sign off on the applicable sections of the QAPP.The Project Personnel Sign-Off Sheet documents that all key project personnel performing work have read the applicable sections of the QAPP and will perform the tasks as described. Although it is not always possible to identify people by name early in the planning stages, the project team should identify by function (e.g., laboratory QC manager) all personnel who are to read and sign off on the applicable sections of the QAPP.

82. Organization ChartWorksheet #5 82 Generating an org chart will help identify all parties involved in major aspects of the project, establish the roles and responsibilities for those personnel, and formalize the relationships between the project team. Can add lines of communication to reporting relationships. This is an example, the project org chart may have different reporting relationships. QA should have lines of communication to highest level of the organizations. Generating an org chart will help identify all parties involved in major aspects of the project, establish the roles and responsibilities for those personnel, and formalize the relationships between the project team. Can add lines of communication to reporting relationships. This is an example, the project org chart may have different reporting relationships. QA should have lines of communication to highest level of the organizations.

83. Communication PathwaysWorksheet #6 83 Communication pathways and modes of communication (faxes, newsletters, electronic mail, reports) should be delineated in the project planning stage and documented in the QAPP. The project team should describe the proper procedures for soliciting concurrence and obtaining approval between project personnel, between different contractors, and between samplers and laboratory staff. Procedures (timing, pathways and types of acceptable communications) must be outlined in sufficient detail to ensure that users of the QAPP understand the processes and the roles and responsibilities associated with those processes when communication is necessary. Issues that should be addressed include: • Who may approve specific types of real time modifications, including who will be notified and the timing of such notification • Nature of required communication forms (e.g. electronic, verbal, written) • Processes and authorities for recommending work stoppage and corrective action. The project team also should document the procedures that will be followed when any project activity originally documented in an approved QAPP requires real-time modification to achieve project goals. These project activities include: • Sampling design • Sample collection procedures • Sample analysis procedures • Data review and reportingCommunication pathways and modes of communication (faxes, newsletters, electronic mail, reports) should be delineated in the project planning stage and documented in the QAPP. The project team should describe the proper procedures for soliciting concurrence and obtaining approval between project personnel, between different contractors, and between samplers and laboratory staff. Procedures (timing, pathways and types of acceptable communications) must be outlined in sufficient detail to ensure that users of the QAPP understand the processes and the roles and responsibilities associated with those processes when communication is necessary. Issues that should be addressed include: • Who may approve specific types of real time modifications, including who will be notified and the timing of such notification • Nature of required communication forms (e.g. electronic, verbal, written) • Processes and authorities for recommending work stoppage and corrective action. The project team also should document the procedures that will be followed when any project activity originally documented in an approved QAPP requires real-time modification to achieve project goals. These project activities include: • Sampling design • Sample collection procedures • Sample analysis procedures • Data review and reporting

84. Personnel Responsibilities and QualificationsWorksheet #7 84 Resumes for each person identified should be attached to the QAPP or their location noted. The lead organization must ensure that the responsible project personnel meet any specific QAPP qualifications, such as laboratory certification or professional engineer (PE).Resumes for each person identified should be attached to the QAPP or their location noted. The lead organization must ensure that the responsible project personnel meet any specific QAPP qualifications, such as laboratory certification or professional engineer (PE).

85. Specialized Training RequirementsWorksheet #8 85 All project personnel must be qualified and experienced in the project tasks for which they are responsible. Special training and expertise may be added to the project as the expertise necessary is defined. This may evolve as the project evolves. All project personnel must be qualified and experienced in the project tasks for which they are responsible. Special training and expertise may be added to the project as the expertise necessary is defined. This may evolve as the project evolves.

86. Secondary Data EvaluationWorksheet #13 Provide additional text as necessary to discuss adequacy of the original QA documentation as compared to current project quality objectives Figure 18 of UFP-QAPP manual provides a flowchart for examining secondary data 86 Secondary data may include data generated for or by external, independent parties which are then transmitted to the current user. Secondary data may also include data collected in other investigations designed to answer different questions than those posed in the current investigation. Using data and information that are not generated for the same quality objectives as the current investigation may result in erroneous decisions; therefore, it is essential to identify use limitations for secondary data.Secondary data may include data generated for or by external, independent parties which are then transmitted to the current user. Secondary data may also include data collected in other investigations designed to answer different questions than those posed in the current investigation. Using data and information that are not generated for the same quality objectives as the current investigation may result in erroneous decisions; therefore, it is essential to identify use limitations for secondary data.

87. Summary of Project TasksWorksheet #14 87 The QAPP should provide a general overview of the activities that will be performed, and how and when they will be performed, based on background information and data, preplanning site visits, and scoping sessions. Specific details for the individual project activities will be provided in later sections of the QAPP.The QAPP should provide a general overview of the activities that will be performed, and how and when they will be performed, based on background information and data, preplanning site visits, and scoping sessions. Specific details for the individual project activities will be provided in later sections of the QAPP.

88. Project Schedule/Timeline TableWorksheet #16 Allow sufficient time for document review and implementation of corrective actions 88

89. Sampling Locations and Methods/SOP RequirementsWorksheet #18 List each sample location and assign it a distinct ID number Provide rationale for sampling location (e.g., background, grid, hot spot, downgradient, etc.) Attach referenced SOPs 89

90. Analytical SOP RequirementsWorksheet #19 Document the rationale for selecting the sample volume, container, preservation, and holding times requirements (e.g., EPA method or regulation) Attach referenced SOPs 90

91. Field Quality Control Sample SummaryWorksheet #20 Provide additional text that describes and justifies the chosen QC samples Identify the requirement guidance if applicable (e.g., QA/QC compendium, DoD QSM, method, etc.) 91

92. Project Sampling SOP ReferencesWorksheet #21 Include SOPs that cover: 92 Standardized sampling procedures provide consistency between samplers; facilitate collection of accurate, precise, and representative samples; and help to ensure data comparability and usability.Standardized sampling procedures provide consistency between samplers; facilitate collection of accurate, precise, and representative samples; and help to ensure data comparability and usability.

93. Field Equipment Calibration, Maintenance, Testing , and InspectionWorksheet #22 List all field equipment (e.g., tools, gauges, pumps) If applicable, include acceptance criteria for calibration or inspection, and provide corrective actions for when deficiencies occur 93 The QAPP should describe all procedures and documentation activities that will be performed to ensure that field sampling equipment is available and in working order when needed, and keep equipment clean and free of target analytes or interferences.The QAPP should describe all procedures and documentation activities that will be performed to ensure that field sampling equipment is available and in working order when needed, and keep equipment clean and free of target analytes or interferences.

94. Analytical SOP ReferencesWorksheet #23 Include SOPs for each analytical group, matrix, and concentration level that will be investigated Proprietary SOPs may be submitted as confidential business information 94 All analytical procedures that will be used in the project must be documented in the QAPP or attached document(s) to allow for review and approval.All analytical procedures that will be used in the project must be documented in the QAPP or attached document(s) to allow for review and approval.

95. Analytical Instrument CalibrationWorksheet #24 Ensure that acceptance criteria correlate with other project measurement performance criteria 95 To ensure that the analytical methods and the selected instrumentation meet the project requirements for selective, sensitive, accurate, and precise detection and quantitation of the analytes of interest, it is necessary to describe completely the calibration procedures for each analytical instrument, as well as demonstrate the ability of the analytical technique to accurately and precisely identify and quantitate the target analytes/COCs at the required QLs and within the required measurement ranges.To ensure that the analytical methods and the selected instrumentation meet the project requirements for selective, sensitive, accurate, and precise detection and quantitation of the analytes of interest, it is necessary to describe completely the calibration procedures for each analytical instrument, as well as demonstrate the ability of the analytical technique to accurately and precisely identify and quantitate the target analytes/COCs at the required QLs and within the required measurement ranges.

96. Analytical Instrument and Equipment Maintenance, Testing, and InspectionWorksheet #25 Describe the procedures and documentation activities that will be performed to ensure that all analytical instrumentation and equipment are available and in working order when needed 96 Instrument and equipment maintenance logs must be kept to document analytical instrumentation and equipment maintenance, testing, and inspection activities.Instrument and equipment maintenance logs must be kept to document analytical instrumentation and equipment maintenance, testing, and inspection activities.

97. Sample Handling SystemWorksheet #26 Describe how samples will be delivered or shipped to the laboratory Describe the procedures that will be used to maintain sample custody and integrity and to document the implementation of chain-of-custody procedures Identify how long the sample will be kept after it has been analyzed 97 Proper sample tracking systems support the chain-of-custody procedures, which in turn help to ensure sample authenticity and data defensibility.Proper sample tracking systems support the chain-of-custody procedures, which in turn help to ensure sample authenticity and data defensibility.

98. Sample Custody RequirementsWorksheet #27 Describe the procedures that will be used to maintain sample custody and integrity Identify sample numbering, sample labeling, and sample tracking systems Provide examples of chain-of-custody documentation that will be used during the project provided 98 Examples of chain-of-custody documentation that may be used during the project include chain-of-custody forms, sample identification forms, custody seals, laboratory sample receipt forms, etc.Examples of chain-of-custody documentation that may be used during the project include chain-of-custody forms, sample identification forms, custody seals, laboratory sample receipt forms, etc.

99. QC SamplesWorksheet #28 Provide additional text that describes and justifies the chosen QC samples Identify the requirement guidance if applicable (e.g., QA/QC compendium, DoD QSM, method, etc.) Describe corrective actions and, and identify personnel responsible for performing those corrective actions 99

100. Project Documents and RecordsWorksheet #29 100 Sample collection and field measurement records include, field data collection sheets, chain-of-custody records, airbills, communication logs, corrective action reports, documentation of corrective action results, documentation of deviation from methods, documentation of internal QA review, electronic data deliverables, identification of QC samples, meteorological data from field (e.g., wind, temperature), sampling instrument decontamination records, sampling instrument calibration logs, sampling location and sampling plan, sampling notes and drilling logs, and sampling report. Analytical records include, chain-of-custody records, sample receipt forms and sample tracking forms, preparation and analysis forms and/or logbooks, tabulated data summary forms and raw data for field samples, standards, QC checks, and QC samples, case narrative, sample chronology (time of receipt, extraction, and analysis), identification of QC samples, communication logs, corrective action reports, definitions of laboratory qualifiers, documentation of corrective action results, documentation of laboratory method deviations, electronic data deliverables, instrument calibration reports, laboratory name, laboratory sample identification numbers, reporting forms completed with actual results, signatures for laboratory sign-off (e.g., laboratory QA manager), and standards traceability records. Project data assessment records include, field sampling audit checklists, analytical audit checklists, PT sample results, data review reports, telephone logs, corrective action reports, laboratory assessment, laboratory QA plan, MDL study information, and ELAP accreditation.Sample collection and field measurement records include, field data collection sheets, chain-of-custody records, airbills, communication logs, corrective action reports, documentation of corrective action results, documentation of deviation from methods, documentation of internal QA review, electronic data deliverables, identification of QC samples, meteorological data from field (e.g., wind, temperature), sampling instrument decontamination records, sampling instrument calibration logs, sampling location and sampling plan, sampling notes and drilling logs, and sampling report. Analytical records include, chain-of-custody records, sample receipt forms and sample tracking forms, preparation and analysis forms and/or logbooks, tabulated data summary forms and raw data for field samples, standards, QC checks, and QC samples, case narrative, sample chronology (time of receipt, extraction, and analysis), identification of QC samples, communication logs, corrective action reports, definitions of laboratory qualifiers, documentation of corrective action results, documentation of laboratory method deviations, electronic data deliverables, instrument calibration reports, laboratory name, laboratory sample identification numbers, reporting forms completed with actual results, signatures for laboratory sign-off (e.g., laboratory QA manager), and standards traceability records. Project data assessment records include, field sampling audit checklists, analytical audit checklists, PT sample results, data review reports, telephone logs, corrective action reports, laboratory assessment, laboratory QA plan, MDL study information, and ELAP accreditation.

101. Analytical ServicesWorksheet #30 Identify organizations or laboratories that will provide the analytical services (for all on-site screening, on-site definitive, and off-site laboratory analytical work, including all prime laboratories, subcontractor laboratories, and backup laboratories. 101

102. Assessment and OversightWorksheets #31-#33 Establish assessments that ensure project activities are implemented as planned Evaluate activities with the most influence on data quality Describe rationale for chosen assessments Describe how assessments and identified deficiencies will be documented and communicated (e.g., verbal debriefing after audit and/or written audit report) Describe what type of corrective action responses will be required, and how the responses will be documented Include at least one field sampling assessment at the start of field sampling activities 102 Appropriately scheduled assessments allow management to implement corrective action measures in a timely manner, thereby minimizing the impact of nonconformance on achieving project quality objectives. All data generation and collection operations should include at least one field sampling technical systems audit (TSA) at the start of field sampling activities so that effective corrective action measures can be implemented to mitigate the extent and impact of identified nonconformances.Appropriately scheduled assessments allow management to implement corrective action measures in a timely manner, thereby minimizing the impact of nonconformance on achieving project quality objectives. All data generation and collection operations should include at least one field sampling technical systems audit (TSA) at the start of field sampling activities so that effective corrective action measures can be implemented to mitigate the extent and impact of identified nonconformances.

103. Planned Project AssessmentsWorksheet #31 Possible assessment types include: 103 A readiness review is a systematic, documented review of readiness for the startup or continued use of a facility, process, or activity. A field sampling assessment examines sampling design, equipment, instrumentation, supplies, personnel, training, sampling procedures, chain-of-custody, sample handling and tracking, data reporting, data handling and management, data tracking and control, and data review procedures for conformance with the QAPP. An on-site analytical assessment examines on-site analytical procedures during which the facility (e.g., mobile lab, trailer), equipment, instrumentation, supplies, personnel, training, analytical methods and procedures, laboratory procedures, sample handling and tracking, data reporting, data handling and management, data tracking and control, and data review procedures are checked for conformance with the QAPP. An off-site laboratory assessment examines the laboratory facility, equipment, instrumentation, supplies, personnel, training, analytical methods and procedures, laboratory procedures, sample handling and tracking, data reporting, data deliverables, data handling and management, data tracking and control, and data review procedures are checked for conformance with the QAPP. Split sampling assesses interlaboratory precision and accuracy. The sampler collects one field sample and then physically splits it into two representative sample aliquots. The samples are then sent to different laboratories for analysis. PT sample results provide information on routine laboratory performance and the overall accuracy and bias of the analytical method. A data review assessment is a thorough review of the complete data review process, including a review of the sampling and analysis verification, sampling and analysis validation, and data usability assessment steps, to ensure that the process conforms to the procedures specified in the QAPP. A management system assessment is a review of an organization to determine if the management structure, policies, and procedures are sufficient to ensure that an effective quality system is in place that supports the generation of usable project data.A readiness review is a systematic, documented review of readiness for the startup or continued use of a facility, process, or activity. A field sampling assessment examines sampling design, equipment, instrumentation, supplies, personnel, training, sampling procedures, chain-of-custody, sample handling and tracking, data reporting, data handling and management, data tracking and control, and data review procedures for conformance with the QAPP. An on-site analytical assessment examines on-site analytical procedures during which the facility (e.g., mobile lab, trailer), equipment, instrumentation, supplies, personnel, training, analytical methods and procedures, laboratory procedures, sample handling and tracking, data reporting, data handling and management, data tracking and control, and data review procedures are checked for conformance with the QAPP. An off-site laboratory assessment examines the laboratory facility, equipment, instrumentation, supplies, personnel, training, analytical methods and procedures, laboratory procedures, sample handling and tracking, data reporting, data deliverables, data handling and management, data tracking and control, and data review procedures are checked for conformance with the QAPP. Split sampling assesses interlaboratory precision and accuracy. The sampler collects one field sample and then physically splits it into two representative sample aliquots. The samples are then sent to different laboratories for analysis. PT sample results provide information on routine laboratory performance and the overall accuracy and bias of the analytical method. A data review assessment is a thorough review of the complete data review process, including a review of the sampling and analysis verification, sampling and analysis validation, and data usability assessment steps, to ensure that the process conforms to the procedures specified in the QAPP. A management system assessment is a review of an organization to determine if the management structure, policies, and procedures are sufficient to ensure that an effective quality system is in place that supports the generation of usable project data.

104. Assessment Findings and Corrective Action ResponsesWorksheet #32 Describe the types of corrective action responses that will be required, and how those responses are documented 104 Assessment findings that require corrective action initiate a sequence of events that includes documentation of deficiencies, notification of findings, request for corrective action, implementation of corrective action, and follow-up assessment of the corrective action’s effectiveness.Assessment findings that require corrective action initiate a sequence of events that includes documentation of deficiencies, notification of findings, request for corrective action, implementation of corrective action, and follow-up assessment of the corrective action’s effectiveness.

105. QA Management ReportsWorksheet #33 Use additional text to sufficiently describe what each report contains Ensure that an evaluation of measurement error as determined from the assessments is included in applicable reports 105 Periodic QA management reports ensure that managers and stakeholders are updated on project status and results of all QA assessments, and allow project managers to implement timely and effective corrective actions so data generated can meet PQOs.Periodic QA management reports ensure that managers and stakeholders are updated on project status and results of all QA assessments, and allow project managers to implement timely and effective corrective actions so data generated can meet PQOs.