1 / 57

Data Integrity

Data Integrity. Charlie Appleby, U.S. EPA Region 4 SESD Quality Assurance Section. “There can be no friendship without confidence [trust], and no confidence [trust] without integrity.” Samuel Johnson “Transparency is the key to trust.” Steven Hill

cain-saunders
Download Presentation

Data Integrity

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Data Integrity Charlie Appleby, U.S. EPA Region 4 SESD Quality Assurance Section

  2. “There can be no friendship without confidence [trust], and no confidence [trust] without integrity.” Samuel Johnson “Transparency is the key to trust.” Steven Hill “Real integrity is doing the right thing, knowing that nobody’s going to know whether you did it or not.” Oprah Winfrey Integrity

  3. Data with Integrity • Data of known and documented quality • Representative, Comparable and Complete • Defensible and Usable for its intended purpose, the first time. • Best Practices for the Detection and Deterrence of Laboratory Fraud, 1997

  4. Data Integrity Requirements • Careful planning prior to sample collection • Coordination between stakeholders • Communication lines

  5. Planning For Success • DQO Process • Decisions • Data Needs • Data Quality Requirements

  6. Planning For Success • Take time to plan • Define the the project boundaries • Include all stakeholders in the entire process • Establish lines of communication providing essential information to all personnel involved in the project

  7. Project Leader Project QA Contractor Support Management RSCC Laboratory Peer Reviewer QA Manager Data QualityNo Task for a One-Man Band

  8. Coordination For Success • Between • Branches • Divisions • Agencies

  9. Planning for Quality • Quality Management Plan • Quality Assurance Project Plan

  10. It Comes Down to People and Communication

  11. Corporate Policies Reflect Values • Hiring practices - checking references • Ethics training • Data integrity training • Complete technical training • A quality system

  12. In the future, employees will either be superstars or perspiration wipers. Those who aren’t qualified to do either will become managers. – Scott Adams

  13. Ethics Policy • Conduct all business with integrity and in an ethical manner • Responsibility of each staff member and manager to hold to the highest ethical standard of professional conduct in the performance of all duties

  14. Data Integrity Policy • To ensure work is of highest integrity • Employees responsible and accountable for the integrity and validity of their own work • Employees to respect and adhere to the principles of ethical conduct • Fabrication or falsification of work results are direct assaults on the integrity of the laboratory and will not be tolerated

  15. Documenting the Quality System,The QA Manual • The Corporate Mission, Values & Vision • Organizational structure and responsibilities • Procedures for documenting lab ops • Sample receipt • Stds/reagent prep • Completing Training • Document control • Corrective action

  16. The QA Manual (continued) • Data verification, approval, and reporting • Facility/data security • Emergency procedures • Corrective action policies and procedures • Index of SOPs • Reports to management

  17. Cracks in the Quality System • Entropy – Newton?

  18. Cracks in the Quality System • Lax documentation in sample receiving, • Poor hiring decisions, • Failure to complete or document training, • Lack of cross-training, • Missed SOP updates, internal audits,

  19. Cracks in the Quality System • Lax peer review, • Poor document control, • Poor housekeeping, • Turnover, • Drop in data quality

  20. Whither the Quality SystemVulnerabilities • Inappropriate practices • Lost business/revenue • Excessive turnover • Fraudulent activities

  21. What is Laboratory Fraud? • Intentional misrepresentation of lab data to hide known or potential problems • Make data look better than it really is Dr. Bruce Woods

  22. Potential Areas of Lab Fraud • Procedural Deceptions: • Not following critical steps of methodology • Short-cutting sample prep, calibration, analysis • Measurement Deceptions: • Directly altering results • Time and date, conditions of experiment

  23. Examples of Procedural Fraud • Leaving out hydrolysis step in herbicide analysis to avoid hassle. • Not prepping the PE sample before analysis (direct injection). • Not digesting metal samples for analysis due to heavy workload

  24. More Examples • Selectively background subtracting spectra from other peaks to make tuning criteria pass in GC/MS analysis. • Using calibration procedures that are not allowed by the required method.

  25. Examples of Measurement Fraud • Deceptive GC Peak Integration • Time Travel • Dry Labbing

  26. Preventing QA System PitfallsWhat can the lab do? • Independent QA Officer, • Ethics Policy, • Internal audits, • Certifications, • Managers who keep the vision fresh

  27. Detecting QA System ProblemsWhat can EPA do? • Independent data validation, • Monitoring PT sample performance, • Data tape audits, • On-site laboratory audits • Announced • Unannounced • Follow-up audits

  28. Preventing QA System Weaknesses • Contract language, • Clear QA/QC requirements, • Incentives / Disincentives • Pre-award audits, • Past performance assessment, • Performance Testing

  29. Case Studies Let’s Test your Knowledge

  30. Challenges for EPA Why do we need a vision for data integrity? “Though leaders in the middle may not always be the inventors of the vision, they are almost always its interpreters.” • John C. Maxwell We are what we repeatedly do. Excellence then is not an act, it is a habit. • Aristotle

  31. Vision Statement Creation • First, identify the mission • Protect Human Health and the Environment • Identify values • science-based policies and programs • adherence to the rule of law • overwhelming transparency • Distill and refine

  32. Elements of vision • Clarity • Connection of Past, Present, and Future • Purpose • Goals • Challenge • Stories • Passion

  33. Vision Exercise • Imagine the ideal state • Identify needed skills/competencies • Evaluate strategy • Clarify the forces you will face • Be realistic

  34. Data Integrity Starter Quiz Are you DI Savvy? Questions borrowed from SHOQ Quality Assurance Manuals Inc. ISO 17025 Culture Quiz

  35. Management and technical personnel should have the necessary: A. Personnel B. Authority and resources C. Instrumentation

  36. The laboratory’s quality system policies and objectives should be defined in a: A. Quality Policy Statement B. Quality Manual C. Standard Operating Procedure

  37. Document control means: A. Ways to reduce paper B.Documents are identified, authorized, reviewed C. Give all documentation to supervisor

  38. A laboratory is not responsible to the client for the work of a subcontractor A. True B. False

  39. Records should be maintained of all client complaints and of: A. How angry the client was B. The investigations and corrective actions taken by the lab C. How loud the client complained D. To CYA in court

  40. The procedure for corrective action must start with: A. Finger Pointing B. Risk Assessment C. An investigation to determine the root cause(s) of the problem D. A judicious application of CYA

  41. Controlled records should be: A. Legible, readily retrievable, and in a suitable environment B. Designed for auditors C. Controlled by IT personnel

  42. Internal audits are conducted to verify: A. Compliance of operations with quality system B. We won’t get caught again! C. The cost vs benefit of each test offered D. Compliance of operations with EPA requirements

  43. Management reviews determine: A. Continuing suitability and effectiveness of the quality system B. That there will always be another Dilbert cartoon C. Employee requirements are met through 365 degree feedback

  44. Training records are essential to: A. Writing Job Descriptions B. Accrediting the analyst C. Ensure competence and authorize personnel

  45. Methods must be sufficiently validated as well as: A. Maximize profits B. Meet the needs of the client and appropriate for the test C. Easily implemented by the lab

  46. Primary measurement standards must be traceable by means of an unbroken chain of calibrations or comparisons linking them to: A.The last standard entered in the log B. NTIS C. Check samples

  47. Sampling generally happens prior to reception at a lab, and therefore has little affect on final results: A.True B. False

  48. Three levels of data/peer review are necessary to: A.Keep the analysts feeling insecure B. Give the manager something to do C. Ensure the data are accurate, defensible, and meet the clients’ needs

  49. Case Studies Let’s Test your Knowledge

  50. Case Study 1 • An analyst is preparing a method blank associated with a batch of samples which will be digested for metals determinations. The analyst selects a specific beaker which he/she always uses to digest the blank because it seems to produce non-detect or very low results . This practice is: • Perfectly acceptable • A deceptive lab practice • An improper lab practice • None of the above • Both B & C

More Related