Ensuring Accurate Forensic DNA Results

1. Chapter 13 Quality Assurance Fundamentals of Forensic DNA Typing Slides prepared by John M. Butler June 2009

2. Chapter 13 – Quality Assurance Chapter Summary DNA testing is a multi-step, technical process that needs to be performed by qualified and effectively trained personnel to ensure that accurate results are obtained and interpreted correctly. Community-wide quality assurance standards, laboratory accreditation and audits, regular analyst proficiency tests and on-going continuing education, method and instrument validation, standard operating procedures, adequate controls and reference materials along with secondary reviews by other qualified analysts all act as checks and controls on DNA results produced in forensic laboratories. These quality assurance measures help maintain good laboratory practice and produce accurate scientific results. Casework management systems (laboratory information management systems, LIMS) along with robotics can further aid production of quality results in a timely fashion.

3. Brief Historical Overview Profiles in DNA (Sept 1999) 3(2): 10-11 Quality problems in late 1980s with DNA testing TWGDAM established under FBI Lab sponsorship in 1988 NRC I (1992) and NRC II (1996) issued reports recommending formal QA programs DNA Identification Act of 1994 lead to formation of DNA Advisory Board (DAB) DAB Standards issued in Oct 1998 and Apr 1999 When DAB was dissolved in 2000, SWGDAM assumed leadership role NIST had membership on the DNA Advisory Board and actively participates in SWGDAM

4. ASCLD-LAB Accreditation Proficiency Testing of Analysts Inspections/ Audits DAB Standards-SWGDAM Guidelines Validated Methods (using standards and controls) Ensuring Accurate Forensic DNA Results

5. NDIS Procedures FBI Quality Assurance Standards Audits SWGDAM Guidelines Laboratory-specific Standard Operating Procedures and Interpretation Guidelines Accreditation Validation (Reference Materials) Instrument Instrument Instrument Training, Proficiency Tests, and Continuing Education Individual Analyst Practice Individual Analyst Practice Each Case Report Each Case Report Each Case Report Each Case Report Case Review Defense Review If court presentation…

6. Suspect developed Crime committed Biological material transferred May match another (K’) Reference (Known) sample “K” Evidence (Question) sample “Q” Database Search Steps Involved Steps Involved May be Inconclusive due to Lack of Available Reference Collection Collection Q U A L I T Y A S S U R A N C E Q U A L I T Y A S S U R A N C E Exclusion (no match) Sample Storage Sample Storage Serology Characterization Q ≠ K Extraction Extraction Q K DNA Profile Comparison Quantitation Quantitation Biology Amplification Amplification Q = K May be Inconclusive due to Forensic Issues (degradation, mixtures, low levels) STR Markers STR Markers Inclusion (match) Separation/ Detection Separation/ Detection Technology Data Interpretation Report (with statistical weight) Data Interpretation Statistical Interpretation Genetics Plea Court Profile put on database Profile put on database

7. Scientific Working Group on DNA Analysis Methods (SWGDAM) • Organized originally by FBI Laboratory as Technical Working Group on DNA Analysis Methods (TWGDAM) in 1988 • Meets semiannually – each January and July • Organized into eight subcommittees: • Quality Assurance, CODIS, mtDNA, Mass Disasters/Missing Persons, Expert Systems, Serology, Y-STRs, and Mixture Interpretation • Membership (usually ~50 attend) from public forensic DNA laboratories around the U.S.

8. Organizations Aiding Forensic DNA Standardization The NIST Human Identity Project Team participates in EDNAP, ENFSI, and ISFG. • European DNA Profiling Group (EDNAP) • Working group of International Society of Forensic Genetics (ISFG) • Examine technologies and run interlab studies • 28 participants from 19 different countries • European Network of Forensic Science Institutes (ENFSI) • Defines policy within European Union • ENFSI DNA Working Group equivalent of SWGDAM • 85 participants from 32 different countries http://www.isfg.org/ ednap/ednap.htm http://www.isfg.org/ http://www.enfsi.eu/ Have challenges with language differences due to many countries involved

9. DNA Identification Act (1994) Public Law 103-322 42 § 14131. Quality assurance and proficiency testing standards • Publication of quality assurance and proficiency testing standards (1) (A) Not later than 180 days after September 13, 1994, the Director of the Federal Bureau of Investigation shall appoint an advisory board on DNA quality assurance methods from among nominations proposed by the head of the National Academy of Sciences and professional societies of crime laboratory officials. (B) The advisory board shall include as members scientists from State, local, and private forensic laboratories, molecular geneticists and population geneticists not affiliated with a forensic laboratory, and a representative from the National Institute of Standards and Technology. (C) The advisory board shall develop, and if appropriate, periodically revise, recommended standards for quality assurance, including standards for testing the proficiency of forensic laboratories, and forensic analysts, in conducting analyses of DNA. DNA Advisory Board (DAB)

10. DAB Standards issued in 1998-1999Quality Assurance Standards (QAS) 1. SCOPE 2. DEFINITIONS 3. QUALITY ASSURANCE PROGRAM   4. ORGANIZATION AND MANAGEMENT   5. PERSONNEL 6. FACILITIES   7. EVIDENCE (SAMPLE) CONTROL 8. VALIDATION 9. ANALYTICAL PROCEDURES   10. EQUIPMENT CALIBRATION AND MAINTENANCE   11. REPORTS   12. REVIEW 13. PROFICIENCY TESTING   14. CORRECTIVE ACTION   15. AUDITS 16. SAFETY 17. SUBCONTRACTOR OF ANALYTICAL TESTING FOR WHICH VALIDATED PROCEDURES EXIST Revised Quality Assurance Standards go into effect July 1, 2009 http://www.fbi.gov/hq/lab/fsc/backissu/july2000/codis2a.htm http://www.fbi.gov/hq/lab/fsc/backissu/july2000/codis1a.htm

11. Checks and Controls on DNA Results

12. Three Possible Outcomes Butler, J.M. (2005) Forensic DNA Typing, 2nd Edition, p. 385 • Match – Peaks between the compared STR profiles have the same genotypes and no unexplainable differences exist between the samples. Statistical evaluation of the significance of the match is usually reported with the match report (see Chapter 21). • Exclusion (Non-match) – The genotype comparison shows profile differences that can only be explained by the two samples originating from different sources. • Inconclusive – The data does not support a conclusion as to whether the profiles match. This finding might be reported if two analysts remain in disagreement after review and discussion of the data and it is felt that insufficient information exists to support any conclusion.

13. What It Means to “Validate” An Analysis Process • How reproducible is an analysis? • If the same evidence is examined multiple times, would the same conclusion be reached each time? • How robust is an analysis? • Are results obtained every time (or a high percentage of the time) evidence is tested? • How reliable is an analysis? • If known samples are examined, are the expected results obtained? • Are situations with insufficient evidence reaching conclusions of “no results”?

14. http://www.fbi.gov/hq/lab/fsc/current/backissu.htm Forensic Science Publications

15. Chapter 13 – Points for Discussion • How is validation helpful to the overall quality assurance program in a laboratory? • What role do organizations like SWGDAM and ENFSI play in quality assurance? • Why are reference materials valuable in aiding quality assurance?