Challenges In Validating Analytical Methods in an Independent Lab

1. Challenges In Validating Analytical Methods in an Independent Lab

2. Overview • The purpose of this presentation is to review the process and highlight the complexity of method validation in a third party lab.

3. Lab Function

4. What is the laboratory’s function? • Receive samples • Perform tests • Deliver results

5. What does a laboratory need to do? • Consistently deliver data that is • Accurate • Reliable • Valuable

6. Lab Assumptions • Methods validated • For specific matrix • Quality system in place for analysis • Analysts • Facility • Analysis tool • Sampling plans are appropriate

7. The Four BIG Questions: • What? • Why? • How? • Where?

8. What is our product and purpose? • A laboratory produces data • Data is used to make decisions

9. Microbiological Data may be Used to Assess: • The safety of food • Verification/validation procedures in HACCP • Adherence to GMP/GHP • The utility (suitability) of a food or ingredient for a particular purpose • The keeping quality (shelf-life) of certain perishable foods • Acceptability of a food or ingredient from a source for which there is not confidence in the process

10. Validation Challenges

11. Changing Technology • Inherent difficulty for labs to take advantage of technical improvements • Labs should have the ability to bring technical innovation to the user • Keeping up with technology leads to increased costs

12. Regulatory Issues • In the event of a regulatory action, validated testing is a virtual necessity • In the event of a legal dispute, validated testing is also a virtual necessity • Juries trust validated tests more than non validated

13. Method Comparison / Validation Issues • Most properties can be measured • When 2 or more alternative methods exist for measuring the same property, how do you compare ? • Do they really measure the same thing ?

14. Method Comparison / Validation Issues • Philosophically , different methods can’t measure exactly the same thing • No measuring technique responds to only one single property • Relationship of methods could very well depend on the material being measured

15. What is considered • Functional relationship between alternative methods • Technical and Economic Merit • Can be >$ 200,000 US • Is the change worth the cost.

16. Biggest Issues • Huge diversity of sample types • Most commercial methods are validated against a limited number of analytes • Sample prep isn’t considered • Sample compositing isn’t considered in most validations

17. Validation Process

18. Validation • Valid “ well grounded or justifiable ; logically correct “

19. Validation • Validate “ to support or corroborate on a sound or authoritative basis”

20. Validation “ an act , process to determine the degree of validity of a measuring device”

21. Validation Process • Overview • Measurement • Evaluation • Verification • Summary

22. “An element of chance enters into every measurement ; hence every set of measurements is inherently a sample of certain more or less unknown conditions. Even in those few instances where we believe that the objective reality being measured is a constant, the measurements of this constant are influenced by chance or unknown causes.” W.A. Shewart

23. No two things are alike, but even if they were , we would still get different values when we measured them. W.A. Shewart

24. Overview • Microbiological analysis will continue to be a cornerstone used to determine the safety and quality of foods in domestic and international trade • Microbiological data are important to determine compliance with Food Safety Objectives, microbiological criteria, and for HACCP validation and verification

25. Overview (cont.) • Microbiological data used to determine acceptability of products in domestic and international trade must be reliable and consistent among trading partners • Both regulators and industry need to maximize the capacity and credibility of laboratory testing for both official and routine purposes

26. Process Choices • Manual • Human – majority of tests • Semi Automated • Human / Machine - growing percentage • Automated • On its way

27. Measurement

28. What is Measurement • 4 scales • Nominal , Ordinal, Interval, Ratio • Relationship to some property • Direct or indirect • Production process • Sampling through to Decision making • Performance characteristics • Rugged, Practical, Specific, Reliable

29. Measurement Considerations • Measurement unit reflects variation • Consistent over time • Unbiased • Characterize product relative to spec limits • Reflect product that has not been measured

30. Measurement Considerations • Usefulness in process control • Detects differences • Technique comparison • Product information from measurement

31. Microbiological Testing Applications • Water testing • Environmental pathogen programs • Incoming ingredient testing • Finished product analysis • Pathogenic organisms • Spoilage organisms • Finished product challenge studies • Process validation studies

32. Evaluation Process • Design the study • Appropriate to deliver needed info • Choose matrix • Sample to test • Choose methods • New vs. old • Choose measurement instruments • Humans included • Choose reference material • If possible • Perform statistical analysis

33. Statistical performance • Standard deviation • Repeatability • Reproducibility • Operator bias • Operator error • Test bias • Test error

34. Method verification and proficiency testing are essential components of a laboratory’s quality systemand are necessary to determine Uncertainty of a microbiological data result

35. Method Validation - Reliability 1. Reproducibility – between labprecision. 2. Repeatability – within-lab precision. 3. Systematic error or bias – deviation from the ‘true’ value. 4. Specificity – ability to measure what is intended to be measured. 5. Limit of reliable measurement – smallest increment that can be measured with confidence. 6. Uncertainty in result (AOAC)

36. Validation Types

37. Validations Overview • Single Lab • Intralaboratory • Interlaboratory

38. Single Lab • One lab - one matrix – one analyte • Matrix - analyte specific method • Extreme validity • Difficult reproducibility • “In –House “ methods

39. Intralaboratory • Within a lab • Somewhat like single validation • Typical statistical measures are used to verify performance • Normally cover multiple matrices for one analyte

40. Interlaboratory • Throughout larger lab system • Multiple sites • Complex measure of ruggedness • Both within and between lab variation measured • Very Expensive

41. Process

42. Testing • Key step in process • Success is dependent on previous steps • Requires in depth planning • Requires stringent quality systems

43. Testing Considerations • Product knowledge • Previous use • Cost and value • Analysis performance

44. Testing • Method Selection • Analyte • Matrix • Time • Money • Method Execution • Receipt • Prep • Test • Result

45. Types of tests • Quantitative • Microbiological • Chemical • Qualitative • Microbiological • Chemical

46. Validation Process - Quantitative • Determines equivalence of methods for an analyte based on a numerical scale • Determines proper testing conditions to achieve accurate results • Determines appropriate field of use

47. Validation Process - Qualitative • Determines equivalence of methods for an analyte based on a yes or no scale • Determines proper testing conditions to achieve accurate results • Determines appropriate field of use

48. Matrix Considerations • What is the matrix being tested? • What information do we need ? • Qualitative • Quantitative • Are there matrix effects on the test?

49. Analyte • Microbiological • Chemical • Physical

50. Validation Examples