1 / 17

Validation of Qualitative Microbiological Test Methods

Validation of Qualitative Microbiological Test Methods. Pieta IJzerman-Boon (MSD) Edwin van den Heuvel (TUe, UMCG/RUG). NCS Conference Brugge, October 2014. 1. Contents. Introduction Statistical Detection Mechanisms Validation Issues Likelihood-Based Inference Conclusions.

vielka-buck
Download Presentation

Validation of Qualitative Microbiological Test Methods

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Validation of Qualitative Microbiological Test Methods Pieta IJzerman-Boon (MSD) Edwin van den Heuvel (TUe, UMCG/RUG) NCS Conference Brugge, October 2014 1

  2. Contents • Introduction • Statistical Detection Mechanisms • Validation Issues • Likelihood-Based Inference • Conclusions

  3. Introduction • Guidelines on validation do not agree 3

  4. Introduction • In this presentation we will show an optimal validation strategy: • Compare methods • Two dilutions • Optimal densities for the two dilutions • Required number of samples • Optimal validation strategy differs substantially from the guidelines

  5. Statistical Detection Mechanisms • Suppose a test sample is tested with a qualitative test • The sample contains X organisms • X=0: sample is sterile • X>0: sample is contaminated • The outcome of the test is Z • Z=1: positive test result • Z=0: negative test result

  6. Statistical Detection Mechanisms • Classification of test result • So we need to look at the conditional probabilities • The function describing this detection probability is referred to as the detection mechanism Number of Organisms X=0 X>0 False Negative Okay Z=0 Test Result False Positive Okay Z=1

  7. Statistical Detection Mechanisms • Zero-deflated binomial mechanism: • h is the false positive rate: p (0)=h • p is the detection proportion: if h=0 then it is the probability to detect just one organism: p (1)=p • If h=0 and p=1 the test method is perfect • h and p are related to specificity and accuracy • The binomial mechanism (h=0) was introduced in Van den Heuvel and IJzerman-Boon (2013)

  8. Statistical Detection Mechanisms h =0, p=0.85 h =0.10, p=0.70

  9. Validation Issues • Estimate detection mechanism via experiments • Exact low spikes of X cannot be generated • Hence the detection probability p (x) cannot be estimated, only the average proportion over samples • Expected proportion of positive test results: • Assume that the number of organisms X ~ Poi(l)

  10. Validation Issues • The detection proportion p cannot be estimated • Without knowledge on the average number of organisms l in the test samples • With serial dilution experiments • The false positive rate h can always be estimated using samples from a blank dilution (l=0) • Compare alternate with compendial method • Using the same  for both methods • Likelihood ratio test (LRT)

  11. Likelihood-Based InferenceExperimental Design Suppose we test samples from the same dilution with two methods Alternate method: i=1 Compendial method: i=2 Dilution has on average  organisms per sample Number of samples tested per method: n Expected proportion of positive results now depends on method i (i=1,2): 11

  12. Likelihood-Based InferenceExperimental Design Asymptotic distribution of LRT for comparing these proportions converges to -distribution with Hence, power can be optimized by maximizing Bacterial density  can be optimized independently from sample size n There is a single optimal density 12

  13. Likelihood-Based InferenceExperimental Design Compendial: 2=0.01p2=0.95 13

  14. Likelihood-Based InferenceSimulations Simulation Results: Single dilution Average density l Detection proportions pAL=0.7 and pCM=1 Power (%) of likelihood ratio test LRT for differences in detection probabilities for various false positive rates 14

  15. Conclusions Optimal strategy when parameters are unknown Compare alternate with compendial method Two dilutions are needed Blank dilution Dilution with on average l~2 organisms Sample size should be at least n=200 False positive rates can be tested with LRT Accuracy pAL/pCM can be tested with appropriate CIs as an alternative for the LRT for the ratio pAL/pCM (IJzerman-Boon and Van den Heuvel, 2014) 15

  16. Conclusions Differences with guidelines Only specificity and accuracy need to be considered Two dilutions are needed, using five 10-fold dilutions is a loss of power The optimal density is ~2 CFU/unit, ~5 CFU/unit is much too high Use 200 instead of 5 samples per method and dilution to detect a 30% drop in accuracy with 80% power 16

  17. References • IJzerman-Boon PC, Van den Heuvel ER, Validation of Qualitative Microbiological Test Methods, Submitted, 2014. • Van den Heuvel ER, IJzerman-Boon PC, A Comparison of Test Statistics for the Recovery of Rapid Growth-Based Enumeration Tests, Pharmaceutical Statistics, 2013; 12(5): 291-299. • EP 5.1.6 Alternative Methods for Control of Microbiological Quality • USP <1223> Validation of Alternative Microbiological Methods • ICH Q2 (R1) Validation of Analytical Procedures

More Related