CDRH Review Practice for Over-the-Counter in Vitro Diagnostic Tests

1. CDRH Review Practice for Over-the-Counter in Vitro Diagnostic Tests Center for Devices and Radiological Health (CDRH) Office of in Vitro Diagnostic Device Evaluation and Safety Arleen Pinkos Scientific Reviewer CDRH OTC Review Practice, BPAC, 051103

2. Overview of Presentation A. Background B. Three Elements of OTC Review • Accuracy and Reliability • Labeling • Risk vs. Benefit C. Conclusion

3. OTC History 817 OTC products cleared for home use • Diabetic Products • Hormones • Drugs of Abuse • Urine Dipsticks • Fecal Occult Blood • Lipids, PT

4. OTC Infectious Disease Devices • None cleared or approved • Devices reviewed for OTC- Self diagnosis: • Group A Streptococcus, and Influenza A and B • In professional hands: • Sensitivity averaged 55-80%, dependent on specimen matrix • Specificity averaged 80-90% • False negatives – risks are life threatening • False positives – risk of masking serious underlying disease

5. Indications for Use: What Drives the Review • Need to know who, what, when, and why • Special conditions for use that impact performance • Contraindications • Drives: • Protocol requirements • Questions raised • Data required

6. Three Review Elements 1. Is the device accurate and reliable in the hands of lay-users? 2. Is the device adequately labeled? 3. Do benefits outweigh the risks?

7. First Review Element Is the device accurate and reliable in the hands of the user? • Can the test provide the right answer? • Can the lay user get the right answer? • Human Factors (HF) • Risk Mitigation & Safeguards • Stress Studies

8. Can the test provide the right answer? Analytical and Clinical Performance • From initial professional-use submission • Analytical and Clinical Sensitivity and Specificity Accurate (true) result • Quantitative -Bias, total error • Qualitative- Clinical Sensitivity and Specificity, Predictive Values

9. Can the test provide theright answer? Other Important Parameters • Cross-reactivity & Interference • Reproducibility • Reference intervals • Environmental factors, stability, reportable range, calibration, linearity, lot to lot variability, etc.

10. Can the lay user get the right answer? Consumer Study Goal Demonstrate comparable performance when used as intended • Representative Lay Users • Enrollment criteria, demographics • “Home” environment • Evaluate entire test system • Consumers perform all steps, unassisted • Labeling – Representative labeling, or according to marketing plan.

11. Consumer StudySample Considerations • Clinical or spiked – • Challenging and span expected range • Prospectively collected if: • establish clinical sensitivity and specificity • demonstrate ability to collect the sample, or • sample is not stable • Biohazard • Properly characterized- WHO, CDC, reference method, or to another device

12. Consumer StudyProtocol Considerations • Comprehensive • Enough participants and samples • Correct Statistical Tools • Masking and randomizing- avoid bias • Written prior to study

13. Human Factors (HF) • A discipline concerned with how people interact with the device Why are HF so important? • Users- varying abilities and behaviors • Impact on safety and effectiveness

14. Human Factors “Errors” occur when users: • Fail to perform necessary actions • Add actions • Choose the wrong action • Perform actions poorly

15. Human Factors Who is responsible for “user” errors? “The instructions clearly state that results are to be read from right to left.” “How was I to know they’d read it left to right?”

16. Toenail Cutter ACME All user errors cannot be prevented but some can be avoided

17. Risk Mitigation & SafeguardsWhat do we expect regarding errors? Effective procedures to address HF and other potential problems: • Hazard Analysis: Identify potential errors and estimate their risk • Mitigate Risks: Eliminate or reduce risks to acceptable level • Verify effectiveness of safeguards

18. Risk Mitigation & Safeguards Examples • Built-in features to detect: • Short sample • Wrong sample • Reagent integrity • Internal QC- often limited scope • Lock-out mechanisms? • Labeling- techniques, but when is it adequate? • Goal- alert before results are compromised

19. Risk Mitigation & SafeguardsQuality Control • Is External QC • Available and practical? • Adequate? • Needed? • Will consumers do it ?!!

20. Stress Studies Demonstrating Effective Mitigation Need to verify effectiveness of safeguards What are stress studies? • Challenges to the system in order to characterize performance under various conditions of use

21. Stress Studies How are they performed? • Artificially simulate potential errors • Concentrate on areas prone to errors or failures • Technique • Follow directions • Deviations in environmental conditions • Observe results

22. Stress Study Examples

23. Second Review ElementIs the device adequately labeled? • Are the step-by-step instructions simple, concise, and easy to follow? • 7th grade reading level? • Do consumers know who should use the device and when? • Clear instructions for interpretation and follow-up? • Limitations? • Go to a doctor? • What if it didn’t work? • Repeat test?

24. LabelingHow do you know its adequate? Verified-Consumer Study • Value is Limited • “Trial effect” • Will they read it? Questionnaire • Completed by consumers after testing • Must be challenging: Who, what, where, when, how & why?

25. Third Review ElementRisk vs. Benefit Consideration Lay users are unique: • Unregulated environment • May lack medical and technical training: • evaluate other medical information • collect and transport samples • read and follow directions • May or may not carry out follow-up actions • Highly variable

26. Risk vs. Benefit Risk 1. What is the impact of a false-positive or false-negative result on the user and public health e.g., no follow-up, adverse medical conditions, unnecessary treatment? 2. What are the medical risks to the user and public health if there is a delay in obtaining a professional examination and treatment, or if the device gives an equivocal result?

27. Risk vs. Benefit Benefit 1.What is the clinical benefit to the patient and public health in terms of screening, diagnosing, or monitoring a particular disease or condition, or risk factor? 2. What are the medical benefits to the patient and public health in having the test available for home-use as opposed to having the test performed only by health care professionals?

28. CONCLUSIONThe Final Decision • Is the device accurate and reliable in the hands of the lay user? • Does the Labeling adequately convey all information necessary to use the device safely and effectively? • Do benefits outweigh risks? If yes, generally found suitable for home-use