Evidence-Based Medicine Week 5 - Diagnosis/Screening

1. Evidence-Based MedicineWeek 5 - Diagnosis/Screening Department of Medicine - Residency Training Program Tuesdays, 9:30 a.m. - 12:00 p.m., UW Health Sciences Library

2. Steps in Practicing EBM • Convert the need for information into an answerable question. • Track down the best evidence with which to answer that question. • Critically appraise the evidence for its validity, impact, and applicability. • Integrate the evidence with our clinical expertise and our patient’s characteristics and values.

3. Clinical Importance/ Test Accuracy Validity Applicability Strategies for Critical Appraisal of Studies on Therapy

4. Validity Strategies for Critical Appraisal of Studies on Therapy

5. Step 1: Are The Results Valid? 1. Investigators used an appropriate/acceptable reference standard for comparison. 2. Comparison to reference standard was independent and blinded. 3. Investigators assembled an appropriate spectrum of patients to whom the diagnostic test will be applied in clinical practice. 4. Results of test being evaluated do not influence decision to perform reference standard or this “verification bias” is adequately addressed by clinical follow-up. 5. Lastly, has the test been validated in a second, independent group of patients.

6. More on Reference Standard • Also called “gold standard”. • Absolute reference standard not typically available. Even then, absolute standards aren’t always absolute (lack of agreement re: biopsy specimens). • Recognize the clinical utility of “long-term follow-up” as a reference standard. • If you can’t accept the reference standard (within reason), then the article is unlikely to provide valid results for your purposes.

7. More on Spectrum of Patients • Biased patient selection can have significant impact on measures of test performance. • If study is done retrospectively then there is a good chance that bias was present when investigators chose who had testing done. • Spectrum of patients should closely reflect population from which your patient emerges. If not estimates of test performance are unlikely to be accurate.

8. Clinical Importance/ Test Accuracy Strategies for Critical Appraisal of Studies on Therapy

9. Step 2: Clinically Important/Test Accuracy What measure of a test’s properties is most clinically useful? • Sensitivity • Specificity • Positive Predictive Value • Negative Predictive Value • Likelihood Ratio Proportion of people with the target disorder in whom the test result is positive. Proportion of people without the target disorder in whom the test result is negative. Proportion of people with positive test results who have the disorder. Proportion of people with negative test results who are free of the disorder.

10. Formulas from Standard 2 x 2 Sensitivity = a/(a + c) Specificity = d/(b + d) PPV = a/(a + b) NPV = d/(c + d) false-positive error rate (Type I) = b/(b + d) false-negative error rate (Type II) = c/(a + c) positive LR = sensitivity/false positive error rate negative LR = false negative error rate/specificity

11. Likelihood Ratios (LR’s) • Unlike predictive values, LR’s are not influenced by prevalence of disease. • LR’s can be used to categorize the values of any test whose results occur along a continuum avoiding the need to dichotomize a test and recalculate sensitivity and specificity for every cutpoint. • LR is the odds of disease given a specified test value divided by the odds of disease in the entire population. • LR’s reflect what actually takes place in clinical reasoning.

12. High Post-Test Probability Low Post-Test Probability Clinical Probability Not Proportions Clinical features of presentation including characteristics of patient, history, and exam. Knowing likelihood ratio allows you to calculate post-test probability Test (can include distinct features of presentation in history or examination). Pre-Test Probability

13. Likelihood Ratio Nomogram For more information and more examples of likelihood ratios please see Steve McGee’s article entitle “Simplifying Likelihood Ratios” posted on the course web site in the “diagnosis” folder.

14. What do LRs mean really?

15. Likelihood Ratio* * - Odds of disease given specified value divided by odds in population. Calculating Likelihood Ratios (102/14)/(251/630) = 18.3 (105/217)/(251/630) = 1.2 (39/273)/(251/630) = 0.36 (5/126)/(251/630) = 0.10 In standard 2 x 2 table -- shows how information is lost positive LR = sensitivity/false positive error rate = (102/251)/(14/630) = 18.3 negative LR = false negative error rate/specificity = (149/251)/616/630) = 0.61

16. Applicability Strategies for Critical Appraisal of Studies on Therapy

17. Applicable to Our Patient? 1. The test is worth doing (that is the patient’s pre-test probability can be estimated to be intermediate). 2. Patient who will undergo test falls within the spectrum of people studied (differing test properties in different subpopulations). 3. The test can be reproduced and interpreted in the clinical setting. 4. The results of the test will change management of patient’s problem. 5. Testing effects clinical outcome (right heart catheterization in critically ill).

18. Special Considerations for Studies of Screening Tests • Evidence required goes beyond the accuracy of the test for early diagnosis and encompasses hard evidence that patients are better off in the long run screened rather than unscreened. • Ultimately requires RCT’s of the screening test with both disease-specific and all-cause mortality as end points often measured in quality adjusted life years.

19. Rules for Assessing the Value of Screening • Evidence confirms that early diagnosis leads to improved survival, quality of life, or both. • Diagnosed patients are willing to undergo treatment that alters the natural history of disease. • Time and energy needed to confirm the diagnosis and provide life-long treatment are well spent. • The frequency and severity of the target disorder warrant the effort and expenditure.

20. Diagnosis Questions