EVALUATING

1. EVALUATING • After retrieving the literature, you have to evaluate or critically appraise the evidence for its validity and applicability to your patient and adherence to truth.  • This often requires knowledge of basic statistics and a familiarity with the terminology of EBM … Example: positive predictive value likelihood ratio number needed to treat (NNT)

2. Physicians do critical appraisals for cases that they see quite often in their practice • Where as for cases that are less frequently seen they conserve their time by seeking out critical appraisals already done by others such as Cochrane, Best Evidence, etc.  • For those cases that are very infrequently seen in their clinics, they seek advice from a colleague who is an authority in this specialty.

3. Levels of Evidence Depends on the Type of Research Methods Used • Strong evidence from at least 1 systematic review of multiple well-designed RCT. • Strong evidence from at least 1 properly-designed RCT of appropriate size. • Evidence from well-designed trials without randomization, or case-control studies. • Evidence from well-designed non-experimental studies from more than 1 center or research group. • Opinions of respected authorities, based on clinical evidence, descriptive studies or reports of expert committees.

4. RCT Cohort Studies Case Control Studies Case Series/Case Reports Expert Opinion Animal Research/ In Vitro Studies Evidence Pyramid • The base has the largest number of literature studies and provides the least strength of evidence.   • If you do not find an upper level of evidence, go to the next one.  • Remember that there may be no good evidence to support clinical judgment.

5. The “best” evidence for therapy question • This is found in double-blind randomized controlled trials

6. The “best” evidence for diagnosis question conditionpresent conditionpresent conditionpresent patients suspectedof disease patients suspectedof disease patients suspectedof disease  diagnostic test &gold standard  diagnostic test &gold standard  diagnostic test &gold standard conditionabsent conditionabsent conditionabsent • This is found in controlled trials of prospective studies that compare tests with a “gold standard” test condition present patients suspected of disease Diagnostic Test & Gold Standard condition absent

7. The “best” evidence for etiology question • This is found in cohort studies.

8. The “best” evidence for prognosis question • This is found in cohort or case control studies Suffer target outcome Prognostic Factors time patients Does not Suffer target outcome

9. CRITICALLY APPRAISE the collected literature according to category

10. Therapy   When evaluating the literature to answer a therapy question ask yourself: • Was the study randomized and double blind to prevent bias? • Was follow-up > 80%? • Were the groups similar at the start of the trial? • Were all enrolled patients included in the conclusion of the study? • Was the study valid?  did the authors answer the question?

11. Therapy   When evaluating the literature to answer a therapy question ask yourself: • Do the results present an unbiased estimate of the treatment effect? • How large is the treatment effect? • Will the results help my patient? • Were the study patients similar to your patient? • Are the benefits worth the harm and cost?

12. The features common to evaluation of therapeutic interventions: • Random allocation • Single, double or triple blind methods • Placebo • RCT

13. Concepts used in therapy studies: • Number Needed to Treat (NNT) • Relative Risk Reduction (RRR) • Intention To Treat Analysis

14. Diagnosis  • Diagnostic tests are evaluated in a manner to ascertain which are more accurate, faster, less expensive, less invasive than existing diagnostic tests.  • Good diagnostic tests must provide positive results when the disease is present, and negative results when the patient does not have the disease.  • In contrast to therapeutic evaluations, all persons involved in a new diagnostic test must receive the test.  • The results are compared with the results of the standard  "gold standard" test.

15. To evaluate a diagnosis question ask yourself: • Did the authors do a blind comparison with a gold standard? • Did patients in the study undergo both the diagnostic test and the gold standard? • Did the paper describe the method for doing the test? • Were the patients tested similar to your patient? • Are the results of the test useful? • Did the patient sample include an appropriate spectrum of patients similar to those found in general practice?

16. The features used in diagnostic studies: • Sensitivity vs. Specificity • Positive and Negative Predictive Value • False positive and false negative reaction • Likelihood Ratio of a positive and negative test • Receiver operator characteristic curve  (ROC curve)

17. Etiology • Two methods predominate to assess whether something causes disease: • Cohort study - the one with the strongest evidence (persons with exposures are followed in time to assess outcomes and the results are compared with a similar group that does not have the exposure) • Case-control study - the second method for testing etiology which is more common but has less evidence because of a larger potential for bias

18. To evaluate a etiology question ask yourself • Were the exposures and outcome measured similarly in both groups (exposed and non exposed patients)? • Was the comparison group similar to the outcome group in all aspects except for the variable in question? • Was follow up sufficiently long and complete?

19. To evaluate a etiology question • Terms used in search strategy include     - Cohort studies        - Case control studies     - Follow up studies     - Risk

20. Prognosis  • Uses cohort studies to see how the disease is progressing

21. To evaluate a prognosis question ask yourself: • Was the patient sample selected to reflect a well-defined point in the course of disease? • Was the follow-up adequate and complete (>80%)? • Was there objective and unbiased outcome criteria used?

22. To evaluate a prognosis question Terms used in search strategies: • Cohort studies • Incidence (number of new cases in a given period of time) • Prevalence (number of current cases at a specific time) • Follow-up studies • Disease progression 

23. Applying the results of a study to individual patients Once you determine that the study methodology is valid, you must then examine the results and their applicability to your patient.

24. Therapy • Is my patient so different from those in the study group that the results cannot be applied? • According to the study results how much could my patient benefit from the treatment? • Are the treatment and its consequences consistent with my patient's values and beliefs?

25. Diagnosis  • Is the test affordable, accurate and available locally? • Can it estimate the pretest probability of the disease in question? • Will the posttest probability affect my management?

26. Etiology  • Can the study results be extrapolated to my patient? • What is my patient's risk for adverse effects? • Can my patient's preferences & expectations be met by an alternative therapy?

27. Prognosis    • Is my patient similar to the patients in the study group? • Will the evidence alter my choice of treatment?

28. Final words to remember: • EBM builds on and reinforces but never replaces your clinical judgment or experience.