Module 3 Hierarchy of Evidence: Observational Studies

1. Module 3Hierarchy of Evidence: Observational Studies

2. How Clinicians View the World • The observational approach: Describe the world as we find it • The interventional approach Perturb the system, and then describe the result of what we’ve done

3. If You Like to Watch • Are you looking at stuff that’s already there? --a descriptive observational study • Are you developing a special plan (a study design) to observe and gather data with a specific question in mind? --an analytic observational study

4. Types of Observational Studies • Case report - clinical observation • Case series – clinical observation • Ecologic associations – existing data • Cross sectional (prevalence) – existing/new data • Case control – new data • Cohort • retrospective – existing/newdata • prospective – new data

5. The Hierarchy of Evidence for Observational Studies Descriptive observational studies generate an hypothesis: At the individual level Case reports (clinical observation) Case series (clinical observation) At the population level Ecological associations (existing data) Cross sectional studies (existing or new data)

6. The Hierarchy of Evidence for Observational Studies Analytic observational studies support a hypothesis: • Case control studies • Cohort studies • Retrospective • Prospective

7. Why Do We Need Alternatives to Randomized Trials? • Randomized trials may not be feasible • Unethical • Randomization to cigarette smoking • Impractical • Very expensive • Study results are delayed • We need alternatives

8. The Basis of the Hierarchy:Evidence of Causal Relationship Major: Strength of Association Consistency Biological Plausibility Temporality Other: Biologic Gradient (dose-response) Experimental Evidence Analogy Alternative Explanations Explored Cessation Effects Hill AB (1965)

9. A Chance Patient Observation You are treating a patient who has Parkinson’s disease with a dopamine agonist and you notice that he has developed valvular heart disease

10. Case Series • You spotted the case report of a Parkinson’s patient who developed cardiac valvular disease while on dopamine agonist therapy • You decide to search your case files and find a dozen Parkinson’s disease patients who have been found to have cardiac valvular disease while taking dopamine agonists

11. Ecologic Association In counties where sales of dopamine agonists are high, the rate of valvular heart disease is high. More of a population basis to suggest an association No link between use and illness

12. Case-Control Study Design PD with Valvular Disease PD without Valvular Disease Note the direction of enquiry: starting with outcome and asking about the frequency of exposure and non-exposure

13. Quantitative Measures in Case-Control Studies Odds of disease Exposed: a/b Not Exposed: c/d a b Odds Ratio a/b = ad c/d bc c d Disease Yes No Exposure Yes (dopamine agonist) No Remember: if the frequency of the disease is low, the odds ratio is a good approximation of the relative risk

15. Case-Control Study Two-By-Two Table:Valvular Heart Disease and Exposure to Dopamine Agonists

16. Case-Control Study Results:Valvular Heart Disease and Exposure to Dopamine Agonists • Odds ratio: 8.78 • 95% confidence interval: 1.96 – 39.2 • P-value for the point estimate: 0.00000079 If you have PD and valvular heart disease, your odds of having been treated with a dopamine agonist are 8.78 times what they would be if you do not have valvular heart disease

17. Advantages of Case-Control Studies • Efficient for the study of rare diseases (outcomes) • Typically requires smaller sample sizes and is often less expensive than cohort studies • Can evaluate multiple risk factors in one study • Improved feasibility based on sample size and cost (often the only feasible study design for very rare diseases)

18. Disadvantages / Challenges • Inefficient for rare exposures • Temporal relationship of exposure and outcome may not be clear • Selection bias common - frequency of exposure amongst the sample of cases or controls is not representative of the source population • Recall bias common - systematic difference in recollections of exposure between cases and controls • If multiple risk factors are evaluated, some associations my arise due to chance alone

19. Cohort Studies • Definition of Cohort: A group of individuals that are all similar in some trait and move forward together as a unit • Definition of a Cohort Study: The observation of a cohort, over time, to measure outcome(s)

20. Cohort Study Design

21. Quantitative Measures in Cohort Studies Risk of disease a a+b c c+d a c b d Relative risk a a+b c c+d Disease Yes No  Yes Exposure  No Follow forward to find new cases Thus, you have incident data and calculate risk and risk ratios

22. Advantages • Good for rare exposures • Offers an opportunity for maximal investigator control over: • Exposure classification • Uniform follow-up • Case finding • Can evaluate multiple outcomes in one study • When prospective and done well, may come close (but not quite equal) to a clinical trial in providing reliable data and reliable evidence

23. Disadvantages / Challenges • Large • Change in methods over time • Loss to follow-up is often problematic • When prospective, especially expensive and time consuming • When retrospective, in general, the reliability is close to that of an ecologic association study – it may generate questions, but not answers • If evaluating multiple outcomes, some may appear associated due to chance alone

24. Cohort vs. Randomized Clinical Trial

25. Summing Up

27. CASE-CONTROL STUDY Exposure Identify Outcome in a Patient Group YES Disease (cases) NO Odds Ratio YES No disease (controls) NO COHORT STUDY Outcome (disease) Exposure YES Risk Ratio Absolute Risk Difference Relative Risk YES Identify Patient Group NO YES NO NO

28. Final Words Observational studies are often the most practical way to answer research questions Bias is a major challenge Thus interventional (i.e., randomized, masked) studies are preferred to guide treatment decisions