Retrospective Studies of Clinical Outcomes A Primer for Clinicians - PowerPoint PPT Presentation

retrospective studies of clinical outcomes a primer for clinicians n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Retrospective Studies of Clinical Outcomes A Primer for Clinicians PowerPoint Presentation
Download Presentation
Retrospective Studies of Clinical Outcomes A Primer for Clinicians

play fullscreen
1 / 68
Retrospective Studies of Clinical Outcomes A Primer for Clinicians
229 Views
Download Presentation
mahsa
Download Presentation

Retrospective Studies of Clinical Outcomes A Primer for Clinicians

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Retrospective Studies of Clinical OutcomesA Primer for Clinicians Marc D. Silverstein, MD FACP

  2. Overview • Role of retrospective studies in a research program • “Anatomy” & “physiology” of research • Observational research designs • Cross-sectional studies • Cohort studies • Case-control studies • Human Subjects & IRB Review

  3. Objectives • Describe 4 research designs • Describe 3 threats to validity • List advantages of retrospective studies • List disadvantages of retrospective studies • Understand requirements and processes for IRB review of retrospective studies

  4. Research • Definition • Types of Research

  5. Definition of Clinical ResearchNIH Director’s Panel, 1997 • Patient-oriented research Research conducted with human subjects (or on material of human origin such as tissues, specimens and cognitive phenomena) for which an investigator (or colleague) directly interacts with human subjects • Mechanisms of human disease • Therapeutic interventions • Clinical trials • Development of new technologies • Epidemiologic and behavioral studies • Outcomes research and health services research

  6. Health Services Research • Health services research is the multidisciplinary field of scientific investigation that studies how social factors, financing systems, organizational structures and processes, health technologies, and personal behaviors affect access to health care, the quality and cost of health care, and ultimately our health and well-being. • Its research domains are individuals, families, organizations, institutions, communities, and populations. AcademyHealth

  7. Outcomes Research • Research on measures of changes in patient outcomes - patient health status and satisfaction - resulting from specific medical and health interventions • Attributing changes in outcomes to medical care requires distinguishing the effects of care from the effects of the many other factors that influence patients’ health and satisfaction AcademyHealth

  8. PatientCare Formulate the research question Translate research into practice OutcomesResearch Patient Care and Outcomes Research

  9. Physiology of Research • Design the study • Implement the study • Make valid (causal) inferences

  10. Research Question Truth in the Universe The Goal

  11. Design & Implementation Design Implement Research Question Actual Study Study Plan

  12. Design Implement Research Question Truth in the Universe Actual Study Findings in the Study Study Plan Truth in the Study Infer Infer Causal Inferences

  13. Research Question Truth in the Universe Study Plan Truth in the Study Actual Study Findings in the study Target Population Phenomenon of Interest Intended Sample Intended Variables Actual Subjects Actual Measurements Errors May Occur in Design & Implementation of Research Error Error Design Implement Infer Infer

  14. Research Question Truth in the Universe Study Plan Truth in the Study Actual Study Findings in the study Target Population Phenomenon of Interest Intended Sample Intended Variables Actual Subjects Actual Measurements Errors May Occur in Making Inferences about Internal or External Validity Design Implement Error Error Infer Infer

  15. Research Question Truth in the Universe Study Plan Truth in the Study Actual Study Findings in the study Target Population Phenomenon of Interest Intended Sample Intended Variables Actual Subjects Actual Measurements Errors May Occur Anywhere … Error Error Design Implement Error Error Infer Infer

  16. Threats to Validity • Chance • Bias • Confounding

  17. Threats to Validity • Chance – random error due to unknown sources of variation that distort sample and measurements in either directions • Bias – systematic error that distorts sample and measurements in one direction • Confounding – an external factor that is associated with a predictor variable and an outcome variable

  18. Reducing Random Error • Increase sample size • Statistical analyses

  19. Reducing Systematic Error (Bias) • Population-based studies • Inclusion and exclusion data • Standardize measurement • Train and certify observer • Refine instrument • Automate instruments • Blinded measurements

  20. Reducing Confounding • Anticipate potential confounders • Measure potential confounders • Matching, restriction, stratification • Multivariate analysis

  21. “I cannot give any scientist of any age better advice than this: the intensity of the conviction that a hypothesis is true has no bearing on whether it is true or not.” P.B. Medawar

  22. Anatomy of Research • Research Question • Significance • Methods

  23. Research Question

  24. Characteristics of a Good Research Question • Feasible • Interesting • Novel • Ethical • Relevant

  25. “It can be said with complete confidence that any scientist of any age who wants to make important discoveries must study important problems.” P.B. Medawar

  26. Hypotheses • Simple (versus complex) • Specific (versus vague) • In advance (versus after-the-fact)

  27. Estimation • In clinical studies the goal is often to estimate a risk of an outcome or the magnitude of impact on a clinical measurement

  28. Strengths Intervention is not feasible or ethical Rapid & efficient Existing data Less time Expenses are lower Case-control studies for rare events Limitations Do not permit true assessment of time sequence of factors and outcomes Subject to bias and confounding Limited power to study rare risk factors or rare outcomes (surveys & cohort studies) Observational Studies

  29. Study Designs • Observational studies • Cross sectional studies • Cohort studies • Case Control studies • Experiments

  30. Cross-Sectional Study Design

  31. Risk Factor Disease Risk Factor No Disease No Risk Factor Disease No Risk Factor No Disease Cross Sectional Study Population Sample

  32. Cross Sectional Study Example The Probability of Malignancy in Solitary Pulmonary Nodules Swensen, Silverstein, Ilstrup et al Arch Intern Med 1997; 157: 849

  33. The Probability of Malignancy in Solitary Pulmonary Nodules • Can clinical and radiological SPN characteristics predict malignancy in SPNs • Retrospective cohort at multi-specialty group practice • New diagnosis of 4mm – 30 mm solitary pulmonary nodule • Radiological indeterminate SPN with no calcification on thin section CT • Exclude patients with primary lung cancer or other cancer within 5 years

  34. The Probability of Malignancy in Solitary Pulmonary Nodules • Outcomes determined by radiological follow-up for 2 or more years, surgical diagnosis, transthoracic needle biopsy, bronchoscopy biopsy or washings • Clinical characteristics (age, smoking, history of other cancer) and radiological characteristics (size, location, edge characteristics – lobulation, spiculation, shagginess) • Multivariate analysis with logistic regression • Predictors developed in 2/3 random sample and validated in remaining 1/3 sample

  35. DiscriminationROC Curve Area (0.83, 0.80)

  36. Calibration – Observed vs Predicted Probability

  37. Threats to ValidityMalignancy in SPN, 1 • Large number of SPN’s (629) • Referral population • Clinically relevant SPNs (5-30 mm) • Excludes low risk (< 4mm) • Excludes high risk (> 30 mm) • No calcification on thin section CT (benign) • Cancer diagnosis • Cohort study for outcomes after 2 years • Some SPN’s indeterminate classification

  38. Threats to ValidityMalignancy in SPN, 2 • Independent review single radiologist • Swenson, Silverstein, Edell et al. SPN: Clinical Prediction vs Physicians, Mayo Clin Proc, 1999 • Analysis • Discrimination and calibration • Independent sample for validation • Distance to assess referral bias • Included all SPNs • Malignant vs (indeterminate + benign) • (Malignant + indeterminate) vs benign

  39. Cohort Study Design

  40. Population Sample Risk Factor Disease No Disease No Risk Factor Disease No Disease Cohort Study Design

  41. Population Sample Risk Factor Disease No Disease No Risk Factor Disease No Disease Prospective Cohort Study Design The Present The Future

  42. Population Sample Risk Factor Disease No Disease No Risk Factor Disease No Disease Retrospective Cohort Study Design The Past The Present

  43. Cohort Study Example Long-term Survival of a Cohort of Community Residents with Asthma Silverstein, Reed, O’Connell et al N Eng J Med 1994;331: 1537

  44. Long-term Survival of a Cohort of Community Residents with Asthma • Asthma mortality based on general US population death certificates with asthma listed as underlying cause of death • Residents of Rochester, MN with first asthma diagnosis 1/1/1964-12/31/1983 • Explicit pre-defined criteria, review of all medical records from all providers of care • Medical records and autopsy reports used to classify deaths as due to asthma or other conditions

  45. Long-term Survival of a Cohort of Community Residents with Asthma • 2499 patients with definite or probable asthma • Mean duration follow-up 14 years (range 0-29 years) • 140 deaths in 32,605 person-years of follow-up • Survival not significantly different form expected • Survival worse in asthmatics with other lung disease • 4% of deaths in persons with asthma were due to asthma

  46. Observed vs. Expected Survival

  47. Survival in Asthma only vs Asthma and Other Lung Disease

  48. Threats to ValidityLong-term Survival in Asthma, 1 • Large population-based cohort (2499) • Yunginger, Reed O’Connell, A Community based study of the Epidemiology of asthma, Am Rev Resp Dis, 1992 • Asthma diagnosis • Beard, Yunginger, Reed et al, Interobserver Variability in Medical Record Review: An Epidemiological Study of Asthma, J Clin Epid, 1992

  49. Threats to ValidityLong-term Survival in Asthma, 2 • Asthma deaths • 14 years of follow-up • Small number of deaths (140) • Classification of deaths • Hunt, Silverstein, Reed et al. Accuracy of Death Certificate in a Population-Based Study of asthmatic Patients, JAMA, 1993 • Review of all death certificates & autopsy reports • 13 out of state

  50. Case-Control Study Design