retrospective studies of clinical outcomes a primer for clinicians
Download
Skip this Video
Download Presentation
Retrospective Studies of Clinical Outcomes A Primer for Clinicians

Loading in 2 Seconds...

play fullscreen
1 / 68

Retrospective Studies of Clinical Outcomes A Primer for Clinicians - PowerPoint PPT Presentation


  • 174 Views
  • Uploaded on

Retrospective Studies of Clinical Outcomes A Primer for Clinicians. Marc D. Silverstein, MD FACP. Overview. Role of retrospective studies in a research program “Anatomy” & “physiology” of research Observational research designs Cross-sectional studies Cohort studies Case-control studies

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Retrospective Studies of Clinical Outcomes A Primer for Clinicians' - mahsa


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
retrospective studies of clinical outcomes a primer for clinicians

Retrospective Studies of Clinical OutcomesA Primer for Clinicians

Marc D. Silverstein, MD FACP

overview
Overview
  • Role of retrospective studies in a research program
  • “Anatomy” & “physiology” of research
  • Observational research designs
    • Cross-sectional studies
    • Cohort studies
    • Case-control studies
  • Human Subjects & IRB Review
objectives
Objectives
  • Describe 4 research designs
  • Describe 3 threats to validity
  • List advantages of retrospective studies
  • List disadvantages of retrospective studies
  • Understand requirements and processes for IRB review of retrospective studies
research
Research
  • Definition
  • Types of Research
definition of clinical research nih director s panel 1997
Definition of Clinical ResearchNIH Director’s Panel, 1997
  • Patient-oriented research

Research conducted with human subjects (or on material of human origin such as tissues, specimens and cognitive phenomena) for which an investigator (or colleague) directly interacts with human subjects

    • Mechanisms of human disease
    • Therapeutic interventions
    • Clinical trials
    • Development of new technologies
  • Epidemiologic and behavioral studies
  • Outcomes research and health services research
health services research
Health Services Research
  • Health services research is the multidisciplinary field of scientific investigation that studies how social factors, financing systems, organizational structures and processes, health technologies, and personal behaviors affect access to health care, the quality and cost of health care, and ultimately our health and well-being.
  • Its research domains are individuals, families, organizations, institutions, communities, and populations.

AcademyHealth

outcomes research
Outcomes Research
  • Research on measures of changes in patient outcomes - patient health status and satisfaction - resulting from specific medical and health interventions
  • Attributing changes in outcomes to medical care requires distinguishing the effects of care from the effects of the many other factors that influence patients’ health and satisfaction

AcademyHealth

patient care and outcomes research
PatientCare

Formulate the research question

Translate research into practice

OutcomesResearch

Patient Care and Outcomes Research
physiology of research
Physiology of Research
  • Design the study
  • Implement the study
  • Make valid (causal) inferences
the goal
Research Question

Truth in the Universe

The Goal
design implementation
Design & Implementation

Design

Implement

Research Question

Actual Study

Study Plan

causal inferences
Design

Implement

Research

Question

Truth

in the Universe

Actual Study

Findings

in the Study

Study Plan

Truth

in the Study

Infer

Infer

Causal Inferences
errors may occur in design implementation of research
Research

Question

Truth

in the Universe

Study

Plan

Truth

in the Study

Actual

Study

Findings

in the study

Target

Population

Phenomenon

of Interest

Intended

Sample

Intended

Variables

Actual

Subjects

Actual

Measurements

Errors May Occur in Design & Implementation of Research

Error

Error

Design

Implement

Infer

Infer

errors may occur in making inferences about internal or external validity
Research

Question

Truth

in the Universe

Study

Plan

Truth

in the Study

Actual

Study

Findings

in the study

Target

Population

Phenomenon

of Interest

Intended

Sample

Intended

Variables

Actual

Subjects

Actual

Measurements

Errors May Occur in Making Inferences about Internal or External Validity

Design

Implement

Error

Error

Infer

Infer

errors may occur anywhere
Research

Question

Truth

in the Universe

Study

Plan

Truth

in the Study

Actual

Study

Findings

in the study

Target

Population

Phenomenon

of Interest

Intended

Sample

Intended

Variables

Actual

Subjects

Actual

Measurements

Errors May Occur Anywhere …

Error

Error

Design

Implement

Error

Error

Infer

Infer

threats to validity
Threats to Validity
  • Chance
  • Bias
  • Confounding
threats to validity17
Threats to Validity
  • Chance – random error due to unknown sources of variation that distort sample and measurements in either directions
  • Bias – systematic error that distorts sample and measurements in one direction
  • Confounding – an external factor that is associated with a predictor variable and an outcome variable
reducing random error
Reducing Random Error
  • Increase sample size
  • Statistical analyses
reducing systematic error bias
Reducing Systematic Error (Bias)
  • Population-based studies
  • Inclusion and exclusion data
  • Standardize measurement
  • Train and certify observer
  • Refine instrument
  • Automate instruments
  • Blinded measurements
reducing confounding
Reducing Confounding
  • Anticipate potential confounders
  • Measure potential confounders
  • Matching, restriction, stratification
  • Multivariate analysis
slide21
“I cannot give any scientist of any age better advice than this: the intensity of the conviction that a hypothesis is true has no bearing on whether it is true or not.”

P.B. Medawar

anatomy of research
Anatomy of Research
  • Research Question
  • Significance
  • Methods
characteristics of a good research question
Characteristics of a Good Research Question
  • Feasible
  • Interesting
  • Novel
  • Ethical
  • Relevant
slide25
“It can be said with complete confidence that any scientist of any age who wants to make important discoveries must study important problems.”

P.B. Medawar

hypotheses
Hypotheses
  • Simple (versus complex)
  • Specific (versus vague)
  • In advance (versus after-the-fact)
estimation
Estimation
  • In clinical studies the goal is often to estimate a risk of an outcome or the magnitude of impact on a clinical measurement
observational studies
Strengths

Intervention is not feasible or ethical

Rapid & efficient

Existing data

Less time

Expenses are lower

Case-control studies for rare events

Limitations

Do not permit true assessment of time sequence of factors and outcomes

Subject to bias and confounding

Limited power to study rare risk factors or rare outcomes (surveys & cohort studies)

Observational Studies
study designs
Study Designs
  • Observational studies
    • Cross sectional studies
    • Cohort studies
    • Case Control studies
  • Experiments
cross sectional study
Risk Factor

Disease

Risk Factor

No Disease

No Risk Factor

Disease

No Risk Factor

No Disease

Cross Sectional Study

Population

Sample

cross sectional study example

Cross Sectional Study Example

The Probability of Malignancy in Solitary Pulmonary Nodules

Swensen, Silverstein, Ilstrup et al

Arch Intern Med 1997; 157: 849

the probability of malignancy in solitary pulmonary nodules
The Probability of Malignancy in Solitary Pulmonary Nodules
  • Can clinical and radiological SPN characteristics predict malignancy in SPNs
  • Retrospective cohort at multi-specialty group practice
  • New diagnosis of 4mm – 30 mm solitary pulmonary nodule
  • Radiological indeterminate SPN with no calcification on thin section CT
  • Exclude patients with primary lung cancer or other cancer within 5 years
the probability of malignancy in solitary pulmonary nodules34
The Probability of Malignancy in Solitary Pulmonary Nodules
  • Outcomes determined by radiological follow-up for 2 or more years, surgical diagnosis, transthoracic needle biopsy, bronchoscopy biopsy or washings
  • Clinical characteristics (age, smoking, history of other cancer) and radiological characteristics (size, location, edge characteristics – lobulation, spiculation, shagginess)
  • Multivariate analysis with logistic regression
  • Predictors developed in 2/3 random sample and validated in remaining 1/3 sample
threats to validity malignancy in spn 1
Threats to ValidityMalignancy in SPN, 1
  • Large number of SPN’s (629)
  • Referral population
  • Clinically relevant SPNs (5-30 mm)
    • Excludes low risk (< 4mm)
    • Excludes high risk (> 30 mm)
    • No calcification on thin section CT (benign)
  • Cancer diagnosis
    • Cohort study for outcomes after 2 years
    • Some SPN’s indeterminate classification
threats to validity malignancy in spn 2
Threats to ValidityMalignancy in SPN, 2
  • Independent review single radiologist
    • Swenson, Silverstein, Edell et al. SPN: Clinical Prediction vs Physicians, Mayo Clin Proc, 1999
  • Analysis
    • Discrimination and calibration
    • Independent sample for validation
    • Distance to assess referral bias
    • Included all SPNs
      • Malignant vs (indeterminate + benign)
      • (Malignant + indeterminate) vs benign
cohort study design40
Population

Sample

Risk Factor

Disease

No Disease

No Risk Factor

Disease

No Disease

Cohort Study Design
prospective cohort study design
Population

Sample

Risk Factor

Disease

No Disease

No Risk Factor

Disease

No Disease

Prospective Cohort Study Design

The Present

The Future

retrospective cohort study design
Population

Sample

Risk Factor

Disease

No Disease

No Risk Factor

Disease

No Disease

Retrospective Cohort Study Design

The Past

The Present

cohort study example

Cohort Study Example

Long-term Survival of a Cohort of Community Residents with Asthma

Silverstein, Reed, O’Connell et al

N Eng J Med 1994;331: 1537

long term survival of a cohort of community residents with asthma
Long-term Survival of a Cohort of Community Residents with Asthma
  • Asthma mortality based on general US population death certificates with asthma listed as underlying cause of death
  • Residents of Rochester, MN with first asthma diagnosis 1/1/1964-12/31/1983
  • Explicit pre-defined criteria, review of all medical records from all providers of care
  • Medical records and autopsy reports used to classify deaths as due to asthma or other conditions
long term survival of a cohort of community residents with asthma45
Long-term Survival of a Cohort of Community Residents with Asthma
  • 2499 patients with definite or probable asthma
  • Mean duration follow-up 14 years (range 0-29 years)
  • 140 deaths in 32,605 person-years of follow-up
  • Survival not significantly different form expected
  • Survival worse in asthmatics with other lung disease
  • 4% of deaths in persons with asthma were due to asthma
threats to validity long term survival in asthma 1
Threats to ValidityLong-term Survival in Asthma, 1
  • Large population-based cohort (2499)
    • Yunginger, Reed O’Connell, A Community based study of the Epidemiology of asthma, Am Rev Resp Dis, 1992
  • Asthma diagnosis
    • Beard, Yunginger, Reed et al, Interobserver Variability in Medical Record Review: An Epidemiological Study of Asthma, J Clin Epid, 1992
threats to validity long term survival in asthma 2
Threats to ValidityLong-term Survival in Asthma, 2
  • Asthma deaths
    • 14 years of follow-up
    • Small number of deaths (140)
    • Classification of deaths
      • Hunt, Silverstein, Reed et al. Accuracy of Death Certificate in a Population-Based Study of asthmatic Patients, JAMA, 1993
    • Review of all death certificates & autopsy reports
      • 13 out of state
case control study design51
Population

Risk Factor

Sample Disease

“Cases”

No Risk Factor

Sample No Disease

Risk Factor

“Controls”

No Risk Factor

Case-Control Study Design

The Past

The Present

case control study example

Case Control Study Example

Risk Factors for Deep Vein Thrombosis and Pulmonary Embolism

Heit, Silverstein, Mohr et al

Arch Intern Med 2000; 160: 809-15

risk factors for deep vein thrombosis and pulmonary embolism
Risk Factors for Deep Vein Thrombosis and Pulmonary Embolism
  • Reported risk factors vary in magnitude and independence of each are unknown
  • Population based nested case-control study
  • Cases: 625 Olmsted County residents with first lifetime VTE 1/1/1976-12/31/-1990
  • Controls: 625 Olmsted County residents without VTE
  • Matched on age, sex, calendar year, and medical record number (duration of observation)
risk factors for deep vein thrombosis and pulmonary embolism54
Risk Factors for Deep Vein Thrombosis and Pulmonary Embolism
  • Mean duration of medical records documentation reviewed 34.7 years
  • 30 potential independent risk factors
  • Conditional logistic regression to identify independent risk factors
threats to validity risk factors for vte 1
Threats to ValidityRisk Factors for VTE, 1
  • Population-based inception cohort
    • Definite VTE confirmed by imaging
    • Antemortum and postmortum ascertainment of VTE
    • Silverstein, Heit, Mohr et al Trends in the Incidence of DVT and PE, Arch Intern Med 1998
  • Risk factors
    • Large number of pre-specified risk factors analyzed
    • Matching limits ability to analyze age and sex as risk factors
    • Unable to analyze immobilization due to potential diagnostic suspicion bias
    • Protein C had not been reported at time study was initiated
threats to validity risk factors for vte 2
Threats to ValidityRisk Factors for VTE, 2
  • Multivariate analysis
    • Large number of potential confounders anticipated to control for confounding
    • Bootstrap validation of multivariate analysis
    • Age effect for risk due to varicose veins
    • Treatment effect for risk due to cancer
study components
Study Components
  • Abstract
  • Specific Aims
  • Background & significance
  • Preliminary Data
  • Methods
  • Human Subjects
  • References
methods
Methods
  • Design
  • Setting
  • Subjects
  • Measurements
  • Interventions
  • Analysis & Sample Size
  • Interpretation & Limitations
human subjects
Human Subjects
  • Risks
  • Benefits
  • Risks in Relation to Benefits
  • Privacy & Confidentiality
  • Informed Consent
  • Women, children, minorities, priority populations
irb review observational studies
IRB Review - Observational Studies
  • Record review study (retrospective study)
  • Observational study (prospective study)
irb review record review studies
IRB ReviewRecord Review Studies
  • Risks
    • Criminal or civil liability
    • Damaging employability, financial status or reputation
  • Ethical Issues
    • Privacy
    • Confidentiality
  • Issues fro investigators
    • Data collection
    • Data security
informed consent
Informed Consent
  • Must be obtained, unless IRB approves waiver of informed consent
    • Minimal risk
    • Right and welfare of subjects not adversely affected by waiver of informed consent
    • Research could not be practically carried out without the waiver
category of irb review
Category of IRB Review
  • Exempt Studies
    • Record review studies may be exempt form IRB review
    • Determined by Chair of IRB
    • Minimal risk
  • Expedited Review
    • Some IRBs allow expedited review for record review studies
    • Determined by chair of IRB
      • Publicly available data
      • No personal identifiers and records can not be linked to subject
  • Full IRB Review
informed consent65
Informed Consent
  • Protect human subjects/volunteers
  • Ensure that study subjects understand benefits and risks
  • Provide potential subjects with all the information to reach a decision on whether or not to participate in a research study
informed consent requirements
Informed Consent Requirements
  • Benefits
  • Alternatives
  • Confidentiality
  • Compensation
  • Medical treatment for injury
  • Where to obtain further information
  • Contact person
  • Voluntary participation
objectives68
Objectives
  • Describe 4 research designs
  • Describe 3 threats to validity
  • List advantages of retrospective studies
  • List disadvantages to retrospective studies
  • Understand requirements and processes for IRB review of retrospective studies
ad