External validity to what populations do our study results apply
Download
1 / 39

External validity: to what populations do our study results apply ? - PowerPoint PPT Presentation


  • 95 Views
  • Uploaded on

External validity: to what populations do our study results apply ? . Epidemiology matters: a new introduction to methodological foundations Chapter 12. Seven steps. Define the population of interest Conceptualize and create measures of exposures and health indicators

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' External validity: to what populations do our study results apply ? ' - muniya


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
External validity to what populations do our study results apply

External validity: to what populations do our study results apply?

Epidemiology matters: a new introduction to methodological foundations

Chapter 12


Seven steps
Seven steps

  • Define the population of interest

  • Conceptualize and create measures of exposures and health indicators

  • Take a sample of the population

  • Estimate measures of association between exposures and health indicators of interest

  • Rigorously evaluate whether the association observed suggests a causal association

  • Assess the evidence for causes working together

  • Assess the extent to which the result matters, is externally valid, to other populations

Epidemiology Matters – Chapter 1


Generalizability or external validity refers to our capacity to generalize our results beyond our study sample

Epidemiology Matters – Chapter 12


Question: How can we assess the extent to which results of a study are applicable in populations outside of the underlying population base of particular study?

Answer: Think through characteristics of population of interest and determine how robust study findings might be across populations with similar or different characteristics.

Epidemiology Matters – Chapter 12


  • Validity, four stages

  • Introduction to external validity

  • Prevalence of component causes

  • Causation and study design

  • External versus internal validity

  • Randomized control trials

  • Representative samples

  • Summary

Epidemiology Matters – Chapter 12


  • Validity, four stages

  • Introduction to external validity

  • Prevalence of component causes

  • Causation and study design

  • External versus internal validity

  • Randomized control trials

  • Representative samples

  • Summary

Epidemiology Matters – Chapter 12


Four types of validity
Four types of validity

  • Measurement validity

  • Statistical conclusion validity

  • Internal validity

  • External validity

Epidemiology Matters – Chapter 12


1 measurement validity
1. Measurement validity

  • An association cannot be valid beyond the study sample unless it is valid within the study sample

  • Accuracy and precision are key measurements

  • Have we measured what we wanted to measure?

Epidemiology Matters – Chapter 12


2 statistical conclusion validity
2. Statistical conclusion validity

  • Is the association observed due to chance?

  • We assess this via confidence intervals around estimates of association to describe role of sampling variability

  • We aim to rule out the potential that our results arose by chance in the sampling process from an underlying population of interest

Epidemiology Matters – Chapter 12


3 internal validity
3. Internal validity

  • Assessment of non-comparability between exposed and non-exposed in any study

Epidemiology Matters – Chapter 12


4 external validity
4. External validity

  • Explore external validity after assessing and ensuring

    • Measurement validity

    • Statistical conclusion validity

    • Internal validity

Epidemiology Matters – Chapter 12


  • Validity, four stages

  • Introduction to external validity

  • Prevalence of component causes

  • Causation and study design

  • External versus internal validity

  • Randomized control trials

  • Representative samples

  • Summary

Epidemiology Matters – Chapter 12


External validity
External validity

External validity: the applicability of study findings beyond the study sample

In an epidemiologic study we

  • Identify a population of interest

  • Sample from population – random or purposive

  • Conduct study

  • Sample result should reflect underlying association in population of interest

    Therefore, identifying population of interest is central to exploring external validity once we have our findings

Epidemiology Matters – Chapter 12


  • Validity, four stages

  • Introduction to external validity

  • Prevalence of component causes

  • Causation and study design

  • External versus internal validity

  • Randomized control trials

  • Representative samples

  • Summary

Epidemiology Matters – Chapter 12


Prevalence of component causes
Prevalence of component causes

To understand external validity we must understand the prevalence and distribution of component causes across populations.

Epidemiology Matters – Chapter 12


Prevalence of component causes example
Prevalence of component causes, example

Question: Does exposure to ambient air pollution cause lung cancer?

  • Component cause A: Ambient air pollution and smoking; therefore, smoking will cause lung cancer only among individuals exposed to ambient air pollution

  • Component cause B: Genetics

Epidemiology Matters – Chapter 12


Exposed genetic, diseased

Exposed genetic, diseased, exposed air pollution, smoker

Non-diseased

Diseased

Non-exposed

Exposed air pollution

Smoker

Exposed air pollution, diseased, smoker

Epidemiology Matters – Chapter 12


Prevalence of component causes example lung cancer
Prevalence of component causesexample lung cancer

Population 1

Black = exposed to air pollution

Dots = genetically determined

Hat = smoker

Exposed to air pollution and genetic

(regardless of disease status and smoking status)

2

Un-exposed to air pollution and exposed to genetic

(regardless of disease status and smoking status)

2

Exposed to air pollution and smoker

(regardless of disease status and genetic status)

6

Un-exposed to air pollution and smoker

(regardless of disease status and genetic status)

6

Epidemiology Matters – Chapter 12


Prevalence of component causes example lung cancer1
Prevalence of component causesexample lung cancer

Population 2

Black = exposed to air pollution

Dots = genetically determined

Hat = smoker

Exposed to air pollution and genetic

(regardless of disease status and smoking status)

2

Un-exposed to air pollution and exposed to genetic

(regardless of disease status and smoking status)

2

Exposed to air pollution and smoker

(regardless of disease status and genetic status)

3

Un-exposed to air pollution and smoker

(regardless of disease status and genetic status)

3

Epidemiology Matters – Chapter 12


Prevalence of component causes example lung cancer risk difference
Prevalence of component causesexample lung cancer risk difference

Population 1

Population 2

Black = exposed air pollution

Dots = genetically determined

Hat = smoker

Exposed diseased = 3 = 30% risk

Unexposed diseased = 2 = 20% risk

Risk difference: 30% – 20% = 10%

Interpretation: 10 cases of lung cancer are associated with ambient air pollution per 100 exposed

Exposed diseased = 6 = 60% risk

Unexposed diseased = 2 = 20% risk

Risk difference: 60% – 20% = 40%

Interpretation: 40 cases of lung cancer are associated with ambient air pollution per 100 exposed

Epidemiology Matters – Chapter 12


Prevalence of component causes example lung cancer interpretation
Prevalence of component causesexample lung cancer interpretation

  • Two studies asked the same question

  • Both are internally valid studies because the exposed and unexposed are comparable on genetic determinism

    • Population 1: the causal effect is a risk difference of 40%

    • Population 2 : the causal effect is a risk difference of 10%

      Why do these two causal effects differ?

  • Prevalence of people exposed to ambient air pollution is the same in both studies

  • Prevalence of genetic determinism is same in both studies

  • The reason that these two causal effects diverge is the different prevalence of smoking between the two populations

Epidemiology Matters – Chapter 12


Prevalence of component causes example lung cancer interpretation1
Prevalence of component causesexample lung cancer interpretation

  • When two causes interact the measure of association for the effect of one cause on the outcome will differ across levels of the second cause

  • Air pollution example

    • Ambient air pollution and smoking are causal partners within the same sufficient cause

    • Prevalence of one of them (smoking) influences the causal effect of the other (ambient air pollution) on the outcome (lung cancer)

    • We would therefore expect the causal effect of ambient air pollution on lung cancer to differ across population where prevalence of smoking also differs

  • Therefore, there is no one causal effect for all populations; the causal effect is dependent on prevalence of component causes in each population

  • Therefore, the result from one study will be externally valid to populations in which the distribution of component causes of exposure is similar to the study sample

  • Epidemiology Matters – Chapter 12


    • Validity, four stages

    • Introduction to external validity

    • Prevalence of component causes

    • Causation and study design

    • External versus internal validity

    • Randomized control trials

    • Representative samples

    • Summary

    Epidemiology Matters – Chapter 12


    Causation and study design
    Causation and study design

    • The magnitude of an association will be applicable beyond our study to the extent that the distribution of causal partners of exposure is similar in the population

    • If we want to identify a cause of disease, should it be a cause absolutely and in all types of populations?

    Epidemiology Matters – Chapter 12


    Causation and study design1
    Causation and study design

    • Cause: a factor that was necessary for that disease to occur in an individual at that time; most causes are insufficient and unnecessary in isolation

    • Causal effect: epidemiology studies populations; therefore we focus on the effect of causes

    • We document an association between those who embody cause (exposed) and those who do not (unexposed); this is context specific and dependent on prevalence of component causes

    • Therefore, understanding a cause in context of causal partners is central to theory, design, and analysis

    Epidemiology Matters – Chapter 12


    • Validity, four stages

    • Introduction to external validity

    • Prevalence of component causes

    • Causation and study design

    • External versus internal validity

    • Randomized control trials

    • Representative samples

    • Summary

    Epidemiology Matters – Chapter 12


    External vs internal validity
    External vs. internal validity

    • Internal validity is a prerequisite to external validity

    • To achieve internal validity we need to design a study with narrow population of interest and minimize non-comparability

    • The resulting sample may not reflect broader swath of population beyond underlying population of interest

    • The more narrow a sample becomes - due to strict inclusion and exclusion criteria for internal validity - the less external validity it may have if causal partners of exposure have differing prevalence in study compared with other populations

    Epidemiology Matters – Chapter 12


    External vs internal validity1
    External vs. internal validity

    • Balancing external and internal validity is a a trade off

    • To build a scientific argument for causal effect of exposure on outcome, we select study design and assess internal validity of causal question

    • After causal effect of exposure is established in narrow population, we expand the causal question to ask

      • How often?

      • Among whom?

      • Under what conditions?

    Epidemiology Matters – Chapter 12


    • Validity, four stages

    • Introduction to external validity

    • Prevalence of component causes

    • Causation and study design

    • External versus internal validity

    • Randomized control trials

    • Representative samples

    • Summary

    Epidemiology Matters – Chapter 12


    An example rct
    An example, RCT

    Question: Does weight-loss drug reduce obesity among school-age children?

    Study details:

    • Recruit children for randomized drug trial with body mass index 25< (BMI) < 40

    • Exclude children with diabetes

    • Parents must be fully participatory (monitoring children’s drug regime and attend study clinic once per week)

    • Baseline survey and monthly measurements

    • Children randomized to receive weight loss drug or placebo

    • Follow-up over two years

      Study results:

    • Mean BMI among drug group declines 31.5 to 26.7 (4.8 points BMI)

    • Mean BMI among placebo group declines from 31.4 to 28.5 (2.9 points BMI)

    • Reduction of 1.9 (95% CI 0.9 – 2.9) more points of BMI in drug group than placebo group

      Conclusion: Weight-loss drug reduced obesity among school-age children

    Epidemiology Matters – Chapter 12


    An example rct1
    An example, RCT

    Questions to ask about external validity

    • Are these results externally valid to a broader population all overweight children in Farrlandia?

    • What about overweight children in other places?

    • What information would we need to know in order to inform this issue?

    Epidemiology Matters – Chapter 12


    An example rct2
    An example, RCT

    Are we confident of a causal effect in the study?

    • Can only be externally valid if internally valid

    • Good reason to conclude that results obtained are approximation of causal effect of drug for population

      Consider characteristics of population from which participants were drawn

    • Good adherers to study protocol

    • Diabetes-free

    • Actively participating parents

    • There is evidence that drug is effective in reducing BMI

      Consider characteristics of larger population to assess external validity

    • Does action of drug interact with other factors?

    • Do other factors have a different prevalence in general the general population of overweight children who may be prescribed the drug?


    • Validity, four stages

    • Introduction to external validity

    • Prevalence of component causes

    • Causation and study design

    • External versus internal validity

    • Randomized control trials

    • Representative samples

    • Summary

    Epidemiology Matters – Chapter 12


    Example representative sample
    Example,representative sample

    Question: Do sales tax on sugar-sweetened beverages reduce obesity among children aged 7 to 13?

    Study details:

    • Enumerate all school-age children in Farrlandia

    • Take random sample of 1,000 eligible Farrlandianswho are between age 7 to 13 and live in Farrlandia

    • Measure BMI before tax goes into effect

    • Measure BMI after tax across a two-year period

      Studyresults:

    • Mean BMI school-age children prior to the tax = 26.7 (95% C.I. 24.2-29.3)

    • Mean BMI of school-age children was 24.3 (95% C.I. 23.7-24.9) two years after tax

      Conclusion: Tax lowered mean BMI among school-age children

    Epidemiology Matters – Chapter 12


    Example representative sample1
    Example, representative sample

    Snowtownis considering a similar tax

    • Are Farrlandianresults externally valid to Snowtown?

    • What information would we need to know about Farrlandians and Snowtownians?

    Epidemiology Matters – Chapter 12


    Example representative sample2
    Example, representative sample

    Are we confident of a causal effect in the study?

    • Internal validity: If school lunches changed to healthier offerings during study period we would not make a causal claim that tax reduced BMI

      What are potential causal partners of soda tax?

    • External validity

    • Soda availability is component cause

      Is the distribution of causal partners similar across populations?

    • Soda is plentiful in Farrlandia

    • Soda is hard to find in Snowtown

    Epidemiology Matters – Chapter 12


    • Validity, four stages

    • Introduction to external validity

    • Prevalence of component causes

    • Causation and study design

    • External versus internal validity

    • Randomized control trials

    • Representative samples

    • Summary

    Epidemiology Matters – Chapter 12


    Seven steps1
    Seven steps

    • Define the population of interest

    • Conceptualize and create measures of exposures and health indicators

    • Take a sample of the population

    • Estimate measures of association between exposures and health indicators of interest

    • Rigorously evaluate whether the association observed suggests a causal association

    • Assess the evidence for causes working together

    • Assess the extent to which the result matters, is externally valid, to other populations

    Epidemiology Matters – Chapter 1


    epidemiologymatters.org

    Epidemiology Matters – Chapter 1


    ad