external validity to what populations do our study results apply
Download
Skip this Video
Download Presentation
External validity: to what populations do our study results apply ?

Loading in 2 Seconds...

play fullscreen
1 / 39

External validity: to what populations do our study results apply ? - PowerPoint PPT Presentation


  • 95 Views
  • Uploaded on

External validity: to what populations do our study results apply ? . Epidemiology matters: a new introduction to methodological foundations Chapter 12. Seven steps. Define the population of interest Conceptualize and create measures of exposures and health indicators

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' External validity: to what populations do our study results apply ? ' - muniya


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
external validity to what populations do our study results apply

External validity: to what populations do our study results apply?

Epidemiology matters: a new introduction to methodological foundations

Chapter 12

seven steps
Seven steps
  • Define the population of interest
  • Conceptualize and create measures of exposures and health indicators
  • Take a sample of the population
  • Estimate measures of association between exposures and health indicators of interest
  • Rigorously evaluate whether the association observed suggests a causal association
  • Assess the evidence for causes working together
  • Assess the extent to which the result matters, is externally valid, to other populations

Epidemiology Matters – Chapter 1

slide3

Generalizability or external validity refers to our capacity to generalize our results beyond our study sample

Epidemiology Matters – Chapter 12

slide4

Question: How can we assess the extent to which results of a study are applicable in populations outside of the underlying population base of particular study?

Answer: Think through characteristics of population of interest and determine how robust study findings might be across populations with similar or different characteristics.

Epidemiology Matters – Chapter 12

slide5

Validity, four stages

  • Introduction to external validity
  • Prevalence of component causes
  • Causation and study design
  • External versus internal validity
  • Randomized control trials
  • Representative samples
  • Summary

Epidemiology Matters – Chapter 12

slide6

Validity, four stages

  • Introduction to external validity
  • Prevalence of component causes
  • Causation and study design
  • External versus internal validity
  • Randomized control trials
  • Representative samples
  • Summary

Epidemiology Matters – Chapter 12

four types of validity
Four types of validity
  • Measurement validity
  • Statistical conclusion validity
  • Internal validity
  • External validity

Epidemiology Matters – Chapter 12

1 measurement validity
1. Measurement validity
  • An association cannot be valid beyond the study sample unless it is valid within the study sample
  • Accuracy and precision are key measurements
  • Have we measured what we wanted to measure?

Epidemiology Matters – Chapter 12

2 statistical conclusion validity
2. Statistical conclusion validity
  • Is the association observed due to chance?
  • We assess this via confidence intervals around estimates of association to describe role of sampling variability
  • We aim to rule out the potential that our results arose by chance in the sampling process from an underlying population of interest

Epidemiology Matters – Chapter 12

3 internal validity
3. Internal validity
  • Assessment of non-comparability between exposed and non-exposed in any study

Epidemiology Matters – Chapter 12

4 external validity
4. External validity
  • Explore external validity after assessing and ensuring
    • Measurement validity
    • Statistical conclusion validity
    • Internal validity

Epidemiology Matters – Chapter 12

slide12

Validity, four stages

  • Introduction to external validity
  • Prevalence of component causes
  • Causation and study design
  • External versus internal validity
  • Randomized control trials
  • Representative samples
  • Summary

Epidemiology Matters – Chapter 12

external validity
External validity

External validity: the applicability of study findings beyond the study sample

In an epidemiologic study we

  • Identify a population of interest
  • Sample from population – random or purposive
  • Conduct study
  • Sample result should reflect underlying association in population of interest

Therefore, identifying population of interest is central to exploring external validity once we have our findings

Epidemiology Matters – Chapter 12

slide14

Validity, four stages

  • Introduction to external validity
  • Prevalence of component causes
  • Causation and study design
  • External versus internal validity
  • Randomized control trials
  • Representative samples
  • Summary

Epidemiology Matters – Chapter 12

prevalence of component causes
Prevalence of component causes

To understand external validity we must understand the prevalence and distribution of component causes across populations.

Epidemiology Matters – Chapter 12

prevalence of component causes example
Prevalence of component causes, example

Question: Does exposure to ambient air pollution cause lung cancer?

  • Component cause A: Ambient air pollution and smoking; therefore, smoking will cause lung cancer only among individuals exposed to ambient air pollution
  • Component cause B: Genetics

Epidemiology Matters – Chapter 12

slide17

Exposed genetic, diseased

Exposed genetic, diseased, exposed air pollution, smoker

Non-diseased

Diseased

Non-exposed

Exposed air pollution

Smoker

Exposed air pollution, diseased, smoker

Epidemiology Matters – Chapter 12

prevalence of component causes example lung cancer
Prevalence of component causesexample lung cancer

Population 1

Black = exposed to air pollution

Dots = genetically determined

Hat = smoker

Exposed to air pollution and genetic

(regardless of disease status and smoking status)

2

Un-exposed to air pollution and exposed to genetic

(regardless of disease status and smoking status)

2

Exposed to air pollution and smoker

(regardless of disease status and genetic status)

6

Un-exposed to air pollution and smoker

(regardless of disease status and genetic status)

6

Epidemiology Matters – Chapter 12

prevalence of component causes example lung cancer1
Prevalence of component causesexample lung cancer

Population 2

Black = exposed to air pollution

Dots = genetically determined

Hat = smoker

Exposed to air pollution and genetic

(regardless of disease status and smoking status)

2

Un-exposed to air pollution and exposed to genetic

(regardless of disease status and smoking status)

2

Exposed to air pollution and smoker

(regardless of disease status and genetic status)

3

Un-exposed to air pollution and smoker

(regardless of disease status and genetic status)

3

Epidemiology Matters – Chapter 12

prevalence of component causes example lung cancer risk difference
Prevalence of component causesexample lung cancer risk difference

Population 1

Population 2

Black = exposed air pollution

Dots = genetically determined

Hat = smoker

Exposed diseased = 3 = 30% risk

Unexposed diseased = 2 = 20% risk

Risk difference: 30% – 20% = 10%

Interpretation: 10 cases of lung cancer are associated with ambient air pollution per 100 exposed

Exposed diseased = 6 = 60% risk

Unexposed diseased = 2 = 20% risk

Risk difference: 60% – 20% = 40%

Interpretation: 40 cases of lung cancer are associated with ambient air pollution per 100 exposed

Epidemiology Matters – Chapter 12

prevalence of component causes example lung cancer interpretation
Prevalence of component causesexample lung cancer interpretation
  • Two studies asked the same question
  • Both are internally valid studies because the exposed and unexposed are comparable on genetic determinism
    • Population 1: the causal effect is a risk difference of 40%
    • Population 2 : the causal effect is a risk difference of 10%

Why do these two causal effects differ?

  • Prevalence of people exposed to ambient air pollution is the same in both studies
  • Prevalence of genetic determinism is same in both studies
  • The reason that these two causal effects diverge is the different prevalence of smoking between the two populations

Epidemiology Matters – Chapter 12

prevalence of component causes example lung cancer interpretation1
Prevalence of component causesexample lung cancer interpretation
  • When two causes interact the measure of association for the effect of one cause on the outcome will differ across levels of the second cause
  • Air pollution example
      • Ambient air pollution and smoking are causal partners within the same sufficient cause
      • Prevalence of one of them (smoking) influences the causal effect of the other (ambient air pollution) on the outcome (lung cancer)
      • We would therefore expect the causal effect of ambient air pollution on lung cancer to differ across population where prevalence of smoking also differs
  • Therefore, there is no one causal effect for all populations; the causal effect is dependent on prevalence of component causes in each population
  • Therefore, the result from one study will be externally valid to populations in which the distribution of component causes of exposure is similar to the study sample

Epidemiology Matters – Chapter 12

slide23

Validity, four stages

  • Introduction to external validity
  • Prevalence of component causes
  • Causation and study design
  • External versus internal validity
  • Randomized control trials
  • Representative samples
  • Summary

Epidemiology Matters – Chapter 12

causation and study design
Causation and study design
  • The magnitude of an association will be applicable beyond our study to the extent that the distribution of causal partners of exposure is similar in the population
  • If we want to identify a cause of disease, should it be a cause absolutely and in all types of populations?

Epidemiology Matters – Chapter 12

causation and study design1
Causation and study design
  • Cause: a factor that was necessary for that disease to occur in an individual at that time; most causes are insufficient and unnecessary in isolation
  • Causal effect: epidemiology studies populations; therefore we focus on the effect of causes
  • We document an association between those who embody cause (exposed) and those who do not (unexposed); this is context specific and dependent on prevalence of component causes
  • Therefore, understanding a cause in context of causal partners is central to theory, design, and analysis

Epidemiology Matters – Chapter 12

slide26

Validity, four stages

  • Introduction to external validity
  • Prevalence of component causes
  • Causation and study design
  • External versus internal validity
  • Randomized control trials
  • Representative samples
  • Summary

Epidemiology Matters – Chapter 12

external vs internal validity
External vs. internal validity
  • Internal validity is a prerequisite to external validity
  • To achieve internal validity we need to design a study with narrow population of interest and minimize non-comparability
  • The resulting sample may not reflect broader swath of population beyond underlying population of interest
  • The more narrow a sample becomes - due to strict inclusion and exclusion criteria for internal validity - the less external validity it may have if causal partners of exposure have differing prevalence in study compared with other populations

Epidemiology Matters – Chapter 12

external vs internal validity1
External vs. internal validity
  • Balancing external and internal validity is a a trade off
  • To build a scientific argument for causal effect of exposure on outcome, we select study design and assess internal validity of causal question
  • After causal effect of exposure is established in narrow population, we expand the causal question to ask
      • How often?
      • Among whom?
      • Under what conditions?

Epidemiology Matters – Chapter 12

slide29

Validity, four stages

  • Introduction to external validity
  • Prevalence of component causes
  • Causation and study design
  • External versus internal validity
  • Randomized control trials
  • Representative samples
  • Summary

Epidemiology Matters – Chapter 12

an example rct
An example, RCT

Question: Does weight-loss drug reduce obesity among school-age children?

Study details:

  • Recruit children for randomized drug trial with body mass index 25< (BMI) < 40
  • Exclude children with diabetes
  • Parents must be fully participatory (monitoring children’s drug regime and attend study clinic once per week)
  • Baseline survey and monthly measurements
  • Children randomized to receive weight loss drug or placebo
  • Follow-up over two years

Study results:

  • Mean BMI among drug group declines 31.5 to 26.7 (4.8 points BMI)
  • Mean BMI among placebo group declines from 31.4 to 28.5 (2.9 points BMI)
  • Reduction of 1.9 (95% CI 0.9 – 2.9) more points of BMI in drug group than placebo group

Conclusion: Weight-loss drug reduced obesity among school-age children

Epidemiology Matters – Chapter 12

an example rct1
An example, RCT

Questions to ask about external validity

  • Are these results externally valid to a broader population all overweight children in Farrlandia?
  • What about overweight children in other places?
  • What information would we need to know in order to inform this issue?

Epidemiology Matters – Chapter 12

an example rct2
An example, RCT

Are we confident of a causal effect in the study?

  • Can only be externally valid if internally valid
  • Good reason to conclude that results obtained are approximation of causal effect of drug for population

Consider characteristics of population from which participants were drawn

  • Good adherers to study protocol
  • Diabetes-free
  • Actively participating parents
  • There is evidence that drug is effective in reducing BMI

Consider characteristics of larger population to assess external validity

  • Does action of drug interact with other factors?
  • Do other factors have a different prevalence in general the general population of overweight children who may be prescribed the drug?
slide33

Validity, four stages

  • Introduction to external validity
  • Prevalence of component causes
  • Causation and study design
  • External versus internal validity
  • Randomized control trials
  • Representative samples
  • Summary

Epidemiology Matters – Chapter 12

example representative sample
Example,representative sample

Question: Do sales tax on sugar-sweetened beverages reduce obesity among children aged 7 to 13?

Study details:

  • Enumerate all school-age children in Farrlandia
  • Take random sample of 1,000 eligible Farrlandianswho are between age 7 to 13 and live in Farrlandia
  • Measure BMI before tax goes into effect
  • Measure BMI after tax across a two-year period

Studyresults:

  • Mean BMI school-age children prior to the tax = 26.7 (95% C.I. 24.2-29.3)
  • Mean BMI of school-age children was 24.3 (95% C.I. 23.7-24.9) two years after tax

Conclusion: Tax lowered mean BMI among school-age children

Epidemiology Matters – Chapter 12

example representative sample1
Example, representative sample

Snowtownis considering a similar tax

  • Are Farrlandianresults externally valid to Snowtown?
  • What information would we need to know about Farrlandians and Snowtownians?

Epidemiology Matters – Chapter 12

example representative sample2
Example, representative sample

Are we confident of a causal effect in the study?

  • Internal validity: If school lunches changed to healthier offerings during study period we would not make a causal claim that tax reduced BMI

What are potential causal partners of soda tax?

  • External validity
  • Soda availability is component cause

Is the distribution of causal partners similar across populations?

  • Soda is plentiful in Farrlandia
  • Soda is hard to find in Snowtown

Epidemiology Matters – Chapter 12

slide37

Validity, four stages

  • Introduction to external validity
  • Prevalence of component causes
  • Causation and study design
  • External versus internal validity
  • Randomized control trials
  • Representative samples
  • Summary

Epidemiology Matters – Chapter 12

seven steps1
Seven steps
  • Define the population of interest
  • Conceptualize and create measures of exposures and health indicators
  • Take a sample of the population
  • Estimate measures of association between exposures and health indicators of interest
  • Rigorously evaluate whether the association observed suggests a causal association
  • Assess the evidence for causes working together
  • Assess the extent to which the result matters, is externally valid, to other populations

Epidemiology Matters – Chapter 1

slide39

epidemiologymatters.org

Epidemiology Matters – Chapter 1

ad