sample size and power
Download
Skip this Video
Download Presentation
Sample Size and Power

Loading in 2 Seconds...

play fullscreen
1 / 64

Sample Size and Power - PowerPoint PPT Presentation


  • 93 Views
  • Uploaded on

Sample Size and Power. Steven R. Cummings, MD Director, S.F. Coordinating Center. The Secret of Long Life. Resveratrol In the skin of red grapes Makes mice Run faster Live longer. What I want to show. Consuming reservatrol prolongs healthy life. Sample Size Ingredients.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Sample Size and Power' - kuper


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
sample size and power

Sample Size and Power

Steven R. Cummings, MD

Director, S.F. Coordinating Center

the secret of long life
The Secret of Long Life
  • Resveratrol
  • In the skin of red grapes
  • Makes mice
    • Run faster
    • Live longer
what i want to show
What I want to show
  • Consuming reservatrol prolongs healthy life
sample size ingredients
Sample Size Ingredients
  • Testable hypothesis
  • Type of study
  • Statistical test
    • Type of variables
  • Effect size (and its variance)
  • Power and alpha
my research question
My research question
  • I need to plan the study
  • My question is

Does consuming reservatrol

lead to a long and healthy life?

what s wrong with the question
What’s wrong with the question?
  • I need to plan the study
  • My question is

Does consuming reservatrol

lead to a long and healthy life?

what s wrong with the question1
What’s wrong with the question?

Does consuming reservatrol

lead to a long and healthy life?

  • Vague
  • Must be measurable
consuming resveratrol
Consuming resveratrol
  • Most rigorous design: randomized placebo-controlled trial
  • Comparing red wine to placebo would be difficult
  • Resveratrol supplements available and widely used
measurable specific
Measurable (specific)
  • Consuming resevertrol = taking resveratrol supplements vs. taking placebo
  • Prolong healthy life = reduces all-cause mortality

Do people randomized to get a resveratrol supplement have a lower mortality rate than those who get a placebo?

in whom
In whom?

Do people randomized to get a resveratrol supplement have a lower mortality rate than those who get a placebo?

  • Must study a sample from the larger ‘target population’
  • What is the target population?
in whom1
In whom?
  • Elderly men and women (≥70 years)
the research hypothesis the alternative hypothesis
The research hypothesisThe ‘alternative’ hypothesis

Men and women > age 70 years randomized to get a resveratrol supplement have a lower mortality rate than those who get a placebo.

  • Cannot be tested statistically
  • Statistical tests can only reject null hypothesis - that there is no effect
the null hypothesis
The Null Hypothesis

Men and women > age 70 years randomized to receive a resveratrol supplement do not have lower mortality rate than those who receive placebo.

  • Can be rejected by statistical tests
ingredients for sample size
Ingredients for Sample Size

 Testable hypothesis

  • Type of study
  • Statistical test
    • Type of variables
  • Effect size (and its variance)
  • Power and alpha
type of study
Type of study
  • Descriptive
    • Only one variable / measurements
  • What proportion of centenarians take resveratrol supplements?
    • Confidence interval for proportions
  • What is the mean red wine intake of centenarians?
    • Confidence interval for the mean
sample size for a descriptive study
Sample size for a descriptive study
  • “What proportion of centenarians take resveratrol supplements?”
  • How much precision do you want?
    • Sample size is based on the width of the confidence interval (Table 6D and 6E)
  • For example, assume that 20% of centenarians take resveratrol
    • I want to be confident that the truth is within ±10%
type of study1
Type of study
  • Analytical: comparison
    • Cross-sectional
      • Mean red wine intake in centenarians vs. 60-80 years old
    • Randomized trial
      • Elders who get resveratrol have lower mortality than those who get placebo
ingredients for sample size1
Ingredients for Sample Size

 Testable hypothesis

 Type of study: analytical (RCT)

  • Statistical test
    • Type of variables
  • Effect size (and its variance)
  • Power and alpha
types of variables
Types of variables?
  • Dichotomous
    • Treatment or placebo
  • Continuous
    • Walking speed
the types of variables
The types of variables?

Men and women > age 70 years randomized to receive a resveratrol supplement do not have lower mortality rate than those who receive placebo

  • Dichotomous: reseveratrol or placebo
  • Dichotomous: mortality rate
    • 3-4% per year*; 3 year study: 10%
  • Statistical test: Chi-squaree (Tables 6B)

* ~ mean annual male @ 78 yrs

ingredients for sample size2
Ingredients for Sample Size

 Testable hypothesis

 Type of study: analytical (RCT)

 Statistical test

 Type of variables

  • Effect size (and its variance)
  • Power and alpha
effect size the hardest part
Effect sizethe hardest part

Considerations

  • What is likely, based on other data?
  • Pilot study
  • Estimates from biomarkers
  • What difference is important to detect?
    • “We don’t want to miss a ____ difference”
  • What can we afford to find?
the effect of resveratrol on mortality rate
The effect of resveratrol on mortality rate?
  • What is likely, based on other data?
  • Pilot study
  • Estimates from biomarkers
  • What difference is important to detect?
    • “We don’t want to miss a _1%_ difference”
  • What can we afford?
    • 1%: too expensive; 5%: cheap

* ~ mean annual male @ 78 yrs

the effect of resveratrol on mortality rate1
The effect of resveratrol on mortality rate?
  • Finding a smaller effect is important to health
  • Power to find a larger effect is important for your budget
  • Too small! vs. too large!
the science of effect sizes too large too small just right
The Science of Effect SizesToo large! Too small!Just right.
  • Smaller effect is important to health
  • Larger effect is important for your budget
effect size
Effect size

Men and women > age 70 years randomized to receive a resveratrol supplement do not have lower mortality rate than those who receive placebo

  • Placebo rate: 10%
  • Resveratrol rate: 8%
  • Chi-squared (Table 6B.2)

* ~ mean annual male @ 78 yrs

ingredients for sample size3
Ingredients for Sample Size

 Testable hypothesis

 Type of study: analytical (RCT)

 Statistical test

 Type of variables

 Effect size (and its variance)

  • Power and alpha
i will need to convince people
I will need to convince people
  • The result must be statistically significant

Customarily, P<0.05

AKA

  • Probability of a type I error (oops, we lied)
  • (alpha) = 0.05
i will need to convince skeptics
I will need to convince skeptics
  • Very small chance that we are fooling you

(alpha) = 0.01

P<0.01

  • Smaller means larger sample size
two sided vs one sided
Two-sided vs. one-sided 
  • Use 2-sided  if the result could go the opposite way you want
  • 1-sided reduces sample size somewhat
  • You may believe that your effect could only go one way!
    • Resveratrol could not increase mortality!
  • Be humble.
    • The history of research is filled with results that contradicted expectations
  • A 1-sided test is almost never the best choice
if it s true i don t want to miss it
If it’s true, I don’t want to miss it
  • The chance of missing the effect ()

customarily 20%

AKA

  • Type II error
  •  (beta): 0.20
  • Power = 1- 0.80
i really don t want to miss it
I really don’t want to miss it
  •  = .10
  • Power (1- ) = 0.90
  • Greater power means larger sample size
we have all of the ingredients
We have all of the ingredients

 Testable hypothesis

 Type of study: analytical (RCT)

 Statistical test: Chi-squared

 Effect size 10% vs 8%

 Power: 0.90; alpha: 0.20

from table 6b 2
From Table 6B.2
  • Sample size: 4,401
  • Per group
  • Total: 8,802
  • Does not include drop-outs
    • 20% drop-out: 11,002 total sample size
alternatives
Alternatives
  • Tweak : one-sided
    • Almost never appropriate
  • Tweak the power: 0.80
  • Modest effect: 3,308 (6,616 total)
alternatives1
Alternatives
  • Tweak and 
    • = 0.20
    • 3,308/group; 6,616 total
  • Also increase the effect size
    • 10% vs. 6%
alternatives2
Alternatives
  • Tweak and 
    • = 0.20
    • 3,308/group; 6,616 total
  • Also increase the effect size
    • 10% vs. 6%
    • 930 / group; 1,680 total
    • Big difference, still not affordable
    • Not believable
alternatives a new hypothesis
Alternatives: a new hypothesis
  • Change the outcome measure
    • Continuous measurement
  • A ‘surrogate’ for mortality rate
    • Strongly associated with mortality rate
    • Likely to be influenced by resveratrol
  • Walking speed
mice on resveratrol
Mice on resveratrol
  • Mice fed resveratrol
    • Live 25% longer
    • Are significantly faster
    • Have greater endurance
increased gait speed 0 1 m s in 1 year and survival over 8 years
Increased gait speed (0.1 m/s) in 1 year and survival over 8 years

Faster by ≥0.1 m/s

Slower

~20% decreased

mortality rate

what you need to know about a continuous variable
What you need to know about a continuous variable
  • Outcome: change in walking speed
  • Mean value in the population
  • Effect size
    • Change in walking speed
  • Variability in the change
what you need to know about a continuous variable1
What you need to know about a continuous variable
  • Outcome: change in walking speed
  • Mean value in the population = 1.0 m/sec
  • Effect size
    • Change in walking speed
    • 1.0 to 1.1 m/sec
  • Variability in the change
variability
Variability
  • No variability
    • Extremely reproducible
    • Relatively small sample size
  • Highly variable
    • Poor reproducibility
    • Relatively large sample size
  • Assessed by the Standard Deviation
variability1
Variability
  • Standard deviation for the measurement
    • Cross-sectional: 0.25 m / sec
  • However, we are interested in change
  • Standard deviation of change in speed?
what if you don t know the sd
What if you don’t know the SD?
  • Standard deviation of change in speed?
  • If you cannot find data from other studies for change over 3 years
  • Pilot study?
  • Well educated guess
estimating s d the 1 4 rule
Estimating S.D.the 1/4 rule
  • Range of changes
  • 0.2 m/sec to -0.6 m/sec
  • Range = 0.8 m/sec
  • 1/4 of the range = 0.2 m/sec
slide53
E/S
  • Effect size: 0.1 m/sec difference in change
  • Standard deviation: 0.2 m/sec
  • E/S = 0.5
the new ingredients
The new ingredients

 New testable hypothesis

 Type of study: analytical (RCT)

 Statistical test: t-test

  • Continuous variable
  • Difference between means

 Effect size 1.0 vs. 1.1 m/sec; E/S = 0.5

 Power: 0.90; alpha: 0.20

the new ingredients1
The new ingredients

 New testable hypothesis

 Type of study: analytical (RCT)

 T-test

 Effect size 1.0 vs.1.1 m/sec; E/S: 0.5

 Power: 0.90; alpha: 0.20

Sample size: 64 per group; 128 total

With 20% drop out: 160 total

summary
Summary
  • Estimate sample size early
  • Systematically collect the ingredients
  • Effect size is the most difficult - and important - judgement
  • Alternatives that reduce sample size
    • Compromise power
    • Increase effect size
    • Prefer precise continuous outcomes
supplementary slides

Supplementary slides

The effect of increasing the rate of events in the population while maintaining the same effect size

supplementary slides1
Supplementary slides
  • The effect of increasing the rate of events in the population
  • Maintaining the same effect size
alternatives3
Alternatives
  • Increase the event rate
  • Choose an older population with higher mortality
  • Enroll men ≥ 80 years old
    • 3-year mortality: 25%
  • Effect size: 20% reduction: 25% vs. 20%
alternatives4
Alternatives
  • Increase the event rate
  • Choose an older population with higher mortality
  • Enroll men ≥ 80 years old
    • 3-year mortality: 25%
  • Effect size: 20% reduction: 25% vs. 20%
  • 1,133 per group; 2,266 total
ad