Sample size and power
This presentation is the property of its rightful owner.
Sponsored Links
1 / 64

Sample Size and Power PowerPoint PPT Presentation


  • 46 Views
  • Uploaded on
  • Presentation posted in: General

Sample Size and Power. Steven R. Cummings, MD Director, S.F. Coordinating Center. The Secret of Long Life. Resveratrol In the skin of red grapes Makes mice Run faster Live longer. What I want to show. Consuming reservatrol prolongs healthy life. Sample Size Ingredients.

Download Presentation

Sample Size and Power

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Sample size and power

Sample Size and Power

Steven R. Cummings, MD

Director, S.F. Coordinating Center


The secret of long life

The Secret of Long Life

  • Resveratrol

  • In the skin of red grapes

  • Makes mice

    • Run faster

    • Live longer


What i want to show

What I want to show

  • Consuming reservatrol prolongs healthy life


Sample size ingredients

Sample Size Ingredients

  • Testable hypothesis

  • Type of study

  • Statistical test

    • Type of variables

  • Effect size (and its variance)

  • Power and alpha


My research question

My research question

  • I need to plan the study

  • My question is

    Does consuming reservatrol

    lead to a long and healthy life?


What s wrong with the question

What’s wrong with the question?

  • I need to plan the study

  • My question is

    Does consuming reservatrol

    lead to a long and healthy life?


What s wrong with the question1

What’s wrong with the question?

Does consuming reservatrol

lead to a long and healthy life?

  • Vague

  • Must be measurable


Consuming resveratrol

Consuming resveratrol

  • Most rigorous design: randomized placebo-controlled trial

  • Comparing red wine to placebo would be difficult

  • Resveratrol supplements available and widely used


Measurable specific

Measurable (specific)

  • Consuming resevertrol =taking resveratrol supplements vs. taking placebo

  • Prolong healthy life = reduces all-cause mortality

    Do people randomized to get a resveratrol supplement have a lower mortality rate than those who get a placebo?


In whom

In whom?

Do people randomized to get a resveratrol supplement have a lower mortality rate than those who get a placebo?

  • Must study a sample from the larger ‘target population’

  • What is the target population?


In whom1

In whom?

  • Elderly men and women (≥70 years)


The research hypothesis the alternative hypothesis

The research hypothesisThe ‘alternative’ hypothesis

Men and women > age 70 years randomized to get a resveratrol supplement have a lower mortality rate than those who get a placebo.

  • Cannot be tested statistically

  • Statistical tests can only reject null hypothesis - that there is no effect


The null hypothesis

The Null Hypothesis

Men and women > age 70 years randomized to receive a resveratrol supplement do not have lower mortality rate than those who receive placebo.

  • Can be rejected by statistical tests


Ingredients for sample size

Ingredients for Sample Size

 Testable hypothesis

  • Type of study

  • Statistical test

    • Type of variables

  • Effect size (and its variance)

  • Power and alpha


Type of study

Type of study

  • Descriptive

    • Only one variable / measurements

  • What proportion of centenarians take resveratrol supplements?

    • Confidence interval for proportions

  • What is the mean red wine intake of centenarians?

    • Confidence interval for the mean


Sample size for a descriptive study

Sample size for a descriptive study

  • “What proportion of centenarians take resveratrol supplements?”

  • How much precision do you want?

    • Sample size is based on the width of the confidence interval (Table 6D and 6E)

  • For example, assume that 20% of centenarians take resveratrol

    • I want to be confident that the truth is within ±10%


Type of study1

Type of study

  • Analytical: comparison

    • Cross-sectional

      • Mean red wine intake in centenarians vs. 60-80 years old

    • Randomized trial

      • Elders who get resveratrol have lower mortality than those who get placebo


Ingredients for sample size1

Ingredients for Sample Size

 Testable hypothesis

 Type of study: analytical (RCT)

  • Statistical test

    • Type of variables

  • Effect size (and its variance)

  • Power and alpha


Type of statistical tests depends on the types of variables

Type of statistical testsDepends on the types of variables


Types of variables

Types of variables?

  • Dichotomous

    • Treatment or placebo

  • Continuous

    • Walking speed


The types of variables

The types of variables?

Men and women > age 70 years randomized to receive a resveratrol supplement do not have lower mortality rate than those who receive placebo

  • Dichotomous: reseveratrol or placebo

  • Dichotomous: mortality rate

    • 3-4% per year*; 3 year study: 10%

  • Statistical test: Chi-squaree (Tables 6B)

* ~ mean annual male @ 78 yrs


Ingredients for sample size2

Ingredients for Sample Size

 Testable hypothesis

 Type of study: analytical (RCT)

 Statistical test

 Type of variables

  • Effect size (and its variance)

  • Power and alpha


Effect size the hardest part

Effect sizethe hardest part

Considerations

  • What is likely, based on other data?

  • Pilot study

  • Estimates from biomarkers

  • What difference is important to detect?

    • “We don’t want to miss a ____ difference”

  • What can we afford to find?


Resveratrol pronged survival of mice fed high calorie diet

Resveratrol pronged survival of mice fed high calorie diet

~ 25%

Baur, Nature 2006


The effect of resveratrol on mortality rate

The effect of resveratrol on mortality rate?

  • What is likely, based on other data?

  • Pilot study

  • Estimates from biomarkers

  • What difference is important to detect?

    • “We don’t want to miss a _1%_ difference”

  • What can we afford?

    • 1%: too expensive; 5%: cheap

* ~ mean annual male @ 78 yrs


The effect of resveratrol on mortality rate1

The effect of resveratrol on mortality rate?

  • Finding a smaller effect is important to health

  • Power to find a larger effect is important for your budget

  • Too small! vs. too large!


The science of effect sizes too large too small just right

The Science of Effect SizesToo large! Too small!Just right.

  • Smaller effect is important to health

  • Larger effect is important for your budget


Effect size

Effect size

Men and women > age 70 years randomized to receive a resveratrol supplement do not have lower mortality rate than those who receive placebo

  • Placebo rate: 10%

  • Resveratrol rate: 8%

  • Chi-squared (Table 6B.2)

* ~ mean annual male @ 78 yrs


Ingredients for sample size3

Ingredients for Sample Size

 Testable hypothesis

 Type of study: analytical (RCT)

 Statistical test

 Type of variables

 Effect size (and its variance)

  • Power and alpha


I will need to convince people

I will need to convince people

  • The result must be statistically significant

    Customarily, P<0.05

    AKA

  • Probability of a type I error (oops, we lied)

  • (alpha) = 0.05


I will need to convince skeptics

I will need to convince skeptics

  • Very small chance that we are fooling you

    (alpha) = 0.01

    P<0.01

  • Smaller means larger sample size


Two sided vs one sided

Two-sided vs. one-sided 

  • Use 2-sided  if the result could go the opposite way you want

  • 1-sided reduces sample size somewhat

  • You may believe that your effect could only go one way!

    • Resveratrol could not increase mortality!

  • Be humble.

    • The history of research is filled with results that contradicted expectations

  • A 1-sided test is almost never the best choice


If it s true i don t want to miss it

If it’s true, I don’t want to miss it

  • The chance of missing the effect ()

    customarily 20%

    AKA

  • Type II error

  •  (beta): 0.20

  • Power = 1- 0.80


I really don t want to miss it

I really don’t want to miss it

  •  = .10

  • Power (1- ) = 0.90

  • Greater power means larger sample size


We have all of the ingredients

We have all of the ingredients

 Testable hypothesis

 Type of study: analytical (RCT)

 Statistical test: Chi-squared

 Effect size 10% vs 8%

 Power: 0.90; alpha: 0.20


From table 6b 2 comparing two proportions

From Table 6B.2Comparing two proportions


From table 6b 2

From Table 6B.2

  • Sample size: 4,401

  • Per group

  • Total: 8,802

  • Does not include drop-outs

    • 20% drop-out: 11,002 total sample size


Alternatives

Alternatives

  • Tweak : one-sided

    • Almost never appropriate

  • Tweak the power: 0.80

  • Modest effect: 3,308 (6,616 total)


From table 6b 2 comparing two proportions1

From Table 6B.2Comparing two proportions


Alternatives1

Alternatives

  • Tweak and 

    • = 0.20

    • 3,308/group; 6,616 total

  • Also increase the effect size

    • 10% vs. 6%


From table 6b 2 comparing two proportions2

From Table 6B.2Comparing two proportions


Alternatives2

Alternatives

  • Tweak and 

    • = 0.20

    • 3,308/group; 6,616 total

  • Also increase the effect size

    • 10% vs. 6%

    • 930 / group; 1,680 total

    • Big difference, still not affordable

    • Not believable


Alternatives a new hypothesis

Alternatives: a new hypothesis

  • Change the outcome measure

    • Continuous measurement

  • A ‘surrogate’ for mortality rate

    • Strongly associated with mortality rate

    • Likely to be influenced by resveratrol

  • Walking speed


Mice on resveratrol

Mice on resveratrol

  • Mice fed resveratrol

    • Live 25% longer

    • Are significantly faster

    • Have greater endurance


Increased gait speed 0 1 m s in 1 year and survival over 8 years

Increased gait speed (0.1 m/s) in 1 year and survival over 8 years

Faster by ≥0.1 m/s

Slower

~20% decreased

mortality rate


What you need to know about a continuous variable

What you need to know about a continuous variable

  • Outcome: change in walking speed

  • Mean value in the population

  • Effect size

    • Change in walking speed

  • Variability in the change


What you need to know about a continuous variable1

What you need to know about a continuous variable

  • Outcome: change in walking speed

  • Mean value in the population = 1.0 m/sec

  • Effect size

    • Change in walking speed

    • 1.0 to 1.1 m/sec

  • Variability in the change


Variability

Variability

  • No variability

    • Extremely reproducible

    • Relatively small sample size

  • Highly variable

    • Poor reproducibility

    • Relatively large sample size

  • Assessed by the Standard Deviation


Variability1

Variability

  • Standard deviation for the measurement

    • Cross-sectional: 0.25 m / sec

  • However, we are interested in change

  • Standard deviation of change in speed?


What if you don t know the sd

What if you don’t know the SD?

  • Standard deviation of change in speed?

  • If you cannot find data from other studies for change over 3 years

  • Pilot study?

  • Well educated guess


Estimating s d the 1 4 rule

Estimating S.D.the 1/4 rule

  • Range of changes

  • 0.2 m/sec to -0.6 m/sec

  • Range = 0.8 m/sec

  • 1/4 of the range = 0.2 m/sec


Sample size and power 2625841

E/S

  • Effect size: 0.1 m/sec difference in change

  • Standard deviation: 0.2 m/sec

  • E/S = 0.5


The new ingredients

The new ingredients

 New testable hypothesis

 Type of study: analytical (RCT)

 Statistical test: t-test

  • Continuous variable

  • Difference between means

     Effect size 1.0 vs. 1.1 m/sec; E/S = 0.5

     Power: 0.90; alpha: 0.20


The new ingredients1

The new ingredients

 New testable hypothesis

 Type of study: analytical (RCT)

 T-test

 Effect size 1.0 vs.1.1 m/sec; E/S: 0.5

 Power: 0.90; alpha: 0.20

Sample size: 64 per group; 128 total

With 20% drop out: 160 total


Summary

Summary

  • Estimate sample size early

  • Systematically collect the ingredients

  • Effect size is the most difficult - and important - judgement

  • Alternatives that reduce sample size

    • Compromise power

    • Increase effect size

    • Prefer precise continuous outcomes


Supplementary slides

Supplementary slides

The effect of increasing the rate of events in the population while maintaining the same effect size


Supplementary slides1

Supplementary slides

  • The effect of increasing the rate of events in the population

  • Maintaining the same effect size


Alternatives3

Alternatives

  • Increase the event rate

  • Choose an older population with higher mortality

  • Enroll men ≥ 80 years old

    • 3-year mortality: 25%

  • Effect size: 20% reduction: 25% vs. 20%


Alternatives4

Alternatives

  • Increase the event rate

  • Choose an older population with higher mortality

  • Enroll men ≥ 80 years old

    • 3-year mortality: 25%

  • Effect size: 20% reduction: 25% vs. 20%

  • 1,133 per group; 2,266 total


  • Login