- 48 Views
- Uploaded on
- Presentation posted in: General

Sample Size and Power

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Sample Size and Power

Steven R. Cummings, MD

Director, S.F. Coordinating Center

- Resveratrol
- In the skin of red grapes
- Makes mice
- Run faster
- Live longer

- Consuming reservatrol prolongs healthy life

- Testable hypothesis
- Type of study
- Statistical test
- Type of variables

- Effect size (and its variance)
- Power and alpha

- I need to plan the study
- My question is
Does consuming reservatrol

lead to a long and healthy life?

- I need to plan the study
- My question is
Does consuming reservatrol

lead to a long and healthy life?

Does consuming reservatrol

lead to a long and healthy life?

- Vague
- Must be measurable

- Most rigorous design: randomized placebo-controlled trial
- Comparing red wine to placebo would be difficult
- Resveratrol supplements available and widely used

- Consuming resevertrol =taking resveratrol supplements vs. taking placebo
- Prolong healthy life = reduces all-cause mortality
Do people randomized to get a resveratrol supplement have a lower mortality rate than those who get a placebo?

Do people randomized to get a resveratrol supplement have a lower mortality rate than those who get a placebo?

- Must study a sample from the larger ‘target population’
- What is the target population?

- Elderly men and women (≥70 years)

Men and women > age 70 years randomized to get a resveratrol supplement have a lower mortality rate than those who get a placebo.

- Cannot be tested statistically
- Statistical tests can only reject null hypothesis - that there is no effect

Men and women > age 70 years randomized to receive a resveratrol supplement do not have lower mortality rate than those who receive placebo.

- Can be rejected by statistical tests

Testable hypothesis

- Type of study
- Statistical test
- Type of variables

- Effect size (and its variance)
- Power and alpha

- Descriptive
- Only one variable / measurements

- What proportion of centenarians take resveratrol supplements?
- Confidence interval for proportions

- What is the mean red wine intake of centenarians?
- Confidence interval for the mean

- “What proportion of centenarians take resveratrol supplements?”
- How much precision do you want?
- Sample size is based on the width of the confidence interval (Table 6D and 6E)

- For example, assume that 20% of centenarians take resveratrol
- I want to be confident that the truth is within ±10%

- Analytical: comparison
- Cross-sectional
- Mean red wine intake in centenarians vs. 60-80 years old

- Randomized trial
- Elders who get resveratrol have lower mortality than those who get placebo

- Cross-sectional

Testable hypothesis

Type of study: analytical (RCT)

- Statistical test
- Type of variables

- Effect size (and its variance)
- Power and alpha

- Dichotomous
- Treatment or placebo

- Continuous
- Walking speed

Men and women > age 70 years randomized to receive a resveratrol supplement do not have lower mortality rate than those who receive placebo

- Dichotomous: reseveratrol or placebo
- Dichotomous: mortality rate
- 3-4% per year*; 3 year study: 10%

- Statistical test: Chi-squaree (Tables 6B)

* ~ mean annual male @ 78 yrs

Testable hypothesis

Type of study: analytical (RCT)

Statistical test

Type of variables

- Effect size (and its variance)
- Power and alpha

Considerations

- What is likely, based on other data?
- Pilot study
- Estimates from biomarkers
- What difference is important to detect?
- “We don’t want to miss a ____ difference”

- What can we afford to find?

~ 25%

Baur, Nature 2006

- What is likely, based on other data?
- Pilot study
- Estimates from biomarkers
- What difference is important to detect?
- “We don’t want to miss a _1%_ difference”

- What can we afford?
- 1%: too expensive; 5%: cheap

* ~ mean annual male @ 78 yrs

- Finding a smaller effect is important to health
- Power to find a larger effect is important for your budget
- Too small! vs. too large!

- Smaller effect is important to health
- Larger effect is important for your budget

Men and women > age 70 years randomized to receive a resveratrol supplement do not have lower mortality rate than those who receive placebo

- Placebo rate: 10%
- Resveratrol rate: 8%
- Chi-squared (Table 6B.2)

* ~ mean annual male @ 78 yrs

Testable hypothesis

Type of study: analytical (RCT)

Statistical test

Type of variables

Effect size (and its variance)

- Power and alpha

- The result must be statistically significant
Customarily, P<0.05

AKA

- Probability of a type I error (oops, we lied)
- (alpha) = 0.05

- Very small chance that we are fooling you
(alpha) = 0.01

P<0.01

- Smaller means larger sample size

- Use 2-sided if the result could go the opposite way you want
- 1-sided reduces sample size somewhat
- You may believe that your effect could only go one way!
- Resveratrol could not increase mortality!

- Be humble.
- The history of research is filled with results that contradicted expectations

- A 1-sided test is almost never the best choice

- The chance of missing the effect ()
customarily 20%

AKA

- Type II error
- (beta): 0.20
- Power = 1- 0.80

- = .10
- Power (1- ) = 0.90
- Greater power means larger sample size

Testable hypothesis

Type of study: analytical (RCT)

Statistical test: Chi-squared

Effect size 10% vs 8%

Power: 0.90; alpha: 0.20

- Sample size: 4,401
- Per group
- Total: 8,802
- Does not include drop-outs
- 20% drop-out: 11,002 total sample size

- Tweak : one-sided
- Almost never appropriate

- Tweak the power: 0.80
- Modest effect: 3,308 (6,616 total)

- Tweak and
- = 0.20
- 3,308/group; 6,616 total

- Also increase the effect size
- 10% vs. 6%

- Tweak and
- = 0.20
- 3,308/group; 6,616 total

- Also increase the effect size
- 10% vs. 6%
- 930 / group; 1,680 total
- Big difference, still not affordable
- Not believable

- Change the outcome measure
- Continuous measurement

- A ‘surrogate’ for mortality rate
- Strongly associated with mortality rate
- Likely to be influenced by resveratrol

- Walking speed

- Mice fed resveratrol
- Live 25% longer
- Are significantly faster
- Have greater endurance

Faster by ≥0.1 m/s

Slower

~20% decreased

mortality rate

- Outcome: change in walking speed
- Mean value in the population
- Effect size
- Change in walking speed

- Variability in the change

- Outcome: change in walking speed
- Mean value in the population = 1.0 m/sec
- Effect size
- Change in walking speed
- 1.0 to 1.1 m/sec

- Variability in the change

- No variability
- Extremely reproducible
- Relatively small sample size

- Highly variable
- Poor reproducibility
- Relatively large sample size

- Assessed by the Standard Deviation

- Standard deviation for the measurement
- Cross-sectional: 0.25 m / sec

- However, we are interested in change
- Standard deviation of change in speed?

- Standard deviation of change in speed?
- If you cannot find data from other studies for change over 3 years
- Pilot study?
- Well educated guess

- Range of changes
- 0.2 m/sec to -0.6 m/sec
- Range = 0.8 m/sec
- 1/4 of the range = 0.2 m/sec

- Effect size: 0.1 m/sec difference in change
- Standard deviation: 0.2 m/sec
- E/S = 0.5

New testable hypothesis

Type of study: analytical (RCT)

Statistical test: t-test

- Continuous variable
- Difference between means
Effect size 1.0 vs. 1.1 m/sec; E/S = 0.5

Power: 0.90; alpha: 0.20

New testable hypothesis

Type of study: analytical (RCT)

T-test

Effect size 1.0 vs.1.1 m/sec; E/S: 0.5

Power: 0.90; alpha: 0.20

Sample size: 64 per group; 128 total

With 20% drop out: 160 total

- Estimate sample size early
- Systematically collect the ingredients
- Effect size is the most difficult - and important - judgement
- Alternatives that reduce sample size
- Compromise power
- Increase effect size
- Prefer precise continuous outcomes

Supplementary slides

The effect of increasing the rate of events in the population while maintaining the same effect size

- The effect of increasing the rate of events in the population
- Maintaining the same effect size

- Increase the event rate
- Choose an older population with higher mortality
- Enroll men ≥ 80 years old
- 3-year mortality: 25%

- Effect size: 20% reduction: 25% vs. 20%

- Increase the event rate
- Choose an older population with higher mortality
- Enroll men ≥ 80 years old
- 3-year mortality: 25%

- Effect size: 20% reduction: 25% vs. 20%
- 1,133 per group; 2,266 total