1 / 64

Sample Size and Power

Sample Size and Power. Steven R. Cummings, MD Director, S.F. Coordinating Center. The Secret of Long Life. Resveratrol In the skin of red grapes Makes mice Run faster Live longer. What I want to show. Consuming reservatrol prolongs healthy life. Sample Size Ingredients.

kuper
Download Presentation

Sample Size and Power

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Sample Size and Power Steven R. Cummings, MD Director, S.F. Coordinating Center

  2. The Secret of Long Life • Resveratrol • In the skin of red grapes • Makes mice • Run faster • Live longer

  3. What I want to show • Consuming reservatrol prolongs healthy life

  4. Sample Size Ingredients • Testable hypothesis • Type of study • Statistical test • Type of variables • Effect size (and its variance) • Power and alpha

  5. My research question • I need to plan the study • My question is Does consuming reservatrol lead to a long and healthy life?

  6. What’s wrong with the question? • I need to plan the study • My question is Does consuming reservatrol lead to a long and healthy life?

  7. What’s wrong with the question? Does consuming reservatrol lead to a long and healthy life? • Vague • Must be measurable

  8. Consuming resveratrol • Most rigorous design: randomized placebo-controlled trial • Comparing red wine to placebo would be difficult • Resveratrol supplements available and widely used

  9. Measurable (specific) • Consuming resevertrol = taking resveratrol supplements vs. taking placebo • Prolong healthy life = reduces all-cause mortality Do people randomized to get a resveratrol supplement have a lower mortality rate than those who get a placebo?

  10. In whom? Do people randomized to get a resveratrol supplement have a lower mortality rate than those who get a placebo? • Must study a sample from the larger ‘target population’ • What is the target population?

  11. In whom? • Elderly men and women (≥70 years)

  12. The research hypothesisThe ‘alternative’ hypothesis Men and women > age 70 years randomized to get a resveratrol supplement have a lower mortality rate than those who get a placebo. • Cannot be tested statistically • Statistical tests can only reject null hypothesis - that there is no effect

  13. The Null Hypothesis Men and women > age 70 years randomized to receive a resveratrol supplement do not have lower mortality rate than those who receive placebo. • Can be rejected by statistical tests

  14. Ingredients for Sample Size  Testable hypothesis • Type of study • Statistical test • Type of variables • Effect size (and its variance) • Power and alpha

  15. Type of study • Descriptive • Only one variable / measurements • What proportion of centenarians take resveratrol supplements? • Confidence interval for proportions • What is the mean red wine intake of centenarians? • Confidence interval for the mean

  16. Sample size for a descriptive study • “What proportion of centenarians take resveratrol supplements?” • How much precision do you want? • Sample size is based on the width of the confidence interval (Table 6D and 6E) • For example, assume that 20% of centenarians take resveratrol • I want to be confident that the truth is within ±10%

  17. Type of study • Analytical: comparison • Cross-sectional • Mean red wine intake in centenarians vs. 60-80 years old • Randomized trial • Elders who get resveratrol have lower mortality than those who get placebo

  18. Ingredients for Sample Size  Testable hypothesis  Type of study: analytical (RCT) • Statistical test • Type of variables • Effect size (and its variance) • Power and alpha

  19. Type of statistical testsDepends on the types of variables

  20. Types of variables? • Dichotomous • Treatment or placebo • Continuous • Walking speed

  21. The types of variables? Men and women > age 70 years randomized to receive a resveratrol supplement do not have lower mortality rate than those who receive placebo • Dichotomous: reseveratrol or placebo • Dichotomous: mortality rate • 3-4% per year*; 3 year study: 10% • Statistical test: Chi-squaree (Tables 6B) * ~ mean annual male @ 78 yrs

  22. Ingredients for Sample Size  Testable hypothesis  Type of study: analytical (RCT)  Statistical test  Type of variables • Effect size (and its variance) • Power and alpha

  23. Effect sizethe hardest part Considerations • What is likely, based on other data? • Pilot study • Estimates from biomarkers • What difference is important to detect? • “We don’t want to miss a ____ difference” • What can we afford to find?

  24. Resveratrol pronged survival of mice fed high calorie diet ~ 25% Baur, Nature 2006

  25. The effect of resveratrol on mortality rate? • What is likely, based on other data? • Pilot study • Estimates from biomarkers • What difference is important to detect? • “We don’t want to miss a _1%_ difference” • What can we afford? • 1%: too expensive; 5%: cheap * ~ mean annual male @ 78 yrs

  26. The effect of resveratrol on mortality rate? • Finding a smaller effect is important to health • Power to find a larger effect is important for your budget • Too small! vs. too large!

  27. The Science of Effect SizesToo large! Too small!Just right. • Smaller effect is important to health • Larger effect is important for your budget

  28. Effect size Men and women > age 70 years randomized to receive a resveratrol supplement do not have lower mortality rate than those who receive placebo • Placebo rate: 10% • Resveratrol rate: 8% • Chi-squared (Table 6B.2) * ~ mean annual male @ 78 yrs

  29. Ingredients for Sample Size  Testable hypothesis  Type of study: analytical (RCT)  Statistical test  Type of variables  Effect size (and its variance) • Power and alpha

  30. I will need to convince people • The result must be statistically significant Customarily, P<0.05 AKA • Probability of a type I error (oops, we lied) • (alpha) = 0.05

  31. I will need to convince skeptics • Very small chance that we are fooling you (alpha) = 0.01 P<0.01 • Smaller means larger sample size

  32. Two-sided vs. one-sided  • Use 2-sided  if the result could go the opposite way you want • 1-sided reduces sample size somewhat • You may believe that your effect could only go one way! • Resveratrol could not increase mortality! • Be humble. • The history of research is filled with results that contradicted expectations • A 1-sided test is almost never the best choice

  33. If it’s true, I don’t want to miss it • The chance of missing the effect () customarily 20% AKA • Type II error •  (beta): 0.20 • Power = 1- 0.80

  34. I really don’t want to miss it •  = .10 • Power (1- ) = 0.90 • Greater power means larger sample size

  35. We have all of the ingredients  Testable hypothesis  Type of study: analytical (RCT)  Statistical test: Chi-squared  Effect size 10% vs 8%  Power: 0.90; alpha: 0.20

  36. From Table 6B.2Comparing two proportions

  37. From Table 6B.2 • Sample size: 4,401 • Per group • Total: 8,802 • Does not include drop-outs • 20% drop-out: 11,002 total sample size

  38. Alternatives • Tweak : one-sided • Almost never appropriate • Tweak the power: 0.80 • Modest effect: 3,308 (6,616 total)

  39. From Table 6B.2Comparing two proportions

  40. Alternatives • Tweak and  • = 0.20 • 3,308/group; 6,616 total • Also increase the effect size • 10% vs. 6%

  41. From Table 6B.2Comparing two proportions

  42. Alternatives • Tweak and  • = 0.20 • 3,308/group; 6,616 total • Also increase the effect size • 10% vs. 6% • 930 / group; 1,680 total • Big difference, still not affordable • Not believable

  43. Alternatives: a new hypothesis • Change the outcome measure • Continuous measurement • A ‘surrogate’ for mortality rate • Strongly associated with mortality rate • Likely to be influenced by resveratrol • Walking speed

  44. Mice on resveratrol • Mice fed resveratrol • Live 25% longer • Are significantly faster • Have greater endurance

  45. Increased gait speed (0.1 m/s) in 1 year and survival over 8 years Faster by ≥0.1 m/s Slower ~20% decreased mortality rate

  46. What you need to know about a continuous variable • Outcome: change in walking speed • Mean value in the population • Effect size • Change in walking speed • Variability in the change

  47. What you need to know about a continuous variable • Outcome: change in walking speed • Mean value in the population = 1.0 m/sec • Effect size • Change in walking speed • 1.0 to 1.1 m/sec • Variability in the change

  48. Variability • No variability • Extremely reproducible • Relatively small sample size • Highly variable • Poor reproducibility • Relatively large sample size • Assessed by the Standard Deviation

  49. Variability • Standard deviation for the measurement • Cross-sectional: 0.25 m / sec • However, we are interested in change • Standard deviation of change in speed?

More Related