1 / 100

Sample Size and Power

Sample Size and Power. Steven R. Cummings, MD Director, S.F. Coordinating Center. The Secret of Long Life. Resveratrol In the skin of red grapes Makes mice Run faster Live longer. The Secret of Long Life. Resveratrol In the skin of red grapes Makes mice Run faster Live longer.

gracie
Download Presentation

Sample Size and Power

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Sample Size and Power Steven R. Cummings, MD Director, S.F. Coordinating Center

  2. The Secret of Long Life • Resveratrol • In the skin of red grapes • Makes mice • Run faster • Live longer

  3. The Secret of Long Life • Resveratrol • In the skin of red grapes • Makes mice • Run faster • Live longer Mimics ‘sirtuin:’ senses energy and controls DNA transcription

  4. What I want to show • Consuming resveratrol prolongs healthy life

  5. Sample Size Ingredients • Testable hypothesis • Type of study • Statistical test • Type of variables • Effect size (and its variance) • Power and alpha

  6. Sample Size Ingredients • Testable hypothesis • Type of study • Statistical test • Type of variables • Effect size (and its variance) • Power and alpha

  7. My research question • I need to plan the study • My question is Does consuming resveratrol lead to a long and healthy life?

  8. What’s wrong with the question? • I need to plan the study • My question is Does consuming resveratrol lead to a long and healthy life?

  9. What’s wrong with the question? Does consuming resveratrol lead to a long and healthy life? • Vague • Must be measurable

  10. “Consuming resveratrol” • Most rigorous design: randomized placebo-controlled trial • Comparing red wine to placebo would be difficult • But resveratrol supplements are widely available

  11. Measurable (specific) outcome • “Consuming resveratrol” = taking resveratrol supplements vs. taking placebo • “Prolong healthy life” =

  12. Measurable (specific) outcome • “Consuming resevertrol” = taking resveratrol supplements vs. taking placebo • “Prolong healthy life” = reduces all-cause mortality Do people randomized to get a resveratrol supplement have a lower mortality rate than those who get a placebo?

  13. In whom? • Elderly men and women (≥70 years)

  14. The research hypothesis Men and women > age 70 years randomized to get a resveratrol supplement have a lower mortality rate than those who get a placebo.

  15. The research hypothesisThe ‘alternative’ hypothesis Men and women > age 70 years randomized to get a resveratrol supplement have a lower mortality rate than those who get a placebo. • Cannot be tested statistically • Statistical tests only reject null hypothesis - that there is no effect

  16. The Null Hypothesis Men and women > age 70 years randomized to receive a resveratrol supplement do not have lower mortality rate than those who receive placebo. • Can be rejected by statistical tests

  17. Ingredients for Sample Size  Testable hypothesis • Type of study • Statistical test • Type of variables • Effect size (and its variance) • Power and alpha

  18. Types of studies Approach to sample size is different for • Descriptive studies • Only one variable / measurements • Analytical studies • ‘Predictor’ and ‘outcome’ variables

  19. Types of studies Approach to sample size is different for • Descriptive studies • Only one variable / measurements • Analytical studies • ‘Predictor’ and ‘outcome’ variable • From the point of view of sample size estimates, cross-sectional, cohort studies and randomized trials look the same

  20. Descriptive studies • Descriptive • Only one variable / measurements For example: • What proportion of people who live to >100 years (centenarians) take resveratrol supplements? • What is the mean red wine intake of centenarians? • Sample size based on confidence intervals • Not covered in this lecture

  21. Analytical studies • Analytical means a comparison • Cross-sectional • Mean red wine intake in centenarians vs. 60-80 year olds

  22. Analytical studies • Analytical means a comparison • Cross-sectional • Mean red wine intake in centenarians vs. 60-80 year olds • Cohort study • Those who drink red wine have lower mortality rates than others • Randomized trial • Elders who get resveratrol have lower mortality than those who get placebo

  23. Ingredients for Sample Size  Testable hypothesis  Type of study: analytical (RCT) • Statistical test • Type of variables • Effect size (and its variance) • Power and alpha

  24. This works for most study planning Type of statistical testsDepends on the types of variables

  25. The types of variables? Men and women > age 70 years randomized to receive a resveratrol supplement do not have lower mortality rate than those who receive placebo • Dichotomous: resveratrol or placebo • Continuous: mortality rate

  26. The types of variables? Men and women > age 70 years randomized to receive a resveratrol supplement do not have lower mortality rate than those who receive placebo • Dichotomous: resveratrol or placebo • Continuous: mortality rate What’s wrong?

  27. The types of variables? Men and women > age 70 years randomized to receive a resveratrol supplement do not have lower mortality rate than those who receive placebo • Dichotomous: resveratrol or placebo • Continuous: mortality rate • It is a proportion at certain times • For example, 3% at 1 year

  28. The appropriate test for this randomized trial for mortality

  29. Ingredients for Sample Size  Testable hypothesis  Type of study: analytical (RCT)  Statistical test  Type of variables • Effect size (and its variance) • Power and alpha

  30. Estimating the effect size For randomized trials, • Estimate the expected rate in the placebo • For example, 10% • Specify the rate in the treatment group • For example, 5% (50% decrease) * ~ mean annual female/males @ 78 yrs

  31. Estimating the placebo rate • Best source: cohort studies of similar populations • Another source: data from the census • In this case, we know the mortality rates from our large cohort studies of aging: • 3-4% per year*; for a 3 year study: 10% * ~ mean annual female/males @ 78 yrs

  32. Effect sizethe hardest part What should I assume for the effect of resveratrol on mortality?

  33. Effect sizethe hardest part Ways to choose an effect size: • What is likely, based on other data? • Do a pilot study • Estimate based on effect on biomarkers • What difference is important to detect? • “We don’t want to miss a __%_ difference” • What can we afford?

  34. The effect of resveratrol on mortality rate? • What is likely, based on other data? • Do a pilot study • Estimate based on effect on biomarkers • What difference is important to detect? • “We don’t want to miss a __%_ difference” • What can we afford?

  35. Resveratrol pronged survival of mice fed high calorie diet ~ 25% Baur, Nature 2006

  36. The effect of resveratrol on mortality rate? • What is likely, based on other data? • Pilot study? What endpoint? • No reliable markers for the effect on death • What difference is important to detect? • “We don’t want to miss a ____ difference” • What can we afford to find?

  37. The effect of resveratrol on mortality rate? • What is likely, based on other data? • Do a pilot study • Estimate based on biomarkers • What difference is important to detect? • “We don’t want to miss a _1%_ difference” • What can we afford? • 1%: too big & expensive • 5%: small and cheap

  38. The effect of resveratrol on mortality rate? • Finding a smaller effect is important to health • Allowing a larger effect is important for your budget

  39. The Science of Effect Sizes:Too large! Too small!

  40. The Science of Effect SizesToo large! Too small!Just right. • Smaller effect is important to health • Larger effect is important for your budget

  41. The Science of Effect SizesToo large! Too small!Just right. It requires good judgment, balancing science and feasibility • Smaller effect is important to health • Larger effect is important for your budget

  42. Effect size Men and women > age 70 years randomized to receive a resveratrol supplement do not have lower mortality rate than those who receive placebo • It would be important to find (I don’t want to miss) a 20% decrease • Placebo rate: 10% • Resveratrol rate: 8%

  43. Ingredients for Sample Size  Testable hypothesis  Type of study: analytical (RCT)  Statistical test  Type of variables  Effect size (and its variance) • Power and alpha

  44. (alpha) The probability of finding a ‘significant’ result if nothing is going on

  45. I will need to convince people • Customarily, a result is ‘statistically significant’ if P<0.05 In other words, • Probability of a type I error = 5% • (alpha) = 0.05

  46. I will need to convince skeptics • Very small chance that a positive result is an error (alpha) = 0.01 P<0.01 • A smaller means larger sample size

  47. Two-sided vs. one-sided  • A 2-sided  assumes that the result could go either way • Recognizes that you have two chances of finding something that isn’t really there • Resveratrol decreases mortality • Resveratrol increases mortality • A 1-sided hypothesis reduces sample size (somewhat) • A one-sided  of 0.05 corresponds to a two-sided  of 0.10 • It assumes that the result could, plausibly, go only one way

  48. Two-sided vs. one-sided  • You may believe that your effect could only go one way! • Resveratrol is ‘natural.’ It could not increase mortality! • Be humble. • The history of research is filled with results that contradicted expectations • Vitamin D trial (JAMA 2010): • To everyone’s surprise, ~1500 IU of vitamin D/d increased the risk of falls and fractures in elderly women and men • A 1-sided test is almost never the best choice

  49. Two-sided vs. one-sided  • You may believe that your effect could only go one way! • Resveratrol is ‘natural.’ It could not increase mortality!

  50. Two-sided vs. one-sided  • You may believe that your effect could only go one way! • Resveratrol is ‘natural.’ It could not increase mortality! • Be humble. • The history of research is filled with results that contradicted expectations • Vitamin D trial (JAMA 2010): • To everyone’s surprise, ~1500 IU of vitamin D/d increased the risk of falls and fractures in elderly women and men • A 1-sided test is almost never the best choice

More Related