Power Analysis in Grant Writing

1. Power Analysis in Grant Writing Jill Harkavy-Friedman, Ph.D.

2. Is there a difference?

4. Power is Knowledge

5. Power: How likely are you to detect an effect? • Sample Size: n How many people will you need? • Effect Size: σ (e.g., R2) How much of a difference are you trying to detect? • Significance Level (Type I Error): α How much of a risk are you willing to take of saying there is a difference when there none? • Type II Error: β or (1- α) • How much risk there is of saying that there is no difference when there is a difference.

6. Sample Size: n • Depends on Nature of the question & statistic approach • Group differences: m1-m2=0 • Correlations: r=0 • Regression: R2=0 • Feasibility • Economic, staffing, recruitment • Power needed

7. Effect Size: σ • Amount of difference want to detect • Based on previous literature: average SD • Based on pilot data: SD • Based on size of difference • Small: d=.20 • Medium: d=.50 • Large: d=.80 • Statistical difference ≠ Clinical difference

8. Significance Level (Type I Error): α Normal Curve and Distribution of Sample Means 10.0 7.5 Count 5.0 2.5 2.5% 11% 34% 34% 11% 2.5% 0.0 3 4 5 6

9. The larger the sample size the greater the power • The larger the effect size the greater the power • The larger the significance level the greater the power

11. Power and Sample Size Table at α 1=.05

13. Data Analysis comes first - power second • Determine your hypotheses • Determine your analyses • Determine the parameters for analysis by hypothesis (i.e., power, ES, α) • Conduct power analysis

14. Power analysis will require: • Type of analysis • Sample size • Effect size • Significance level • Number of groups or factors • Plug in the numbers

15. Considerations • What is your question? • What type of data do you have? • What are your hypotheses? • What are your resources? • Clinical vs. statistical significance • How will you present your data? • What you would like the news headline to be?

16. SAGES Research Committee, August 2006 RE: Suggestions to Assist with Completing a Winning Application • Power Analysis: • In order to minimize the reporting of false-negative data, a power analysis should be performed for sample size determination. Power is the capability of a study to detect a difference if the difference really exists. A type II error occurs when a true difference exists between study populations but there are insufficient numbers of subjects to detect this difference. • Any grant submitted without one of the items below will not be eligible for review.

17. Power analysis. Please provide the following data: alpha and beta, sample size needed in each group, what difference is expected. (Example: "A power analysis was performed with a beta of .20 and an alpha of 05. Assuming that a 10% difference exists between patient and control groups, 150 subjects will be needed in each arm. Thus the study would provide an 80% chance that a difference would be detected if one exists.") • If a power analysis in not appropriate for the submitted project, a statement should be included explaining why a power analysis is not appropriate for the study. • Consultation with a statistician is recommended. However, there are many statistical software programs available

18. What to do when you need more power • Increase sample size • Reduce number of variables • Show your data graphically

19. Power Analysis With n=400, alpha2 = .05, and a medium effect size (.30) the power will be >.99 for analyses of variance and .96 for zero order correlations120. The power for a regression analysis that includes 11 variables (i.e. sex, ethnicity, positive symptoms, negative symptoms, aggression, impulsivity, depression, premorbid adjustment, gene marker, family history and substance abuse) with n=400, alpha=.05 and a medium effect size (R2=.10) will be greater than .90. We do not anticipate that all 11 variables will contribute significantly to the model. With 6 variables, a more likely model, the power will be > .90. For the exploratory regression analyses conducted within the group of attempters (n=200) the power will be at least .80 to detect a medium effect size (R2=.10). For correlational studies among the genetic and biochemical measures (approximate n=100) the power will be .80 for a medium effect size (R2=.10) and .95 for a larger medium effect size (R2=.25). The increased sample size will now provide the power necessary to consider attempters and nonattempters separately. Analyses examining single attempters, multiple attempters and nonattempters (approximate sample sizes: 120, 80, 200) will still maintain adequate power (power>.70). Grant application by Jill M. Harkavy-Friedman, PhD

20. Free Power and Sample Size Calculation • Cohen J. Statistical Power Analysis for the Behavioral Sciences (2nd edition). Hillsdale, New Jersey: Lawrence Erlbaum Associates, Publishers, 1988 • http://biostat.mc.vanderbilt.edu/twiki/bin/view/Main/PowerSampleSize • G*Power 2 • http://www.psycho.uni-duesseldorf.de/aap/projects/gpower/ (limited) • G*3 • http://www.psycho.uni-duesseldorf.de/abteilungen/aap/gpower3/