Chocolate Cake Seminar Series on Statistical Applications

1. Chocolate Cake SeminarSeries on Statistical Applications Today’s Talk: Get Empowered by Power Analysis! By William Guo

2. STATISTICAL POWER ANALYSIS

3. Outline of Presentation Examples Concept and Explanation Factors that Influence Power Application of Power Analysis Formula and Result of the Example Software and Calculation

4. Example 1 – EducationANOVA We wish to conduct a study in the area of mathematics education involving different teaching methods to improve standardized math scores in local classrooms. The study will include three different teaching methods and use fourth grade students who are randomly sampled and assigned to the three different teaching methods. The three different teaching methods are: • The traditional teaching method where the teacher explains the concepts and assigns homework problems from the textbook. • The intensive practice method, in which students fill out additional work sheets both before and after school. • The computer assisted method, in which students learn math concepts and skills from using various computer based math learning programs. Students will stay in groups for an entire academic year. Then all students will take a standardized test that has a mean of 550 with a standard deviation of 80. Hypotheses: H0: μ1= μ2= μ3vs. H1: at least one μi, i=1, 2, 3 is different Question: how many students will be needed in each group?

5. Example 2 - ChemistryLinear Regression Model Experience with a certain type of plastic indicates that a relation exists between the hardness (measured in Brinell units) of items molded from the plastic (Y) and the elapsed time since termination of the molding process (X). It is proposed to study this relation by means of linear regression analysis. The regression model is: Yi=β0+β1xi+εi , i=1, 2, …, n Hypotheses: H0 : β1= 0 vs. H1: β1≠ 0 Question: how many experiments do we need for this research?

6. Example 3 - Clinical StatisticsTwo Groups t-test A clinical trial is conducted for a new drug to test its efficacy in lowering blood pressure in patients suffering from hypertension. The control subjects receive a marketed drug. The investigators specify the endpoint as the percentage reduction in diastolic blood pressure (the pressure in the blood vessels while the heart is relaxing). μtr: the true mean percentage reduction for the treatment group μc : the true mean percentage reduction for the control group Hypotheses: H0: μtr= μcvs. H1: μtr> μc The one-sided alternative is taken because researchers are confident that the tested drug cannot do worse than the marketed one. Question: How many people we need for each group?

7. Hypothesis Testing • H0: μ=μ0 H1: μ≠μ0 • H0: μ=μ0 H1: μ<μ0 • H0: μ=μ0 H1: μ>μ0 Null Hypothesis vs. Alternative Hypothesis Decision Rule: If P_value < α, reject the H0, conclude H1 Otherwise, fail to reject H0

8. Type I Error & Type II Error True State of Nature H0 is true H1 is true Fail to reject H0 reject H0 Our Decision Based on data

9. Definition Statistical power: • The power of a statistical test is the probability that the test will reject the null hypothesis when the null hypothesis is false. • i.e. the probability of not committing a Type II error, if the alternative hypothesis is true. • As the power increases, the chances of a Type II error occurring decrease. The probability of a Type II error occurring is β. And power is equal to 1 − β.

10. Factors That Influence Power – 1(Normal Distribution) • Significance level α

11. Factors That Influence Power – 2(Normal Distribution) • Variance σ2

12. Factors That Influence Power – 3 (Normal Distribution) • Difference of μtrand μc

13. Factors That Influence Power - Summary(All Distributions) • Significance level α • Variance σ2 • Difference of μtrand μc • Variance of mean is estimated by s2/n, and n is the sample size. • Power Analysis focuses on the relationship among sample size, statistical power, significance level, and effect size (for two groups t-test, effect size is).

14. Application of Power Analysis • Planning: Designing research studies - primary method to calculate the minimum sample size. • Diagnosis: • 1. Determining whether a specific study has adequate powerfor specific purposes. • 2. Calculating the minimum effect size that is likely to be detected in a study using a given sample size.

15. Power Analysis - Procedure • Set up hypotheses (For t-test: one-side or two-side test). • Determining level of significance α, which is usually set to 0.05 or 0.01 by convention. • The sensitivity of the study is decided by making a conservative estimate of treatment effect (it is the difference of μtrand μc, also called minimum detectable value). • Estimating population standard deviation σ. • Power should be at least 0.5. Most used values are 0.75 and 0.8. • At last, computing the sample size. Effect size

16. Calculating the Minimum Sample Size • Determining level of significance α = 0.05. • The minimum detectable difference δ= μtr– μ = 5. • The probability of type II error β = 0.25, i.e. statistical power is 0.75. • The historical data suggest that σ= 15. • The objective in this example is to find the value of n, the required group size in the clinical trial.

17. Result for Example 3z-test • Formula: n= 2(σ/δ)2(ϕ-1(1-α)-ϕ-1(β))2 In fact, n is taken as the smallest integer exceeding this value. • Plugging α= 0.05, β= 0.25, σ= 15, δ= 5. • We obtain n ≥96.83; that is, n = 97 per group is needed. • In fact, when n=97, Power is 0.751. • Note: Most software use t-test instead of z-test.

18. Some Results for Example 3 • When Power (1-β) is changing: α = 0.05, σ= 15, δ = 5 • When δ is changing: α = 0.05, β = 0.25, σ = 15

19. Software and Calculation • G*Power It is a free software. Just Google “G*Power” or go to website: www.psycho.uni-duesseldorf.de/aap/projects/gpower/ • SAS • SPSS “Sample Power”

20. G*Power – t-test • Tab -> Protocol of power analyses • Stat. test -> Means: difference between… (two groups) • Tails: One • Effect size d: 0.333 • α = 0.05, Power = 0.75

21. G*Power – ANOVA

22. G*Power – Regression

23. Thank you ! ANALYSIS POWER