1 / 33

Power and Sample Size

Power and Sample Size. Boulder 2004 Benjamin Neale Shaun Purcell. I HAVE THE POWER!!!. To be accomplished. Introduce power via e xample Identify factors relevant to power Practical: Empirical power analysis of ACE twin model How to use Mx for power. Simple example.

alize
Download Presentation

Power and Sample Size

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Power and Sample Size Boulder 2004 Benjamin Neale Shaun Purcell I HAVE THE POWER!!!

  2. To be accomplished • Introduce power via example • Identify factors relevant to power • Practical: • Empirical power analysis of ACE twin model • How to use Mx for power

  3. Simple example • Investigate the linear relationship (r) • between two random variables X and Y: • We are testingr=0 vs.r0

  4. How to investigate r • Draw a sample, measure X,Y • Calculate the measure of association r (Pearson product moment corr. coeff.) • Test whether r 0.

  5. How to Test r 0 • Assumed data are normally distributed • Define a null-hypothesis (r = 0) • Chose a level (usually .05) • Assume (null) distribution of the test statistic (t) associated with r=0 • t=r [(N-2)/(1-r2)]

  6. How to Test r 0 • Sample N=40 • r=.303, t=1.867, df=38, p=.06 α=.05 • As p > α, we fail to reject r= 0 • Have we drawn the correct conclusion?

  7. A note on a • Type I error rate = a • probability of deciding r  0 (while in truth r=0) • Usually a is .05...why? DOGMA

  8. N=40, r=0, nrep=1000 – central t(38), a=0.05 (critical value 2.04)

  9. Observed non-null distribution (r=.2) and null distribution

  10. In 23% of tests of r=0, |t|>2.024 (a=0.05), and thus draw the correct conclusion that of rejecting r =0. The probability of rejecting the null-hypothesis (r=0) correctly is 1-b, or the power, when a true effect exists

  11. Hypothesis Testing • Correlation Coefficient hypotheses: • ho (null hypothesis) is ρ=0 • ha (alternative hypothesis) is ρ≠ 0 • Two-sided test, where ρ > 0 or ρ < 0 are one-sided • Null hypothesis usually assumes no effect • Alternative hypothesis is the idea being tested

  12. Summary of Possible Results H-0 true H-0 false accept H-0 1-ab reject H-0 a 1-b a=type 1 error rate b=type 2 error rate 1-b=statistical power

  13. Type I error at rate  Nonsignificant result (1- ) Type II error at rate  Significant result (1-) STATISTICS Non-rejection of H0 Rejection of H0 H0 true R E A L I T Y HA true

  14. Power • The probability of rejection of a false null-hypothesis depends on: • the significance criterion () • the sample size (N) • the effect size (NCP) “The probability of detecting a given effect size in a population from a sample of size N, using significance criterion ”

  15. Standard Case Sampling distribution if HA were true Sampling distribution if H0 were true P(T) alpha 0.05 POWER = 1 -    T Effect Size (NCP)

  16. Impact of Less Cons. alpha Sampling distribution if HA were true Sampling distribution if H0 were true P(T) alpha 0.1 POWER = 1 -    T 

  17. Impact of More Cons. alpha Sampling distribution if HA were true Sampling distribution if H0 were true P(T) alpha 0.01 POWER = 1 -   T 

  18. Increase in Sample Size Sampling distribution if HA were true Sampling distribution if H0 were true P(T) alpha 0.05 POWER = 1 -    T Effect Size (NCP)↑

  19. Increase in Effect Size Sampling distribution if HA were true Sampling distribution if H0 were true P(T) alpha 0.05 POWER = 1 -    T Effect Size (NCP)↑

  20. Effects on Power Recap • Larger Effect Size • Larger Sample Size • Alpha Level shifts <Beware the False Positive!!!> • Type of Data: • Binary, Ordinal, Continuous

  21. When To Do Power Calcs? • Generally study planning stages of study • Occasionally with negative result • No need if significance is achieved • Computed to determine chances of success

  22. Power Calculations Empirical • Attempt to Grasp the NCP from Null • Simulate Data under theorized model • Calculate Statistics and Perform Test • Given α, how many tests p < α • Power = (#hits)/(#tests)

  23. Practical: Empirical Power 1 • We will Simulate Data under a model online • We will run an ACE model, and test for C • We will then submit our data and Shaun will collate it for us • While he’s collating, we’ll talk about theoretical power calculations

  24. Practical: Empirical Power 2 • First get F:\ben\2004\ace.mx and put it into your directory • We will paste our simulated data into this script, so open it now in preparation, and note both places where we must paste in the data • Note that you will have to fit the ACE model and then fit the AE submodel

  25. Practical: Empirical Power 3 • Simulation Conditions • 30% A2 20% C2 50% E2 • Input: • A 0.5477 C of 0.4472 E of 0.7071 • 350 MZ 350 DZ • Simulate and Space Delimited at • http://statgen.iop.kcl.ac.uk/workshop/unisim.html or click here in slide show mode • Click submit after filling in the fields and you will get a page of data

  26. Practical: Empirical Power 4 • With the data page, use control-a to select the data, control-c to copy, and in Mx control-v to paste in both the MZ and DZ groups. • Run the ace.mx script with the data pasted in and modify it to run the AE model. • Report the A, C, and E estimates of the first model, and the A and E estimates of the second model as well as both the -2log-likelihoods on the webpage http://statgen.iop.kcl.ac.uk/workshop/ or click here in slide show mode

  27. Practical: Empirical Power 5 • Once all of you have submitted your results we will take a look at the theoretical power calculation, using Mx. • Once we have finished with the theory Shaun will show us the empirical distribution that we generated today

  28. Theoretical Power Calculations • Based on Stats, rather than Simulations • Can be calculated by hand sometimes, but Mx does it for us • Note that sample size and alpha-level are the only things we can change, but can assume different effect sizes • Mx gives us the relative power levels at the alpha specified for different sample sizes

  29. Theoretical Power Calculations • We will use the power.mx script to look at the sample size necessary for different power levels • In Mx, power calculations can be computed in 2 ways: • Using Covariance Matrices (We Do This One) • Requiring an initial dataset to generate a likelihood so that we can use a chi-square test

  30. Power.mx 1 ! Simulate the data ! 30% additive genetic ! 20% common environment ! 50% nonshared environment #NGroups 3 G1: model parameters Calculation Begin Matrices; X lower 1 1 fixed Y lower 1 1 fixed Z lower 1 1 fixed End Matrices; Matrix X 0.5477 Matrix Y 0.4472 Matrix Z 0.7071 Begin Algebra; A = X*X' ; C = Y*Y' ; E = Z*Z' ; End Algebra; End

  31. Power.mx 2 G2: MZ twin pairs Calculation Matrices = Group 1 Covariances A+C+E | A+C _ A+C | A+C+E / Options MX%E=mzsim.cov End G3: DZ twin pairs Calculation Matrices = Group 1 H Full 1 1 Covariances A+C+E | H@A+C _ H@A+C | A+C+E / Matrix H 0.5 Options MX%E=dzsim.cov End

  32. Power.mx 3 ! Second part of script ! Fit the wrong model to the simulated data ! to calculate power #NGroups 3 G1 : model parameters Calculation Begin Matrices; X lower 1 1 free Y lower 1 1 fixed Z lower 1 1 free End Matrices; Begin Algebra; A = X*X' ; C = Y*Y' ; E = Z*Z' ; End Algebra; End

  33. Power.mx 4 G2 : MZ twins Data NInput_vars=2 NObservations=350 CMatrix Full File=mzsim.cov Matrices= Group 1 Covariances A+C+E | A+C _ A+C | A+C+E / Option RSiduals End G3 : DZ twins Data NInput_vars=2 NObservations=350 CMatrix Full File=dzsim.cov Matrices= Group 1 H Full 1 1 Covariances A+C+E | H@A+C _ H@A+C | A+C+E / Matix H 0.5 Option RSiduals ! Power for alpha = 0.05 and 1 df Option Power= 0.05,1 End

More Related