1 / 55

Statistics for the Social Sciences

Statistics for the Social Sciences. Psychology 340 Fall 2006. Effect sizes & Statistical Power. Outline. Error types revisited Effect size: Cohen’s d Statistical Power Analysis. There really isn’t an effect. There really is an effect. Performing your statistical test.

laasya
Download Presentation

Statistics for the Social Sciences

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Statistics for the Social Sciences Psychology 340 Fall 2006 Effect sizes & Statistical Power

  2. Outline • Error types revisited • Effect size: Cohen’s d • Statistical Power Analysis

  3. There really isn’t an effect There really is an effect Performing your statistical test Real world (‘truth’) H0 is correct H0 is wrong Reject H0 Experimenter’s conclusions Fail to Reject H0

  4. Performing your statistical test Real world (‘truth’) H0 is correct H0 is wrong

  5. Real world (‘truth’) Real world (‘truth’) H0 is correct H0 is correct H0 is wrong H0 is wrong Type I error Type I error Type II error Type II error So there is only one distribution So there are two distributions Performing your statistical test Real world (‘truth’) H0 is correct H0 is wrong The original (null) distribution The new (treatment) distribution The original (null) distribution

  6. Real world (‘truth’) H0 is correct H0 is wrong Type I error Type II error Performing your statistical test Real world (‘truth’) H0 is correct H0 is wrong So there is only one distribution So there are two distributions The original (null) distribution The new (treatment) distribution The original (null) distribution

  7. Real world (‘truth’) H0 is correct H0 is wrong Type I error Type II error Effect Size • Hypothesis test tells us whether the observed difference is probably due to chance or not • It does not tell us how big the difference is H0 is wrong So there are two distributions The new (treatment) distribution The original (null) distribution • Effect size tells us how much the two populations don’t overlap

  8. But this is tied to the particular units of measurement Effect Size • Figuring effect size The new (treatment) distribution The original (null) distribution • Effect size tells us how much the two populations don’t overlap

  9. Effect Size • Standardizedeffect size • Puts into neutral units for comparison (same logic as z-scores) Cohen’s d The new (treatment) distribution The original (null) distribution • Effect size tells us how much the two populations don’t overlap

  10. Effect Size • Effect size conventions • small d = .2 • medium d = .5 • large d = .8 The new (treatment) distribution The original (null) distribution • Effect size tells us how much the two populations don’t overlap

  11. There really isn’t an effect There really is an effect I conclude that there is an effect I can’t detect an effect Error types Real world (‘truth’) H0 is correct H0 is wrong Reject H0 Experimenter’s conclusions Fail to Reject H0

  12. Type I error (): concluding that there is a difference between groups (“an effect”) when there really isn’t. Type II error (): concluding that there isn’t an effect, when there really is. Error types Real world (‘truth’) H0 is correct H0 is wrong Type I error Reject H0 Experimenter’s conclusions Fail to Reject H0 Type II error

  13. Statistical Power • The probability of making a Type II error is related to Statistical Power • Statistical Power: The probability that the study will produce a statistically significant results if the research hypothesis is true (there is an effect) • So how do we compute this?

  14. Real world (‘truth’) H0 is correct H0 is wrong Type I error Type II error a = 0.05 The original (null) distribution Reject H0 Fail to reject H0 Statistical Power Real world (‘truth’) H0: is true (is no treatment effect)

  15. Real world (‘truth’) H0 is correct H0 is wrong Type I error Type II error The new (treatment) distribution The new (treatment) distribution a = 0.05 Fail to reject H0 Statistical Power Real world (‘truth’) H0: is false (is a treatment effect) The original (null) distribution Reject H0

  16. Real world (‘truth’) H0 is correct H0 is wrong Type I error Type II error a = 0.05 b = probability of a Type II error Statistical Power Real world (‘truth’) H0: is false (is a treatment effect) The new (treatment) distribution The original (null) distribution Failing to Reject H0, even though there is a treatment effect Reject H0 Fail to reject H0

  17. Real world (‘truth’) H0 is correct H0 is wrong Type I error Type II error a = 0.05 b = probability of a Type II error Power = 1 - b Statistical Power Real world (‘truth’) H0: is false (is a treatment effect) The new (treatment) distribution The original (null) distribution Failing to Reject H0, even though there is a treatment effect Probability of (correctly) Rejecting H0 Reject H0 Fail to reject H0

  18. Statistical Power 1) Gather the needed information: mean and standard deviation of the Null Population and the predicted mean of Treatment Population • Steps for figuring power

  19. a = 0.05 Statistical Power 2) Figure the raw-score cutoff point on the comparison distribution to reject the null hypothesis • Steps for figuring power From the unit normal table: Z = -1.645 Transform this z-score to a raw score

  20. Statistical Power 3) Figure the Z score for this same point, but on the distribution of means for treatment Population • Steps for figuring power Remember to use the properties of the treatment population! Transform this raw score to a z-score

  21. b = probability of a Type II error Power = 1 - b Statistical Power 4) Use the normal curve table to figure the probability of getting a score more extreme than that Z score • Steps for figuring power From the unit normal table: Z(0.355) = 0.3594 The probability of detecting this an effect of this size from these populations is 64%

  22. Statistical Power Factors that affect Power: • a-level • Sample size • Population standard deviation  • Effect size • 1-tail vs. 2-tailed

  23. a = 0.05 b Power = 1 - b Statistical Power Factors that affect Power: a-level Change from a = 0.05 to 0.01 Reject H0 Fail to reject H0

  24. a = 0.05 a = 0.01 b Power = 1 - b Statistical Power Factors that affect Power: a-level Change from a = 0.05 to 0.01 Reject H0 Fail to reject H0

  25. a = 0.05 a = 0.01 b Power = 1 - b Statistical Power Factors that affect Power: a-level Change from a = 0.05 to 0.01 Reject H0 Fail to reject H0

  26. a = 0.05 a = 0.01 b Power = 1 - b Statistical Power Factors that affect Power: a-level Change from a = 0.05 to 0.01 Reject H0 Fail to reject H0

  27. a = 0.05 a = 0.01 b Power = 1 - b Statistical Power Factors that affect Power: a-level Change from a = 0.05 to 0.01 Reject H0 Fail to reject H0

  28. a = 0.05 a = 0.01 b Power = 1 - b Statistical Power Factors that affect Power: a-level So as the a level gets smaller, so does the Power of the test Change from a = 0.05 to 0.01 Reject H0 Fail to reject H0

  29. Recall that sample size is related to the spread of the distribution a = 0.05 b Power = 1 - b Statistical Power Factors that affect Power: Sample size Change from n = 25 to 100 Reject H0 Fail to reject H0

  30. a = 0.05 b Power = 1 - b Statistical Power Factors that affect Power: Sample size Change from n = 25 to 100 Reject H0 Fail to reject H0

  31. a = 0.05 b Power = 1 - b Statistical Power Factors that affect Power: Sample size Change from n = 25 to 100 Reject H0 Fail to reject H0

  32. a = 0.05 b Power = 1 - b Statistical Power Factors that affect Power: Sample size Change from n = 25 to 100 Reject H0 Fail to reject H0

  33. As the sample gets bigger, the standard error gets smaller and the Power gets larger a = 0.05 b Power = 1 - b Statistical Power Factors that affect Power: Sample size Change from n = 25 to 100 Reject H0 Fail to reject H0

  34. Recall that standard error is related to the spread of the distribution a = 0.05 b Power = 1 - b Statistical Power Factors that affect Power: Population standard deviation Change from  = 25 to 20 Reject H0 Fail to reject H0

  35. a = 0.05 b Power = 1 - b Statistical Power Factors that affect Power: Population standard deviation Change from  = 25 to 20 Reject H0 Fail to reject H0

  36. a = 0.05 b Power = 1 - b Statistical Power Factors that affect Power: Population standard deviation Change from  = 25 to 20 Reject H0 Fail to reject H0

  37. a = 0.05 b Power = 1 - b Statistical Power Factors that affect Power: Population standard deviation Change from  = 25 to 20 Reject H0 Fail to reject H0

  38. As the  gets smaller, the standard error gets smaller and the Power gets larger a = 0.05 b Power = 1 - b Statistical Power Factors that affect Power: Population standard deviation Change from  = 25 to 20 Reject H0 Fail to reject H0

  39. a = 0.05 b Power = 1 - b mno treatment mtreatment Statistical Power Factors that affect Power: Effect size Compare a small effect (difference) to a big effect Reject H0 Fail to reject H0

  40. a = 0.05 b Power = 1 - b Statistical Power Factors that affect Power: Effect size Compare a small effect (difference) to a big effect Reject H0 Fail to reject H0 mno treatment mtreatment

  41. a = 0.05 b Power = 1 - b Statistical Power Factors that affect Power: Effect size Compare a small effect (difference) to a big effect Reject H0 Fail to reject H0

  42. a = 0.05 b Power = 1 - b Statistical Power Factors that affect Power: Effect size Compare a small effect (difference) to a big effect Reject H0 Fail to reject H0

  43. a = 0.05 b Power = 1 - b Statistical Power Factors that affect Power: Effect size Compare a small effect (difference) to a big effect Reject H0 Fail to reject H0

  44. a = 0.05 b Power = 1 - b Statistical Power Factors that affect Power: Effect size Compare a small effect (difference) to a big effect Reject H0 Fail to reject H0

  45. a = 0.05 b Power = 1 - b Statistical Power Factors that affect Power: Effect size Compare a small effect (difference) to a big effect Reject H0 Fail to reject H0

  46. a = 0.05 As the effect gets bigger, the Power gets larger b Power = 1 - b Statistical Power Factors that affect Power: Effect size Compare a small effect (difference) to a big effect Reject H0 Fail to reject H0

  47. a = 0.05 b Power = 1 - b Statistical Power Factors that affect Power: 1-tail vs. 2-tailed Change from a = 0.05 two-tailed to a = 0.05 two-tailed Reject H0 Fail to reject H0

  48. a = 0.05 p = 0.025 b Power = 1 - b p = 0.025 Statistical Power Factors that affect Power: 1-tail vs. 2-tailed Change from a = 0.05 two-tailed to a = 0.05 two-tailed Reject H0 Fail to reject H0

  49. a = 0.05 b Power = 1 - b Statistical Power Factors that affect Power: 1-tail vs. 2-tailed Change from a = 0.05 two-tailed to a = 0.05 two-tailed p = 0.025 p = 0.025 Reject H0 Fail to reject H0

  50. a = 0.05 b Power = 1 - b Statistical Power Factors that affect Power: 1-tail vs. 2-tailed Change from a = 0.05 two-tailed to a = 0.05 two-tailed p = 0.025 p = 0.025 Reject H0 Fail to reject H0

More Related