Post Hoc Tests

1. Post Hoc Tests

2. What is a Post Hoc Test? • Review: • Adjusting Alpha Level • Multiple A Priori Comparisons • What makes a test Post Hoc? • Many tests could be Post Hoc… But, there are set Post Hoc tests

3. Studentized Range Statistic q

4. Studentized Range Statistic q Independent Groups Largest mean Smallest mean Example Fail to Reject Note: arrange means in ascending order! = 3.77 critical value

5. Studentized Range Statistic q • q’s can tell us where differences are (more specific than F) • Solving q’s is just like solving t’s • But we solve a lot of q’s… can we speed things up?

6. Solving for the Smallest Significant Difference Example = 3.77 critical value

7. Solving for the Smallest Significant Difference • Solving for the smallest significant difference will help us make quicker comparisons • But we still need a way to organize things nicely…

8. Newman-Kewls smallest difference required was 6.61 smallest significant difference If 2 steps Example

9. A Better Newman-Kewls Example Example

10. A Better Newman-Kewls Example Read Right to Left UNTIL 1. The row is completed 2. A nonsignificant difference is found 3. Reaching a column which was nonsignificant on the previous row

11. Newman-Kewls Summarized • Newman-Kewls tables help organize your q’s • When doing a set of post hoc comparisons it’s best to use a Newman-Kewls table

12. Unequal N’s Tukey-Kramer Replace with

13. Unequal N’s Behrens-Fisher * * Each particular pairing of means must be examined with a different critical value and their own Thus, the smallest significant difference will vary even for a given

14. A Problem with q So Far… • Why are we doing q’s anyway? Why not do t’s instead? • But is q really controlling our alpha level? NO! Using q’s will give use more Type I Errors

15. Trying to fix q Tukey's HSD Tukey's WSD N-K except If there are 4 means, all differences are treated as 4 steps. r = # of steps between the two means to be compared. What Happens to Alpha Level? Power?

16. Tukey’s HSD and WSD Use Tukey’s WSD, not normal method for q

17. Back to Post Hoc in General • What is a post hoc test again? • What are the real issues with Post Hoc tests? Alpha and Power… • q is just one type of post hoc (one way to balance alpha and power), what are others?

18. Dunnett’s Control vs. Treatment run standard and use or, solve for critical difference (CV) Example Go to Table for * Pros…? Cons…?

19. Sheffé’s Test Linear contrast To evaluate MS(contrast) 1) consult F table and find critical value F.05 (k-1, dferror) (CV) F = MS(error) 2) multiply CV by (k-1). (new CV) It sets the family-wise Type-I Error rate ( in our case) for ALL possible linear contrasts, not merely the pair-wise comparisons. Don’t use when only doing pair-wise, because it will be overly conservative.

20. Post Hoc Summary • When to use what… • q in most situations… but use Tukey’s WSD for critical value • Put things in a Newman-Kewls table • when N’s are unequal, use Tukey’s correction • Dunnett’s when you have one control and multiple treatments • Sheffé’s ONLY when you are doing complex comparisons (i.e., contrasts)

21. Post Hoc Summary • Be aware of the alpha level and power issues… • Why can’t we have a perfect test (i.e., low alpha level and high power)? • How does Tukey’s WSD and HSD relate to this? • How does Dunnett’s relate to this? • How does Sheffé’s relate to this?