Download Presentation
## Comparing Cell Means

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Comparing Cell Means**Planned Comparisons & Post Hoc Tests**Questions**• What is the main difference between planned comparisons and post hoc tests? • Generate numbers (like 0 1, -1 or 1 –1/2, -1/2) to create a contrast appropriate for a given problem. • How many independent comparisons can be made in a given design? • What is the difference between a per comparison and a familywise error rate? • How does Bonferroni deal with familywise error rate problems? • What is the studentized range statistic? How is it used?**Questions (2)**• What is the difference between the Tukey HSD and the Newman-Keuls? • What are the considerations when choosing a post hoc test (what do you need to trade-off)? • Describe (make up) a concrete example where you would use planned comparisons instead of an overall F test. Explain why the planned comparison is the proper analysis. • Describe (make up) a concrete example where you would use a post hoc test. Explain why the post hoc test is needed (not the specific choice of post hoc test, but rather why post hoc test at all).**Planned vs. Post Hoc**• Planned Comparisons or Contrasts • Use instead of overall F test. Planned before the study. • Post Hoc or Incidental tests. • Use after significant overall F test to investigate specific means. No specific plan before study.**Planned Comparisons (1)**Population Comparison: Weights are real numbers not all zero. Sum of weights must equal zero. Sample Comparison:**Planned Comparison (2)**(3 possible comparisons) (Data) (Summary Table)**Sampling Variance of Planned Comparisons**The sample comparison is an unbiased estimate of the population comparison. The variance of the sampling distribution of the comparison: Sampling variance will be large when within cells variance is large, the weights are large, and the number of people in each cell is small. Estimated by: We substitute for**Significance Test**df=N-J; 16-4=12=dfe. t(12) =-2.86, p < .05**Significance Test**df=N-J=**Review**• What is the main difference between planned comparisons and post hoc tests? • Suppose I do a blind orange juice taste test and discover that my means are: If my hypothesis is that Tropicana is better than all others, what are my contrast weights?**Independence of Planned Comparisons**You can make several planned comparisons on the same data. Some of these comparisons are independent; some are dependent. We want them independent. Two comparisons from a normal population with equal sample sizes in each cell are independent if the sum of the products of weights is zero. With unequal sample sizes, it’s:**Independence (2)**One and two are orthogonal; one and three are not. There are J-1 orthogonal comparisons. Use only what you need.**Choosing Comparisons**Usually done on basis of theory. But there are methods to generate all possible orthogonal comparisons.**Error Rates**• With 1 test, we set alpha = Type I error rate. • With multiple tests, original (nominal) alpha is called the per comparison error rate ( ). • With comparisons, we have a family of tests on the same data. Want to know the probability of at least 1 Type I error in the family of tests. Such a probability is called familywise error rate ( ). • For independent tests, • E.g., 10 tests:**Bonferroni Tests**• Familywise error depends on the number of tests (K) and the nominal alpha, . • Bonferroni’s solution is to set: • Suppose we want FW error to be .05 and we will have 4 comparisons. Then Where is an aspiration level. We use the adjusted alpha (.0125) for each of the 4 tests.**Bonferroni Test (2)**• Use the adjusted alpha (e.g., .0125) for each comparison. • Look at the p value on the printout (use .0125 instead of .05). • Use a statistical function (e.g., Excel, SAS) if you want to find the critical value. • E.g., Excel function TINV says with p=.0125 and df=12, t is 2.93.**Review**• How many independent comparisons can be made in a given design? • What is the difference between a per comparison and a familywise error rate? • How does Bonferroni deal with familywise error rate problems?**Post Hoc Tests**• Given a significant F, where are the mean differences? • Often do not have planned comparisons. • Usually compare pairs of means. • There are many methods of post hoc (after the fact) tests.**Scheffé**• Can use for any contrast. Follows same calculations, but uses different critical values. • Instead of comparing the test statistic to a critical value of t, use: Where the F comes from the overall F test (J-1 and N-Jdf).**Scheffé (2)**(Data from earlier problem.) The comparison is not significant because |-2.86|<3.24.**Paired comparisons**Newman Keuls and Tukey HSD are two (of many) choices. Both depend on q, the studentized range statistic. Suppose we have J independent sample means and we find the largest and the smallest. MSerror comes from the ANOVA we did to get the J means. The n refers to sample size per cell. If two cells are unequal, use 2n1n2/(n1+n2). The sampling distribution of q depends on k, the number of means covered by the range (max-min), and on v, the degrees of freedom for MSerror.**Tukey HSD**HSD = honestly significant difference. For HSD, use k = J, the number of groups in the study. Choose alpha, and find the df for error. Look up the value qα. Then find the value: Compare HSD to the absolute value of the difference between all pairs of means. Any difference larger than HSD is significant.**HSD 2**K = 5 groups; n=12 per group, v has 55 df. Tabled value of q with alpha =.05 is 3.98.**Newman-Keuls**Layer refers to how many means apart. Layer 4 Layer 3 Layer 2 Layer 1 Same as HSD except the value of q changes with layers. For layer k-1 (here 4), use HSD. For each layer down, subtract 1 from the value of k for the tabled value of q.**Comparing Post Hoc Tests**The Newman-Keuls found 3 significant differences in our example. The HSD found 2 differences. If we had used the Bonferroni approach,we would have found an interval of 15.91 required for significance (and therefore the same two significant as HSD). Thus, power descends from the Newman-Keuls to the HSD to the Bonferroni. The type I error rates go just the opposite, the lowest to Bonferroni, then HSD and finally Newman-Keuls. Do you want to be liberal or conservative in your choice of tests? Type I error vs Power.**Review**• What is the studentized range statistic? How is it used? • What is the difference between the Tukey HSD and the Newman-Keuls? • What are the considerations when choosing a post hoc test (what do you need to trade-off)? • Describe (make up) a concrete example where you would use planned comparisons instead of an overall F test. Explain why the planned comparison is the proper analysis. • Describe (make up) a concrete example where you would use a post hoc test. Explain why the post hoc test is needed (not the specific choice of post hoc test, but rather why post hoc test at all).