Factorial Analysis of Variance II 1. Follow up tests More fun than a rub down with a cheese grater
Follow-ups for Factorial ANOVA • Recall possible outcomes from Factorial ANOVA: • Main effects • Interactions • What might be missing (not specified) from these results? • Differences between pairs of means within each factor (if levels of factor are > 2) • Differences between cells giving rise to interactions 1. 2.
Follow-ups for Main Effects • For main effects, request follow ups for IV’s with > 2 levels 1. “Post Hoc” lets you request follow-ups, but only to the main effects 2.
Follow-ups for Main Effects To do a post hoc on the main effects: 1. select the variables 2. Slide them over 3. Select the post hoc test 4. Continue
Follow-ups for Interactions • What is an interaction? • Arises from the cell means/SDs • Significant non-parallelism 1. 2. 4. 3.
Follow-ups for Interactions • Subsequent simpler analyses • These can go in at least a couple of directions • With a 3 x 2 ANOVA, you could do: • 2 one-way ANOVAs (one at each level of the IV w/2 levels) • One 1-way ANOVA on low anxiety • One 1-way ANOVA on high 1. 2. • In our example, this would be looking for differences in performance associated with pressure level, within each anxiety level 3. 5. 4.
Follow-ups for Interactions 2. 1. 3.
Follow-ups for Interactions • Subsequent simpler analyses • Second possibility: • 3 t-tests (one at each level of the IV w/3 levels) • In our example, this would be looking for differences in performance associated with anxiety level, within each pressure level • One for low pressure • One for moderate pressure • One for high pressure 1. 2.
Follow-ups for Interactions • Final step – control for type 1 error : • Because you are now conducting multiple tests, you should adjust your significance threshold to control for type 1 error. • The Bonferroni adjustment is suitable here • divide by the number of tests being run • So for 2 1-way ANOVAs, use = .05/2 = .025 • For 3 independent t-tests, use = .05/3 = .017 1.
Follow-ups for Interactions • Follow-ups on significant interactions : • Bear in mind that any test conducted after the initial interaction is less powerful than the initial test • So sometimes you will get no significance from the follow-up despite a significant initial test • In this instance, all you can do is suggest cautiously where the differences lie, “by inspection” 1.
Follow-ups for Interactions • Follow-ups on significant interactions : • Note on ordinal (uncrossed) and disordinal (crossed) interactions • Regardless of whether the interaction crosses or not, there is a good chance that main effects found in these analyses are not genuine (that is their existence depends on the level of the other factor) • Always interpret a main effect with caution if there is a significant interaction involving that main effect 1. 2. Uncrossed – genuine main effect 3. Uncrossed – no genuine main effect 4. Crossed – no genuine main effect
Follow-ups for Factorial ANOVA • Summary • No significant effects -No follow ups • Significant main effect only • Pairwise comparisons within significant effects • Significant main effects and a significant interaction • Caution in interpreting main effects (examine graph of interaction)…may be superseded by interaction • Try to find the locus of the interaction (by further ANOVAs and t-tests with Bonferroni adjustment) • Significant interaction only 1.
(Partial) Flow chart for Factorial ANOVA Adjust DV and try again yes no Run ANOVA 1 Is homogeneity significant? Are there any significant effects? Stop! no yes What are they? Only interaction Only main effects main effect(s) and interaction Use post-hocs to interpret – like t-tests 1. Use post-hocs to interpret main effects, BUT consider plot of interaction to see if genuine. 3. Use adjusted α to interpret significance 2. Split file by one variable and run either t-tests or 1-way ANOVA on other to examine locus of interaction Include homogeneity tests; descriptives; partial η2; request post-hocs if appropriate, and PLOT of interaction. Done.