Analysis of Variance: Some Final Issues

1. Analysis of Variance: Some Final Issues • Degrees of Freedom • Familywise Error Rate (Bonferroni Adjustment) • Magnitude of Effect: Eta Square, Omega Square • Review: Main Effects, Interaction Effects and Simple Effects

2. Degrees of Freedom (df ) • Number of “observations” free to vary. There will be one df associated with the effect you are reporting, and one associated with error. • dftotal = N – 1 (N observations) • dfgroups = g – 1 (Number of groups) • dferror = g (n - 1) (An easier easy to compute: dftotal d – dfgroups)

3. Summary Table F(3,68) = 35.8; p<.05

4. Familywise Error Rate • Suppose you are comparing 5 groups. Significant F only shows that not all groups are equal • What groups are different. Number of pairwise comparisons are 10 • Error rate operates at the level of each comparison. • Error rate increases with number of comparisons.

5. In case of multiple comparisons: Bonferroni adjustment • The more tests we run the more likely we are to make Type I error. • Good reason to hold down number of tests • Run t tests between pairs of groups, as usual • Hold down number of t tests • Reject if t exceeds critical value in Bonferroni table • Works by using a more strict level of significance for each comparison

6. Bonferroni t--cont. • Critical value of a for each test set at .05/c, where c = number of tests run • Assuming familywise a = .05 • e. g. with 3 tests, each t must be significant at .05/3 = .0167 level. • With computer printout, just make sure calculated probability < .05/c • Necessary table is in the book

7. Magnitude of Effect • Why you need to compute magnitude of effect indices • Level of significance tells us nothing about size of effect • T and F values inflate with sample size • Hard to compare with statistic with other kinds of analysis

8. Visual Explanation of Magnitude of Effect

9. Magnitude of Effect--cont. • Eta squared (h2) • Easy to calculate • Somewhat biased on the high side • Percent of variation in the data that can be attributed to treatment differences • Omega squared (w2) • Much less biased than h2 • Not as intuitive • We adjust both numerator and denominator with MSerror • Formula on next slide

10. h2 and w2 for example problem • h2 = .18: 18% of variability in symptoms can be accounted for by treatment • w2 = .12: This is a less biased estimate, and note that it is 33% smaller.

11. R2 is also often used. It is based on the sum of squares. For experiments use Omega Squared. For correlations use R squared. Value of R square is greater than omega squared. Cohen classified effects as Small Effect: .01 Medium Effect: .06 Large Effect: .15

12. Factorial Analysis of Variance • What is a factorial design? • Main effects • Interactions • Simple effects • Magnitude of effect

13. Main effects • There are two factors in the analysis: Type of Rating and Why item was recommended • If you examine effect of “Knowing Why” (ignoring Type of Rating for the time being), you are looking at the main effect of Knowing Why. • If we look at the effect of Type of Rating, ignoring Knowing Why, then you are looking at the main effect of Knowing Why.

14. Simple effects • If you could restrict yourself to one level of one IV for the time being, and looking at the effect of the other IV within that level. • Effect of Knowing Why at one level of Type of Rating at, then that is a simple effect of Knowing Why on Interest Ratings. • Above is identical to t-test Simple Effect of Knowing Why at one level of Type of Rating (I.e., Interest)

15. Interactions (Effect of one variable on the other) Does Knowing Why effect Interest and Confidence differentially?

16. Example Data with computation(cell means and standard deviations)

17. Plotting Results

18. Effects to be estimated • Differences due to instructions • Errors more in condition without instructions • Differences due to gender • Males appear higher than females • Interaction of video and gender • What is an interaction? • Do instructions effect males and females equally? Cont.

19. Estimated Effects--cont. • Error • average within-cell variance • Sum of squares and mean squares • Extension of the same concepts in the one-way

20. Calculations • Total sum of squares • Main effect sum of squares Cont.

21. Calculations--cont. • Interaction sum of squares • Calculate SScells and subtract SSV and SSG • SSerror = SStotal - SScells • or, MSerror can be found as average of cell variances

22. Degrees of Freedom • df for main effects = number of levels - 1 • df for interaction = product of dfmain effects • dferror = N - ab = N - # cells • dftotal = N - 1

23. Calculations for Data • SStotal requires raw data. • It is actually = 171.50 • SSvideo Cont.

24. Calculations--cont. • SSgender Cont.

25. Calculations--cont. • SScells • SSVXG = SScells - SSinstruction- SSgender= 171.375 - 105.125 - 66.125 = 0.125 Cont.

26. Calculations--cont. • MSerror = average of cell variances =(4.62 + 3.52 + 4.22 + 2.82)/4 =58.89/4 = 14.723 • Note that this is MSerror and not SSerror

27. Summary Table

28. Elaborate on Interactions • Diagrammed on next slide as line graph • Note parallelism of lines • Instruction differences did not depend on gender

29. Line Graph of Interaction