1 / 52

# Group comparisons: Intro to factorial design

Group comparisons: Intro to factorial design. 31:43/Josh Rodefer, Ph.D. . Previously (?). Experimental design Design: Between vs Within Compare 2 groups (t test; different types) Paired (within; repeated measures) Unpaired (between) Control &amp; confounds

## Group comparisons: Intro to factorial design

E N D

### Presentation Transcript

1. Group comparisons:Intro to factorial design 31:43/Josh Rodefer, Ph.D.

2. Previously (?) • Experimental design • Design: Between vs Within • Compare 2 groups (t test; different types) • Paired (within; repeated measures) • Unpaired (between) • Control & confounds • Extraneous variables (random vs systematic) • How to control or minimize?

3. Basic issues in experimental • What do you need for an experiment? • An independent variable (what the researcher varies; minimally 2 groups) • Equal assignment to groups (lots of ways to do this) • Controlling extraneous variables (alternative explanations; confounds)

4. Hypothesis testing: 2 samples • Want to test relationship between the means • Use a t-test • Used when you have 2 levels of 1 IV • Dependent samples (1 sample, tested twice) • Independent samples (2 different samples) • aka Within- (dependent) and Between (independent) designs

5. Hypothesis testing: 2 samples • Dependent samples (Within) • Eg, before and after treatment (or pre/post) • 20 people tested on attitude towards recycling after watching nature video • IV=viewing nature video • Level 1: test score pre-video • Level 2: test score post-video • DV score on test • 2 samples, same people in each sample

6. Hypothesis testing: 2 samples • Independent samples (between) • 2 scores are taken, but from different groups; 2 independent samples (random) • Eg, study effects of age on swearing • 20 ten yr olds and 20 fifteen yr olds, record swearing in 2 hr period • IV=age • Level 1= 10 yrs old • Level 2= 15 yrs old

7. More than 2 groups: ANOVA • Analysis of Variance • Tests the differences between treatment groups (conditions) to see if they are significant • Look at variance in DV, partition it into 2 components: • variance due to IV (good) • variance due to error (bad; extraneous variables) • Asks if the ratio (IV variance/error variance) between the two types of variance is greater than would be expected due to chance (or equal; or about =1.00) • F test examines this ratio

8. ANOVA – when to use? • One IV: at least 3 conditions • Between subjects: One-way ANOVA • Eg, effect of psychiatric diagnosis (depression, panic, no disorder) on memory • Within subjects: Repeated measures ANOVA • Eg, experiment where subjects experience all conditions; example Stroop effect, colors/words • Two (or more) IVs • Factorial ANOVA • Effect of psychiatric diagnosis on Stroop effect; multiple IVs

9. First: One way ANOVA • One IV: 3 (or more conditions) • Single test • Tests Null that all group means are equal • Alternative: all group means are not equal • Null must be non-directional (vs t test directional) • Between subjects ANOVA • Analogous to independent samples t-test • (stats trivia: In fact….if you have 2 independent groups, then F=t2)

10. One way ANOVA • Why not just perform 3 t-tests? • Psychiatric diagnosis & memory example • Depression vs panic disorder • Depression vs no disorder • Panic disorder vs no disorder • Same thing, right?

11. One way ANOVA • Multiple tests result in an inflated alpha • Alpha for each test =0.05 (risk for Type I) • If you do 3 tests, then each is at 0.05 and they are additive • Thus, doing 3 t-test increases the experiment-wise Type I error rate from 0.05 to 0.15 (not acceptable) • Must use ANOVA to test all combinations simultaneously, keeping alpha at 0.05

12. One way ANOVA • So, we can calculate the total variance and determine how much is due to IV and how much is error • F ratio = variance due to IV /error variance • Variance due to IV is called MSB (Mean square between groups) • Variance due to extraneous variables or error is called MSW (Mean square within groups) • F test formula: F = MSB/MSW

13. ANOVA • Most data sets have both error variance and variance due to the IV (or, both between and within group variability) • We want to know if the between group variance is due to a true effect of the IV (is this effect real?) • F=MSb/MSw • If F = about 1, then no effect of IV • If F > 1 then may have an effect of IV (need to look up value on F table, depends on critical F and df)

14. Factorial ANOVA: 2 way designs • Have 2 (or more) IVs in the same experiment; eg, test effects of gender and age on humor: • IV gender (M/F) • IV age (10, 20, or 30 yrs old) • DV freq of laughing during comedy show • Why not conduct 2 separate experiments? • Gender on humor; age on humor

15. Factorial ANOVA: 2 way designs • Why do you want to look at 2 IVs? • More efficient: study factor A then study factor B…or…you could study them together • More interesting: can see how things relate together • More representative of the real world

16. Factorial designs • Have 2 (or more) IVs • Each IV is called a ‘factor’ • Each factor has at least 2 levels/conditions • The design must be complete; each level of one IV must occur with each level of the other IV(s) • Described as: 2x2 or 3x5 or…2x3x4 • Terminology: Factors/Main effects; levels; cells • Can be experiment or quasi-experiment (what’s quasi?)

17. Factorial designs • Example: examine effect of sleep and caffeine on reaction time study • IV-1: Sleep: 2 hrs vs 8 hrs • IV-2: Caffeine: zero (water) vs 3 Cokes • DV: computer game reaction time • Both IVs are between subjects variables

18. Factorial design • Randomly select subject for each condition; equal cell sizes • 2 IVs; test for 3 effects: • 2 Main effects (effects of sleep and caffeine) • 1 interaction (interaction of sleep & caffeine) • Thus, 3 null hypotheses, 3 alternative hypotheses, and 3 F tests

19. Factorial designs • Main effect • The effect of one factor (IV) collapsed across levels of the other factor (IV) (ie, you ignore the other IV) • Test one main effect for each factor (IV) • Need to analyze statistics to check for significance

20. Factorial design: Main effect graphic 8 hrs 2 hrs 0 Cokes 3 Cokes Did different amounts of sleep have an effect? Mean of 2 hrs Mean of 8 hrs

21. Factorial design: Main effect graphic 8 hrs 2 hrs 0 Cokes Mean of 0 Cokes 3 Cokes Mean of 3 Cokes Did different quantities of Coca-cola have an effect?

22. Post hoc tests • Post hoc means after the fact • Tests that are performed after finding a significant effect (sig F value) • ANOVAs tell you if there is a difference, but they don’t tell you where it is • Post hoc tests answer that question…

23. Post hoc tests • Would you need a follow up post hoc test after a significant t-test? • Would you need to follow up a significant F-test where there are 3 levels of the IV with a post hoc test (say, for the sleep experiment you used 2, 4 or 8 hrs of sleep as three different levels of the IV).

24. Post hoc tests • So, need to use post hoc tests when… • You have 3 or more means • You have a significant difference with a general/omnibus test (F test) • Post hoc tests (pairwise comparisons) examine all possible combinations of means

25. Post hoc tests • Many types: Tukey’s HSD (honestly significant difference), Bonferroni, Scheffe’s, Kruskal-Wallis, Dunnett’s… • Each tests pairwise hypotheses • Each calculates a particular statistic that resembles a t-test (but it isn’t that simple)

26. Post hoc tests • Consider the 3 level IV, where you find a significant F value • Next need to assess all possible pairwise comparison: • Mean 1 =/diff mean 2 • Mean 1 =/diff mean 3 • Mean 2 =/diff mean 3 • Compute pairwise comparison for each test…ascertain where the difference(s) are statistically significant

27. Next time • More factorials & interactions

28. Previously -- 2nd Lecture • Experimental design (Between vs Within) • Compare 2 groups (paired vs unpaired) • Control & confounds (random vs systematic) • ANOVAs (3 or more groups) • Do overall F tests • Pairwise compartions (post hoc tests) • (like t tests; done after significant F; tells you where the difference is; that is, which two means are significantly different from each other)

29. ANOVA – when to use? • One IV: at least 3 conditions • Between subjects: One-way ANOVA • Eg, effect of psychiatric diagnosis (depression, panic, no disorder) on memory • Within subjects: Repeated measures ANOVA • Eg, experiment where subjects experience all conditions; example Stroop effect, colors/words • Two (or more) IVs • Factorial ANOVA • Effect of psychiatric diagnosis on Stroop effect; multiple IVs

30. Factorial designs • Have 2 (or more) IVs • Each IV is called a ‘factor’ • Each factor has at least 2 levels/conditions • The design must be complete (sometimes called ‘completely crossed’); each level of one IV must occur with each level of the other IV(s) • Described as: 2x2 or 3x5 or…2x3x4 • Can be experiment or quasi-experiment

31. Factorial designs • Example: examine effect of sleep and caffeine on reaction time study • IV-1: Sleep: 2 hrs vs 8 hrs • IV-2: Caffeine: zero (water) vs 3 Cokes • DV: computer game – number correct • Both IVs are between subjects variables

32. Factorial design • Randomly select subject for each condition; equal cell sizes • 2 IVs; test for 3 effects: • 2 Main effects (effects of sleep and caffeine) • 1 interaction (interaction of sleep & caffeine) • Thus, 3 null hypotheses, 3 alternative hypotheses, and 3 F tests

33. Factorial designs • Main effect • The effect of one factor (IV) collapsed across levels of the other factor (IV) (ie, you ignore the other IV) • Test one main effect for each factor (IV)

34. Factorial designs • Interactions are a way to qualify findings • Revisit coke and sleep study 8 hrs 2 hrs 0 Cokes 3 Cokes

35. Factorial Design • Calculate cell means and plot 2 hrs 8 hrs 0 Cokes 3 Cokes

36. Factorial Design • Calculate cell means and plot 2 hrs 8 hrs 0 Cokes 3 Cokes What does the parallel shift suggest to you?

37. Factorial Design • Calculate cell means and plot 2 hrs 8 hrs 0 Cokes 3 Cokes

38. Factorial Design • Calculate cell means and plot 2 hrs 8 hrs 0 Cokes 3 Cokes Spreading interaction (aka ordinal); usually at least one main effect

39. Factorial Design • Calculate cell means and plot 2 hrs 8 hrs 0 Cokes 3 Cokes But what about if we plotted this differently?

40. Factorial Design • Calculate cell means and plot 2 hrs 8 hrs 0 Cokes 3 Cokes

41. Factorial Design • Calculate cell means and plot 2 hrs 8 hrs 0 Cokes 3 Cokes Cross-over interaction (aka disordinal); opposite effects of IV1 at different levels of IV2; usually no main effects, but still can have interaction (!)

42. Interactions • So look at main effects • Look for interactions • Graphing things helps • parallel lines = no interactions • crossing or close to crossing lines = interaction • But ultimately you need stats and p values to make judgement about significance

43. Factorials • So with 2 Independent variables you can have multiple types of designs • Both IV are within • Both IV are between • One IV is within and one IV is between • Also can have 1 true IV, and 1 quasi-IV (sometimes called ‘mixed’ or ‘experi-corr’) • Lets you consider unmanipulated factors like subject variables (sex, age, diagnoses, etc.) • The more correlational something is, the more external validity (and less internal validity) you have

44. Within designs • Simple 2 group comparison or ANOVA, doesn’t matter…. • Limitations? • How to fix?

45. Counter balancing schemes • Reverse order of group presentation (AB vs BA); but remember ANOVA usually have more conditions • Gets complicated quickly…. • Consider a within subjects design IV that has 4 levels, how many different orders of conditions?

46. Counter balancing schemes • Reverse order of group presentation (AB vs BA); but remember ANOVA usually have more conditions • Gets complicated quickly…. • Consider a within subjects design IV that has 4 levels, how many different orders of conditions? • ABCD

47. Counter balancing schemes • Reverse order of group presentation (AB vs BA); but remember ANOVA usually have more conditions • Gets complicated quickly…. • Consider a within subjects design IV that has 4 levels, how many different orders of conditions? • ABCD, ABDC,

48. Counter balancing schemes • Reverse order of group presentation (AB vs BA); but remember ANOVA usually have more conditions • Gets complicated quickly…. • Consider a within subjects design IV that has 4 levels, how many different orders of conditions? • ABCD, ABDC, ACBD,

49. Counter balancing schemes • Reverse order of group presentation (AB vs BA); but remember ANOVA usually have more conditions • Gets complicated quickly…. • Consider a within subjects design IV that has 4 levels, how many different orders of conditions? • ABCD, ABDC, ACBD, ACDB, ADBC, ADCB • BCDA, BCAD, BDAC, BDCA, BACD, BADC • CDAB, CDBA, CABD, CADB, CBAD, CBDA • DABC, DACB, DBAC, DBCA, DCAB, DCBA

50. Counterbalancing • Too confusing…more simple way is a Latin Square design • Number conditions = number of orders • Each condition appears at each position only once C D A B 1st condition 2nd condition 3rd condition 4th condition

More Related