Create Presentation
Download Presentation

Download Presentation

Group comparisons: Intro to factorial design

Download Presentation
## Group comparisons: Intro to factorial design

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Group comparisons:Intro to factorial design**31:43/Josh Rodefer, Ph.D.**Previously (?)**• Experimental design • Design: Between vs Within • Compare 2 groups (t test; different types) • Paired (within; repeated measures) • Unpaired (between) • Control & confounds • Extraneous variables (random vs systematic) • How to control or minimize?**Basic issues in experimental**• What do you need for an experiment? • An independent variable (what the researcher varies; minimally 2 groups) • Equal assignment to groups (lots of ways to do this) • Controlling extraneous variables (alternative explanations; confounds)**Hypothesis testing: 2 samples**• Want to test relationship between the means • Use a t-test • Used when you have 2 levels of 1 IV • Dependent samples (1 sample, tested twice) • Independent samples (2 different samples) • aka Within- (dependent) and Between (independent) designs**Hypothesis testing: 2 samples**• Dependent samples (Within) • Eg, before and after treatment (or pre/post) • 20 people tested on attitude towards recycling after watching nature video • IV=viewing nature video • Level 1: test score pre-video • Level 2: test score post-video • DV score on test • 2 samples, same people in each sample**Hypothesis testing: 2 samples**• Independent samples (between) • 2 scores are taken, but from different groups; 2 independent samples (random) • Eg, study effects of age on swearing • 20 ten yr olds and 20 fifteen yr olds, record swearing in 2 hr period • IV=age • Level 1= 10 yrs old • Level 2= 15 yrs old**More than 2 groups: ANOVA**• Analysis of Variance • Tests the differences between treatment groups (conditions) to see if they are significant • Look at variance in DV, partition it into 2 components: • variance due to IV (good) • variance due to error (bad; extraneous variables) • Asks if the ratio (IV variance/error variance) between the two types of variance is greater than would be expected due to chance (or equal; or about =1.00) • F test examines this ratio**ANOVA – when to use?**• One IV: at least 3 conditions • Between subjects: One-way ANOVA • Eg, effect of psychiatric diagnosis (depression, panic, no disorder) on memory • Within subjects: Repeated measures ANOVA • Eg, experiment where subjects experience all conditions; example Stroop effect, colors/words • Two (or more) IVs • Factorial ANOVA • Effect of psychiatric diagnosis on Stroop effect; multiple IVs**First: One way ANOVA**• One IV: 3 (or more conditions) • Single test • Tests Null that all group means are equal • Alternative: all group means are not equal • Null must be non-directional (vs t test directional) • Between subjects ANOVA • Analogous to independent samples t-test • (stats trivia: In fact….if you have 2 independent groups, then F=t2)**One way ANOVA**• Why not just perform 3 t-tests? • Psychiatric diagnosis & memory example • Depression vs panic disorder • Depression vs no disorder • Panic disorder vs no disorder • Same thing, right?**One way ANOVA**• Multiple tests result in an inflated alpha • Alpha for each test =0.05 (risk for Type I) • If you do 3 tests, then each is at 0.05 and they are additive • Thus, doing 3 t-test increases the experiment-wise Type I error rate from 0.05 to 0.15 (not acceptable) • Must use ANOVA to test all combinations simultaneously, keeping alpha at 0.05**One way ANOVA**• So, we can calculate the total variance and determine how much is due to IV and how much is error • F ratio = variance due to IV /error variance • Variance due to IV is called MSB (Mean square between groups) • Variance due to extraneous variables or error is called MSW (Mean square within groups) • F test formula: F = MSB/MSW**ANOVA**• Most data sets have both error variance and variance due to the IV (or, both between and within group variability) • We want to know if the between group variance is due to a true effect of the IV (is this effect real?) • F=MSb/MSw • If F = about 1, then no effect of IV • If F > 1 then may have an effect of IV (need to look up value on F table, depends on critical F and df)**Factorial ANOVA: 2 way designs**• Have 2 (or more) IVs in the same experiment; eg, test effects of gender and age on humor: • IV gender (M/F) • IV age (10, 20, or 30 yrs old) • DV freq of laughing during comedy show • Why not conduct 2 separate experiments? • Gender on humor; age on humor**Factorial ANOVA: 2 way designs**• Why do you want to look at 2 IVs? • More efficient: study factor A then study factor B…or…you could study them together • More interesting: can see how things relate together • More representative of the real world**Factorial designs**• Have 2 (or more) IVs • Each IV is called a ‘factor’ • Each factor has at least 2 levels/conditions • The design must be complete; each level of one IV must occur with each level of the other IV(s) • Described as: 2x2 or 3x5 or…2x3x4 • Terminology: Factors/Main effects; levels; cells • Can be experiment or quasi-experiment (what’s quasi?)**Factorial designs**• Example: examine effect of sleep and caffeine on reaction time study • IV-1: Sleep: 2 hrs vs 8 hrs • IV-2: Caffeine: zero (water) vs 3 Cokes • DV: computer game reaction time • Both IVs are between subjects variables**Factorial design**• Randomly select subject for each condition; equal cell sizes • 2 IVs; test for 3 effects: • 2 Main effects (effects of sleep and caffeine) • 1 interaction (interaction of sleep & caffeine) • Thus, 3 null hypotheses, 3 alternative hypotheses, and 3 F tests**Factorial designs**• Main effect • The effect of one factor (IV) collapsed across levels of the other factor (IV) (ie, you ignore the other IV) • Test one main effect for each factor (IV) • Need to analyze statistics to check for significance**Factorial design: Main effect graphic**8 hrs 2 hrs 0 Cokes 3 Cokes Did different amounts of sleep have an effect? Mean of 2 hrs Mean of 8 hrs**Factorial design: Main effect graphic**8 hrs 2 hrs 0 Cokes Mean of 0 Cokes 3 Cokes Mean of 3 Cokes Did different quantities of Coca-cola have an effect?**Post hoc tests**• Post hoc means after the fact • Tests that are performed after finding a significant effect (sig F value) • ANOVAs tell you if there is a difference, but they don’t tell you where it is • Post hoc tests answer that question…**Post hoc tests**• Would you need a follow up post hoc test after a significant t-test? • Would you need to follow up a significant F-test where there are 3 levels of the IV with a post hoc test (say, for the sleep experiment you used 2, 4 or 8 hrs of sleep as three different levels of the IV).**Post hoc tests**• So, need to use post hoc tests when… • You have 3 or more means • You have a significant difference with a general/omnibus test (F test) • Post hoc tests (pairwise comparisons) examine all possible combinations of means**Post hoc tests**• Many types: Tukey’s HSD (honestly significant difference), Bonferroni, Scheffe’s, Kruskal-Wallis, Dunnett’s… • Each tests pairwise hypotheses • Each calculates a particular statistic that resembles a t-test (but it isn’t that simple)**Post hoc tests**• Consider the 3 level IV, where you find a significant F value • Next need to assess all possible pairwise comparison: • Mean 1 =/diff mean 2 • Mean 1 =/diff mean 3 • Mean 2 =/diff mean 3 • Compute pairwise comparison for each test…ascertain where the difference(s) are statistically significant**Next time**• More factorials & interactions**Previously -- 2nd Lecture**• Experimental design (Between vs Within) • Compare 2 groups (paired vs unpaired) • Control & confounds (random vs systematic) • ANOVAs (3 or more groups) • Do overall F tests • Pairwise compartions (post hoc tests) • (like t tests; done after significant F; tells you where the difference is; that is, which two means are significantly different from each other)**ANOVA – when to use?**• One IV: at least 3 conditions • Between subjects: One-way ANOVA • Eg, effect of psychiatric diagnosis (depression, panic, no disorder) on memory • Within subjects: Repeated measures ANOVA • Eg, experiment where subjects experience all conditions; example Stroop effect, colors/words • Two (or more) IVs • Factorial ANOVA • Effect of psychiatric diagnosis on Stroop effect; multiple IVs**Factorial designs**• Have 2 (or more) IVs • Each IV is called a ‘factor’ • Each factor has at least 2 levels/conditions • The design must be complete (sometimes called ‘completely crossed’); each level of one IV must occur with each level of the other IV(s) • Described as: 2x2 or 3x5 or…2x3x4 • Can be experiment or quasi-experiment**Factorial designs**• Example: examine effect of sleep and caffeine on reaction time study • IV-1: Sleep: 2 hrs vs 8 hrs • IV-2: Caffeine: zero (water) vs 3 Cokes • DV: computer game – number correct • Both IVs are between subjects variables**Factorial design**• Randomly select subject for each condition; equal cell sizes • 2 IVs; test for 3 effects: • 2 Main effects (effects of sleep and caffeine) • 1 interaction (interaction of sleep & caffeine) • Thus, 3 null hypotheses, 3 alternative hypotheses, and 3 F tests**Factorial designs**• Main effect • The effect of one factor (IV) collapsed across levels of the other factor (IV) (ie, you ignore the other IV) • Test one main effect for each factor (IV)**Factorial designs**• Interactions are a way to qualify findings • Revisit coke and sleep study 8 hrs 2 hrs 0 Cokes 3 Cokes**Factorial Design**• Calculate cell means and plot 2 hrs 8 hrs 0 Cokes 3 Cokes**Factorial Design**• Calculate cell means and plot 2 hrs 8 hrs 0 Cokes 3 Cokes What does the parallel shift suggest to you?**Factorial Design**• Calculate cell means and plot 2 hrs 8 hrs 0 Cokes 3 Cokes**Factorial Design**• Calculate cell means and plot 2 hrs 8 hrs 0 Cokes 3 Cokes Spreading interaction (aka ordinal); usually at least one main effect**Factorial Design**• Calculate cell means and plot 2 hrs 8 hrs 0 Cokes 3 Cokes But what about if we plotted this differently?**Factorial Design**• Calculate cell means and plot 2 hrs 8 hrs 0 Cokes 3 Cokes**Factorial Design**• Calculate cell means and plot 2 hrs 8 hrs 0 Cokes 3 Cokes Cross-over interaction (aka disordinal); opposite effects of IV1 at different levels of IV2; usually no main effects, but still can have interaction (!)**Interactions**• So look at main effects • Look for interactions • Graphing things helps • parallel lines = no interactions • crossing or close to crossing lines = interaction • But ultimately you need stats and p values to make judgement about significance**Factorials**• So with 2 Independent variables you can have multiple types of designs • Both IV are within • Both IV are between • One IV is within and one IV is between • Also can have 1 true IV, and 1 quasi-IV (sometimes called ‘mixed’ or ‘experi-corr’) • Lets you consider unmanipulated factors like subject variables (sex, age, diagnoses, etc.) • The more correlational something is, the more external validity (and less internal validity) you have**Within designs**• Simple 2 group comparison or ANOVA, doesn’t matter…. • Limitations? • How to fix?**Counter balancing schemes**• Reverse order of group presentation (AB vs BA); but remember ANOVA usually have more conditions • Gets complicated quickly…. • Consider a within subjects design IV that has 4 levels, how many different orders of conditions?**Counter balancing schemes**• Reverse order of group presentation (AB vs BA); but remember ANOVA usually have more conditions • Gets complicated quickly…. • Consider a within subjects design IV that has 4 levels, how many different orders of conditions? • ABCD**Counter balancing schemes**• Reverse order of group presentation (AB vs BA); but remember ANOVA usually have more conditions • Gets complicated quickly…. • Consider a within subjects design IV that has 4 levels, how many different orders of conditions? • ABCD, ABDC,**Counter balancing schemes**• Reverse order of group presentation (AB vs BA); but remember ANOVA usually have more conditions • Gets complicated quickly…. • Consider a within subjects design IV that has 4 levels, how many different orders of conditions? • ABCD, ABDC, ACBD,**Counter balancing schemes**• Reverse order of group presentation (AB vs BA); but remember ANOVA usually have more conditions • Gets complicated quickly…. • Consider a within subjects design IV that has 4 levels, how many different orders of conditions? • ABCD, ABDC, ACBD, ACDB, ADBC, ADCB • BCDA, BCAD, BDAC, BDCA, BACD, BADC • CDAB, CDBA, CABD, CADB, CBAD, CBDA • DABC, DACB, DBAC, DBCA, DCAB, DCBA**Counterbalancing**• Too confusing…more simple way is a Latin Square design • Number conditions = number of orders • Each condition appears at each position only once C D A B 1st condition 2nd condition 3rd condition 4th condition