PSYC512: Research Methods Lecture 12

1. PSYC512: Research MethodsLecture 12 Brian P. Dyre University of Idaho PSYC512: Research Methods

2. Lecture 12 Outline • Within vs. between subjects designs • Condition ordering • Subject assignment • More on research design • Using 2 or more groups • Multifactor research – using two or more independent variables • Experimentation vs. Quasi-experimentation PSYC512: Research Methods

3. Features of Experimental Designs • Within-subjects design • Most efficient-requires fewest subjects • Statistically powerful: each subject acts as their own control – eliminates error variance due to subject variables • PROBLEM: Carry-over and ordering effects • Between-subjects design • Randomized Groups: if subject variables are not expected to covary with the measure • Matched Groups: if subject variables are expected to covary with the measure (some argue randomization is still better) PSYC512: Research Methods

4. Controlling for Carry-over and Order Effects • Balance carry-over and order effects across treatments • Randomization and Blocked Randomization • Counterbalancing of N treatments • Complete: present each subject with a unique order and use every possible order: requires N! orders/Ss • Partial (Latin Square): present each subject a unique order carefully chosen from a subset of all possible orders PSYC512: Research Methods

5. Constructing a Latin Square for N treatments • Randomly assign each treatment a number • Five treatments (A, B, C, D, E) are assigned (1, 2, 3, 4, 5) • Determine First Order using: 1, 2, N, 3, N-1, 4, N -2… A, B, E, C, D PSYC512: Research Methods

6. Constructing a Latin Square for N treatments • Fill in N-1 more orders by incrementing down and “wrapping”. For odd N, also use reverse orders N = 4 (4 Ss) N = 5 (10 Ss) A B D C A B E C D D C E B A B C A D B C A D E E D A C B C D B A C D B E A A E B D C D A C B D E C A B B A C E D E A D B C C B D A E PSYC512: Research Methods

7. Controlling for Carry-over and Order Effects • Minimize carry-over and order effects across treatments • Practice Sessions • Breaks • Make the treatment order a between-subjects independent variable • Creates a mixed design PSYC512: Research Methods

8. Controlling for Subject Variance in Between-subjects Designs • Random Assignment • Ensures subject characteristics don’t correlate with treatments, if you have enough subjects • Matched Groups • Assess participants on one or more characteristics that might correlate with the DV • Distribute like-participants to groups • Both methods attempt to equate the groups and treat subject variance as error variance PSYC512: Research Methods

9. Choosing the Number and Type of Independent Variables • Single factor (IV) vs. multiple factors (IVs) • Single factor designs are simpler but more limited in scope • Multiple factors allow for examining the synergistic effects of variables (interactions) • Parametric vs. Non-parametric Designs • Parametric: IV is quantitative (ratio or interval scale) • Non-parametric: IV is qualitative (nominal or ordinal scale) PSYC512: Research Methods

10. Factorial Research Designs • Used to assess the effects of 2 or more independent variables (factors) on your dependent variable • Using multiple IVs in one experiment is • more economical (particularly for within-Ss) • provides more information • Main effects of each IV (separate effects of each IV) • Interaction between the IVs (synergism): effect of one variable changes across the levels of the other variable PSYC512: Research Methods

11. Factorial Research Designs: Possible Outcomes of a 2 x2 Design No effects Main Effect A Only Main Effect B Only B1, B2 B2 DV DV DV B1, B2 B1 A1 A2 A1 A2 A1 A2 Factor A Factor A Factor A PSYC512: Research Methods

12. Factorial Research Designs: Possible Outcomes of a 2 x2 Design Main Effects for A and B, No Interaction Main Effects for A and B, Interaction (ordinal) Main Effect for A only, Interaction (disordinal) Main Effect for B only, Interaction (ordinal) B2 B2 B2 B2 B1 B1 DV DV DV DV B1 B1 A1 A2 A1 A2 A1 A2 A1 A2 Factor A Factor A Factor A Factor A IMPORTANT: always interpret the highest order effect or interaction PSYC512: Research Methods

13. Example Factorial Experiment: Perceiving Heading • An experiment on perceiving the heading direction during steering control vs. riding and pointing PSYC512: Research Methods

14. Example Factorial Experiment: Perceiving Heading • IV1: observer’s task • Steering control: Steer so that you appear to be moving straight ahead • Pointing: Point in the direction towards which you perceive yourself to be moving while “riding” • IV2: simulated velocity of observer movement • Three levels: 50, 100, and 200 ms-1 • Dependent Variable (measure): RMS (root-mean-squared) heading error = sqrt(Serror/N) PSYC512: Research Methods

15. Results • Significant Effects • Task Main Effect • Interaction between task and velocity • IMPORTANT! ALWAYS INTERPRET HIGHEST ORDER EFFECT PSYC512: Research Methods

16. Specialized Research Designs • Combining between-subjects and within-subjects factors in research design – mixed designs • Combining experimental and correlational designs (Analysis of covariance or ANCOVA) • Quasi-Experimental Designs • Pretest-posttest designs • Developmental designs (Longitudinal or cross-sectional) PSYC512: Research Methods

17. Combining Between-Subjects and Within Subjects Designs • The Mixed Design • Also known as “split plot” • Groups of subjects each receive a unique level of the between Ss variable(s) and all levels of the within Ss variable(s) PSYC512: Research Methods

18. Mixed Three-factor Design: Adding Presentation Order as a Factor to Account for Differential Carryover Presentation order Variable (between) Group A (control then pointing) Group B (pointing then control) Found 3-way interaction that indicates differential carryover (asymmetrical transfer) PSYC512: Research Methods

19. Combining Between-Subjects and Within Subjects Designs • Example of a Mixed Design • Task order manipulated between Ss • Group A (control then pointing) • Group B (pointing then control) • 3-way interaction indicates differential carryover effect (asymmetrical transfer) PSYC512: Research Methods

20. Next Time… • Quasi-experimental designs • Small-n designs PSYC512: Research Methods