Single-Variable, Independent-Groups Designs

1. Single-Variable, Independent-Groups Designs Graziano and Raulin Research Methods: Chapter 10 Graziano & Raulin (2000)

2. Experimental Design • Tests one or more hypotheses about causal effects of the independent variable (IV) • Includes at least two levels of the IV • Randomly assigns participants to conditions • Includes specific procedures for testing hypotheses • Includes control for the major threats to internal validity Graziano & Raulin (2000)

3. Variance • Variance is necessary in any research study • Without variance, there is nothing to test • Variance was defined in Chapter 5 • The purpose of research design is to control unwanted sources of variance to allow us to evaluate the effects of the independent variable Graziano & Raulin (2000)

4. Forms of Variance • Systematic between-groups variance • Experimental variance (due to the IV) • Extraneous variance (due to confounding variables) • Nonsystematic within-groups error variance • Due to chance factors and individual differences • We analyze the results of our study using the F-test (ANOVA) • Ratio of between-groups variation to within-groups variation Graziano & Raulin (2000)

5. Controlling Variance • Maximizing experimental variance • Make sure that there are real differences between the groups (using a manipulation check) • Controlling extraneous variance • Make sure the groups are as similar as possible at the start of the study • Therefore, the only difference is the IV manipulation • Minimizing error variance • Control with careful measurement or with special designs (e.g., correlated-group designs) Graziano & Raulin (2000)

6. Nonexperimental Designs • Do not include the critical controls of experimental designs • May still be used, but caution is necessary • Four designs covered in this section • Ex post facto design • Single-group, posttest-only design • Single-group, pretest-posttest design • Pretest-posttest, natural control-group design Graziano & Raulin (2000)

7. Ex Post Facto Design • Design:(naturally occurring event) measurement • No manipulation or pretest measurement • A very weak design • What we do when we try to figure out, after the fact, what caused something to happen • Not good science • Controls none of the potentially confounding variables Graziano & Raulin (2000)

8. Single-Group, Posttest-Only Design • Design:Group A Treatment Posttest • A manipulation, but no pretest or control group to help us evaluate the manipulation • We tend to use an implicit control group (what we think would have happened if there had been no manipulation) • Even with the manipulation, virtually no control over confounding variables Graziano & Raulin (2000)

9. Single-Group, Pretest-Posttest Design • Design:Group A Pretest Treatment Posttest • The pretest allows us to see if a change actually occurred • The pretest documents change, but factors other than the treatment could have accounted for the change • History, maturation, regression to the mean, etc. Graziano & Raulin (2000)

10. Pretest-Posttest,Natural Control-Group Design • Design:Group A Pretest Treatment PosttestGroup B Pretest No Treatment Posttest • Like an experiment except that participants are not randomly assigned to the groups • A reasonably strong design except that it does not control for selection • Selection could be a powerful confounding factor in many studies Graziano & Raulin (2000)

11. Experimental Designs • Meet all criteria for an experiment • Provide more powerful tests of hypotheses • Designs discussed in this chapter • Randomized, posttest-only, control-group design • Randomized, pretest-posttest, control-group design • Multilevel, completely randomized, between-subjects designs • Solomon’s four-group designs Graziano & Raulin (2000)

12. Randomized, Posttest-Only,Control-Group Design • Design:R Group A Treatment PosttestR Group B No Treatment Posttest • Key element is random assignment to groups • Random assignment controls for selection • Other confounding variables are controlled by comparing the treatment and no treatment groups Graziano & Raulin (2000)

13. Randomized, Pretest-Posttest,Control-Group Design • Design:R Group A Pretest Treatment PosttestR Group B Pretest No Treatment Posttest • Adding a pretest allows us to quantify the amount of change following treatment • Also allows us to verify that the groups were equal initially • A strong basic research design, with excellent control over confounding Graziano & Raulin (2000)

14. Multilevel, Randomized,Between-Subjects Design • Design:R Group 1 Pretest Treatment 1 PosttestR Group 2 Pretest Treatment 2 PosttestR Group N Pretest Treatment N Posttest • May or may not include a pretest • Multi-group extension of the basic experimental designs Graziano & Raulin (2000)

15. Solomon’s Four-Group Design • Design:R Group A Pretest Treatment PosttestR Group B Pretest No Treatment PosttestR Group C Treatment PosttestR Group D No Treatment Posttest • Combines two basic experimental designs • Allows us to assess whether there is an interaction between the treatment and the pretest Graziano & Raulin (2000)

16. Statistical Analysis Issues • If the data are nominal, use chi-square • If the data are ordinal, use the Mann-Whitney U-test • If the data are interval or ratio • If there are only two groups, a t-test of the posttest measures will test the hypothesis • More complex designs will require an ANOVA Graziano & Raulin (2000)

17. Analysis of Variance (ANOVA) • Evaluates differences in group means • It does this evaluation by comparing different variance estimates (termed mean squares) • The F statistic is a ratio of • the mean square between-groups • the mean square within-groups • The larger the differences between the group means, the greater the F value Graziano & Raulin (2000)

18. Specific Means Comparisons • A significant F-test means that at least one group is significantly different from at least one other group • If you have more than two groups, you have to do follow-up tests to see which groups differ • Specific means comparisons can be • Planned comparisons (testing specific hypotheses) • Post hoc tests (just looking to see which groups differ) • Easy to do with data analysis computer programs Graziano & Raulin (2000)

19. Other Experimental Designs • Other experimental designs covered in later chapters • Correlated-groups designs (Chapter 11) • Within-subjects designs • Matched-subjects designs • Factorial designs (Chapter 12) • Many variations on factorial designs are possible Graziano & Raulin (2000)

20. Summary • Research is designed to measure and control sources of variance • There are a variety of non-experimental and experimental designs available • Experimental designs have two elements • Random assignment of participants to conditions • A control group Graziano & Raulin (2000)