Quasi-experimental Designs and Applied Research

1. Quasi-experimental Designs and Applied Research Lawrence R. Gordon Psychology Research Methods I

2. Applied Research • Long history in American Psychology • May be “beyond the lab,” but not always • Addresses recognizable problems, • But often advances basic knowledge, too. • E.g. “the cognitive interview” • See example p. 318-9 in text • Further example ...

3. Applying cognitive theory in public health investigations: Enhancing food recall with the Cognitive Interview (CI) • Fisher & Quigley, 1992 • CI based on principles of memory retrieval • 26 undergrads in two conditions • Invited to eat party foods (34 different) • Videotaped • 4-14 days later, recall foods via • No-instruction interview (n=13) • Cognitive interview (n=13)

4. Results - Fisher & Quigley... POINTS: Over twice the recall for CI Basic or applied? Experimental or not?

5. Drawing Causal Conclusions • 3 conditions of causality 1. IV precedes DV 2. IV and DV covary 3. No other possible explanations (all other factors are controlled) • These conditions may be satisfied by true experiments

6. True Experiments • Independent variable is manipulated • At least 2 levels of IV (experimental and control conditions) • Random assignment • Design schematic: IVDV • Exper: {O1} T O2 • R • Control: {O1} O2 • “R”  Random assignment • All other variables are held constant (controlled)

7. Quasi-experiments • Def.: “quasi-”: 1. Almost or somewhat (something is not quite right!); 2. Adjective Elmer Fudd uses to describe Bugs Bunny • Independent variable may not be manipulated • May not have more than one level of the IV (no control group) • May not have a random sample • May not have random assignment • Conditions of causality?

8. Why use quasi-experiments? • Sometimes random samples are not available, random assignment is impossible, control groups are not available or not equivalent • When is this the case? • Often with research in the real world (“application” studies) • Program evaluation • LET’S LOOK AT SOME NON-EXP DESIGNS 

9. One-group posttest only • Design ( “T” = “treatment”, “O” = observation) Schematic: T O • Example 

10. Teaching Demonstrations • Lawson & Reardon, 1997 • Do teaching demonstrations aid understanding? E.g., Systematic desensitization (SD) • Used “case of eraser phobia” • After the demonstration, students agreed that it increased their understanding of SD • mean of 4.41 on a scale of 1-5 • Problems? Revisions?

11. Posttest only with nonequivalent groups • Design schematic: “Exper”: T O Control: O • EXAMPLE 

12. Coping Self-efficacy and Distress after Disaster • Benight et al., 1997 • Does one’s belief that one can successfully cope with problems predict better adjustment following disaster? • Interested in men with HIV, looking at distress and coping following Hurricane Andrew (1992) vs. a non-HIV comparison group • All measures taken after the hurricane

13. CSE and Distress (cont.) • Measured CSE by asking about ability to deal with relevant issues post-hurricane (avoiding looting, finding food and shelter, handling insurance companies) • Measured PTSD symptoms and general distress symptoms • Compared male volunteers with HIV to community volunteers without HIV; the groups were comparable except for HIV

14. CSE and Distress (cont.) • Results showed that higher CSE was related to lower levels of PTSD and general distress • HIV: r = -.70, p < .001; r = -.68, p < .001 • Com: r = -.55, p < .001; r = -.53, p < .001 • Relationships slightly but not significantly stronger in HIV group • Concluded that distress to natural disaster can be alleviated in HIV populations if they feel they can cope • Problems? Limitations?

15. One-group pretest posttest • Design schematic: “Exper”: O1 T O2 • EXAMPLE 

16. Body Image after Weight Loss Efforts • Foster, Wadden, & Vogt, 1997 • Is engaging is a weight loss treatment program enough to promote a more positive body image in obese women? • All participants were involved in a clinical trial studying weight loss treatments (diet, aerobic training, strength training, combos) • Measured body image on 2 validated scales before and after 48 weeks of treatment

17. Body Image after Weight Loss (cont.) • Compared the mean body image scores before and after treatment, t(39) = 6.3 and 7.2, ps = .0001 • Body image increases were not significantly related to body weight decreases • Concluded that involvement in the treatment program, with or without weight loss, promoted better body image • Problems? Revisions?

18. Pretest posttest with nonequivalent groups • Design schematic: “Exper”: O1 T O2 Control: O1 O2 • EXAMPLE 

19. Reducing Recidivism • Henning & Frueh, 1996 • Will a cognitive self-change (CSC) program reduce the rates of re-offense in prisoners who are released? • CSC program: goal is to alter cognitive distortions that may lead to antisocial behavior (minimization, taking role of victim themselves, denial of responsibility)

20. Reducing Recidivism (cont.) • CSC: group treatments 3-5x per week to help recognize and alter distortions in past behavior (for about 10 months) • 55 offenders volunteered for CSC • 141 offenders did not volunteer • Recidivism rates compared • 51% CSC participants, 71% nonparticipants • 2(1) = 7.00, p < .01 • Problems? Revisions?

21. Interrupted time series • Design schematic: “Exper”: O1 O2 O3 O4 O5 T O6 O7 O8 O9 O10 • EXAMPLE 

22. Effects of Disease Diagnosis on Feelings of Hopelessness • Tibben et al., 1997 • Is the hopelessness of being diagnosed with disease long-lasting? • Examined feelings of hopelessness in men at risk for Huntington’s disease before genetic testing and after testing positive for the gene mutation

23. Disease and Hopelessness (cont.)

24. Disease and Hopelessness (cont.) • Results appear to show that participants showed elevated levels of hopelessness immediately after diagnosis, and the levels remained higher than the levels before diagnosis • Problems? Revisions? • Let’s see the rest of the design 

25. Interrupted time series with nonequivalent groups • Design schematic: “Exp”: O1 O2 O3 O4 O5 T O6 O7 O8 O9 O10 Con : O1 O2 O3 O4 O5 O6 O7 O8 O9 O10

26. Disease and Hopelessness II • Based on Tibben et al., 1997 • Compared the hopelessness scores of men before and after testing positive for the Huntington’s Disease gene mutation with the hopelessness scores of men at risk for Huntington’s Disease who were not tested

27. Disease and Hopelessness II (cont.)

28. Disease and Hopelessness II (cont.) • Results show that the men diagnosed with the disease showed higher levels of hopelessness at all times after diagnosis than did men not tested for the disease • Conclusions? • Problems? Revisions?

29. Program Evaluation • “Reforms as experiments” (Campbell,1969) • Kinds of program evaluation • Needs analysis • Formative (“process”) evaluation • Summative (“outcome”) evaluation • Cost-effectiveness analysis • Difficulties / ethical issues • Consent, confidentiality • Perceived injustices • Working with stakeholders

30. Overall Summary • Quasi-experimental designs can be employed when true experiments are impossible • The flaws in these designs do not fully allow for the disqualification for all alternative explanations for the results • Cause-effect statements can not be made with high levels of certainty