Outline

1. Outline 1. Definition of Complex Designs 2. Some important terms 3. Advantages of complex designs • Testing theories • Resolving contradictions • Establishing the external validity of a result 4. Analysis in the presence of an interaction 5. Analysis when there is no interaction 6. Natural Groups designs 7. Ceiling effects

2. A complex design is one in which more than one variable is manipulated at the same time. ‘Complex’ here does not mean ‘difficult to understand.’ Definition of Complex Design

3. Factorial design The most useful kind of complex design is the factorial experiment, in which each variable is manipulated at all levels of each other variable. Some important terms

4. A1 A2 B1 A1B1 A2B1 B2 A2B2 A1B2 The basic 2 X 2 factorial design

5. Training duration Short Long Motor - Short Motor - Long Motor Task Abstract - Short Abstract - Long Abstract The basic 2 X 2 factorial design

6. Factorial design Main effect The effect of one variable in a multi-variable design, ignoring all other variables Some Important Terms

7. A1 A2 B1 B1 A1B1 A2B1 B2 B2 A2B2 A1B2 Comparing these two means gives us the main effect of B A1 A2 Comparing these two means gives us the main effect of A The basic 2 X 2 factorial design

8. Factorial design Main effect Simple main effect The effect of one variable in a multi-variable design, observed at one level of a second variable. Some Important Terms

9. A1 A2 B1 A1B1 A2B1 B2 A2B2 A1B2 Here, A1B1 – A1B2 gives the SME of B at A1 SME = simple main effect

10. A1 A2 B1 A1B1 A2B1 B2 A2B2 A1B2 Here, A2B1 – A2B2 gives the SME of B at A2 SME = simple main effect

11. A1 A2 Here, A1B1 - A2B1 gives the SME of A at B1 B1 A1B1 A2B1 B2 A2B2 A1B2 SME = simple main effect

12. A1 A2 B1 A1B1 A2B1 Here, A1B2 – A2B2 gives the SME of A at B2 B2 A2B2 A1B2 SME = simple main effect

13. Factorial design Main effect Simple main effect Interaction an interaction occurs when the effect of one variable varies at levels of another variable. thus, when there is an interaction between A and B, the SME of A will vary across levels of B (and vice versa). Some important terms

14. A1 A2 B1 400 500 B2 575 425 25 75 SME of B at A2 SME of B at A1 These numbers show observations on some dimension (such as reaction time in milliseconds) The SME of B is much smaller for A1 than for A2 – that’s an interaction of variables A and B

15. Coffee No Coffee Cereal 40 100 60 10 50 60 40 10 SME of Cereal with Coffee No Cereal SME of Coffee is larger with Cereal than without SME of Cereal Without Coffee The SME of Cereal is larger with Coffee than without. DV = a measure of mood quality

16. Godden & Baddeley (1975) Wanted to test context-dependent learning hypothesis Divers learned a list of words, then recalled the list. Each step could be either on land or under the water. Interaction – an example

17. Learning On deck In pool On deck Recall 13.5 8.6 In pool 11.4 8.4 DV = # words recalled out of 15 Is it better to learn on deck or in the pool? It depends upon whether you will have to recall on deck or in the pool. Godden & Baddeley (1975)

18. Factorial design Main effect Simple main effect Interaction Analytical comparisons Tests that determine what is producing a main effect E.g., is B1 different from B2? Is it different from B3? Some important terms

19. Factorial designs Main effect Simple main effect Interaction Analytical comparisons Simple comparisons tests that determine what is producing a simple main effect E.g., is B1 different from B2 at level A1? Is B2 different from B3 at A2? Some important terms

20. Analytical comparisons: Tests that determine what is producing a main effect Simple comparisons: tests that determine what is producing a simple main effect Some important terms

21. Testing theories Complex Designs allow tests that are: more powerful more economical, and less likely to be correct by chance Advantages of complex designs

22. More powerful Variability in your data is either random (E) or associated with a systematic source (T) In a factorial design, associating some variance with the interaction reduces the random error. A systematic source Advantage: Testing theories

23. More powerful More economical Better use made of subjects’ time – test several hypotheses at once. Advantage: Testing theories

24. More powerful More economical Less likely to be correct by chance More complex predictions are less likely to be correct by chance, since there are more ways they can go wrong. Advantage: Testing theories

25. Testing theories More powerful More economical Less likely to be correct by chance Resolving contradictions Advantages of complex designs

26. Testing theories Resolving contradictions Results from different labs sometimes conflict because different researchers unwittingly choose different levels of variables they are not manipulating. If those variables can be identified, they can be manipulated in a new study with a factorial design. Advantages of complex designs

27. Arousal High Low High Difficulty 60 40 Low 50 80 DV = accuracy (% correct) If one lab used a difficult task and another used an easy task, researchers would draw opposite conclusions about the effect of arousal.

28. Testing theories Resolving contradictions Establishing external validity of a result When no interaction is found, it’s safer to generalize effects of eachvariable across levels of the other variable. But don’t generalize the effect of A beyond the levels of B used in the experiment. Advantages of Complex Designs

29. Don’t generalize effect of A beyond levels of B. E.g., if A = stimulus quality and B = stimulus size Levels of B = 2, 4 and 10 cm in our experiment We find no interaction We can generalize the effect of A to 7 cm stimuli, but not to 20 cm stimuli. Advantages of complex designs

30. We don’t know what’s going on in this region – so we shouldn’t say anything about it 2 4 10 7 20 Degraded Clear

31. Once we detect an interaction, the next step is to ‘decompose’ the interaction. That is, compare SMEs of A at levels of B (or vice versa). Which SMEs we examine should be dictated by theory. Analysis when interaction occurs

32. When a variable A does not interact with other variables in the design, you analyze the main effects of A. As before, use simple comparisons to test for differences between pairs of means for levels of A. Analysis when no interaction occurs

33. Simple comparisons Yes More than 2 means? Yes No Finished SME of A at B1? No SME of A at B2? Yes Does A interact with B? Simple comparisons Yes More than 2 means? Yes No Main effect of A? No Finished No Finished Main effect of B?

34. Pratkanis et al. (JPSP 1988) The ‘sleeper effect’ The passage of time improves the effect of a persuasive message This occurs only if message is accompanied by a discounting cue – a cue that causes you to distrust the persuasive message Complex design example

35. Persuasive message: “Dr. Smith’s research shows that orange juice consumption can reduce cholesterol.” Discounting cue: “This research was funded by Tropicana.” Pratkanis et al. (1988)

36. Why does sleeper effect occur? One model: it’s caused by dissociation – over time, link in memory between persuasive message and discounting cue gets weaker. Pratkanis et al. tested this idea Pratkanis et al. (1988)

37. Basic paradigm: People are given a persuasive message about an object or product + a discounting cue Later, they are asked to rate the object or product Pratkanis et al. (1988)

38. Pratkanis et al. used two independent variables Delay Was opinion rating given immediately or six weeks later? Pratkanis et al. (1988)

39. Pratkanis et al. used two independent variables Delay Order Was discounting cue presented before or after persuasive message during original session? Pratkanis et al. (1988)

40. This is the sleeper effect – found when we look at only the variable delay Message is rated more persuasive (higher score) after delay of 6 weeks 15 10 5 0 -5 0 6 wks Pratkanis et al. (1988)

41. There’s no main effect of the variable order (discounting cue given before or after persuasive message during original session) 15 10 5 0 -5 Before After Pratkanis et al. (1988)

42. This interaction shows that we get the sleeper effect only when the cue is presented after the persuasive message Dissociation model can’t explain this 15 10 5 0 -5 0 6 wks cue before message cue after message Pratkanis et al. (1988)

43. The design of this experiment allowed Pratkanis et al. to test the interaction hypothesis The interaction observed – sleeper effect occurred only when discounting cue came after persuasive message – is strong evidence against the dissociation theory of the sleeper effect. Pratkanis et al. (1988)

44. Natural groups designs Designs in which experimenter does not assign subjects to groups Groups are naturally occurring It is very risky to draw conclusions about why such groups differ in performance on some task. Natural groups designs

45. For example: people who are mentally active into their later years are less likely than people who are not mentally active to suffer Alzheimer’s Type Dementia (ATD). Why? Having a healthy brain makes you active? Being active gives you a healthy brain? Natural groups designs

46. A natural groups design is really a correlational study, not an experiment! Thus, in the ATD case, severity of the disease is correlated with mental activity. Dividing the subjects into two groups (With and Without ATD) doesn’t change this. But you can still make an argument for cause… Natural groups designs

47. Halpern & Bower (1982) Studied memory for musical notation People with musical training recall notation better than people without musical training. Is this because of the training? Or are people with better memories drawn to musical training? Natural groups designs

48. Theory: musical training gives musicians the ability to “chunk” notation. A chunk is a unit formed from several smaller pieces, on the basis of knowledge. Examples of “chunks:” BMW CBC IBM NHL SOA ISI JND Halpern & Bower example

49. Halpern & Bower compared natural groups: people with and without musical training used two sets of musical notation: one with structure (so notation stimuli could be chunked) one without structure Halpern & Bower example

50. Note that this design allows us to test the prediction of an interaction: Group by structure Halpern & Bower example % Structured Unstructured Musicians Non-musicians