Outline

1. Outline • Case Studies & Testimonials • Case studies as evidence • The placebo effect • The vividness issue • Correlation & Causation • Third variable problem • Directionality • Selection bias

2. Case studies & testimonials • Case studies • in-depth analyses of particular individuals • cannot be used to establish causal relations • Testimonials • positive reviews of a person or product

3. Case studies as evidence • Historically important in Psychology • Used by Freud • Psychophysics • Complex system can go wrong in complex ways – adequate description may require such lengthy observation that case study is only appropriate approach, at least initially

4. Case studies as evidence • Most useful in early stages of research in any area • Useful for development of theories & hypotheses • Observation may lead to identification of relevant variables that are controlling or influencing behavior - leading to theory building

5. Case studies as evidence • Less useful for testing theories • Case studies cannot rule out alternative hypotheses • Testing theories requires a manipulation – changing conditions under which people perform a task to see if variation in performance predicted by a theory occurs • That process lets you pit one theory against another, and see which (if either) wins

6. Case studies as evidence • The critical idea here is that there can be competing accounts of why something happened. • Our task as scientists is to compare competing accounts and select one.

7. Case studies as evidence • If we are to adjudicate between competing theories, we have to put them in jeopardy. • Jeopardy means that we give all the competing theories a chance to be wrong. • If one of them isn’t wrong in that situation, but the others are, then we reject those others and adopt the one that survived the test. • Manipulation based on theoretical predictions creates the required situation

8. Case studies as evidence • We have to discriminate between alternative accounts • If there isn’t a manipulation that lets you adjudicate between two theories – that is, if they don’t make different predictions about performance – then you don’t have two theories, you only have one. • So, whenever you truly have two theories, you must be able to test them. But not with case studies.

9. Case studies as evidence • The problem with case studies is that they don’t let you create the jeopardy situation • One reasonable account of any improvement in the condition of a case study subject is the placebo effect. • A second issue is that observations in case studies can be biased by the vividness effect.

10. Placebo Effect • Placebo effect: tendency to report that a treatment has helped, regardless of whether it has a real therapeutic effect. • Control group gets a fake treatment • Experimental group gets the real treatment

11. Placebo Effect • If the two groups show the same amount of improvement, then the improvement is due to placebo effect. • If control group shows any improvement, part of treatment effect is placebo effect.

12. Placebo effect • This is about the power of belief: e.g., -physicians say, when a new drug is available, “Hurry up and use it while it still works” • Hawthorne Effect – people’s performance got better when other people were being observed – possibly because “someone cares” • Turing Test – a computer program is intelligent if you can’t tell whether you’re talking to human or computer; problem – people see meaning

13. Bower (1996): Placebo effect of Prozac twice as large as effect of the drug itself. Greenberg & Fisher The limits of biological treatments for psychological distress: comparisons with psychotherapy & panacea (1989) From placebo to panacea: Putting psychiatric drugs to the test (1997) Placebo effects

14. Benefits of psychoactive drugs are largely due to placebo effects Brown (SciAm 1998): expectations have real, measurable effects – prescribe placebos! Placebo effects

15. Placebo effects are widespread, so they are always present as an alternative account of why someone got better when they got a therapy. You cannot rule out placebo effects in case studies. People who got the benefit of placebo effects may offer testimonials, not knowing that it wasn’t their therapy that made the difference. Placebo effects- conclusions

16. The Vividness Issue • Retrieval from memory is influenced by vividness. • E.g.: which is safer, travel by car or travel by airplane? • Statistically, travel by plane is much safer, but many people think it is more dangerous. Why? • One account: reports of airplane crashes are vivid, so easily retrieved from memory – ease biases estimate of whether flying is safe.

17. The Vividness Issue • Stanovich: • <680 car, truck, & motorcycle accident deaths/wk in the U.S. • But (Stanovich says) people don’t call for action because they don’t recognize the problem • Lots of reports of one or two deaths aren’t as vivid as one report of an airplane crash with hundreds of deaths

18. Criticism: • In the case of plane vs. car travel, Stanovich doesn’t know that the traffic safety problem is not recognized – that’s just his opinion. • He only knows that people don’t respond as dramatically to traffic deaths as they would to similar numbers of plane crash deaths • But there’s a difference between not calling for investigations and not responding…

19. Criticism • Gerald Wilde, Queen’s University – homeostatic theory of risk: people set an acceptable risk rate for activities like driving • If driving conditions change in such a way as to move drivers away from their preferred rate, they adjust their behavior to move back to that rate.

20. Criticism • e.g., better brakes? Drive faster. • speed limit reduced? Pay less attention • people are sensitive to driving risks – their behavior does vary in ways that are sensitive to risks • but they may not be aware of that variation in their behavior or be able to articulate those risks as statistics.

21. Criticism • Perhaps professors (including Stanovich) over-value the ability to articulate knowledge and undervalue the ability to modulate behavior without such knowledge.

22. In the new (8th) edition of his book, Stanovich has added a qualification to his comments about vividness, acknowledging that he uses vivid examples himself. He now says that he uses vivid examples to make his points memorable, but he also provides citations to scientific evidence for those points. The latter constitute “proof” of those points. The Vividness Issue

23. Stanovich says that he uses vivid examples only because they are memorable – so in his mind, presumably, the reader will be persuaded by the cited scientific studies, not the vivid examples But will this be the case? Will his distinction between “illustration” and “proof” be as important to readers as it is to him? The Vividness Issue

24. The Vividness Issue • Let’s distinguish between the ideas involved: 1. Do vivid examples prove claims? 2. Are ordinary people unsophisticated consumers of evidence? 3. Does vividness play any useful role in our thinking?

25. 1. Do vivid examples prove claims? • No. As a technical point, Stanovich is quite right on this score.

26. 2. Are ordinary people unsophisticated consumers of evidence? • No. As Wilde’s homeostatic theory of risk, and the evidence he reports, suggest, ordinary people respond rationally to changes in conditions, though they may not be able to articulate those changes or the basis for their responses. • The “wisdom of crowds”

27. 3. Does vividness play any useful role in our thinking? • Our liking for vivid examples – which Stanovich himself takes advantage of – reflects a basic fact about human cognition: • Individual examples help humans to comprehend events or ideas that are otherwise beyond understanding. As Stalin said, “1 death is a tragedy. 1 million deaths is a statistic.”

28. 3. Does vividness play any useful role in our thinking? • As Stalin said, “1 death is a tragedy. 1 million deaths is a statistic.” • To illustrate this point, let’s use as an example the Holocaust…

29. The Holocaust • 6 million Jews, and many other people (Gypsies, gays) were murdered in Germany and German-occupied territories during World War II. • A terrible thing – but how terrible? • We don’t have any measuring scale that lets us answer that question. • Does it help to learn about one victim?

30. Irène Némirovsky • Russian-born Jewish writer • Moved to France as a child, in 1919, following Russian Revolution • Published first novel in 1929; celebrated as a brilliant author • She was refused French citizenship in 1938. • Left Paris after German occupation of France in 1940, went to live in a rural village with husband & 2 daughters • Continued writing

31. Paper was scarce, so she wrote in tiny letters, with a magnifying glass Wrote a novel about the German occupation of France Arrested by French police in July 1942 in front of her two young daughters. Told the daughters she was going away on a trip Irène Némirovsky

32. Died of typhus in a German concentration camp one month later Husband arrested, died in Auschwitz gas chamber in November 1942 Daughters guarded by Catholic nuns & priests through the war, and provided for by Irène’s publisher & a friend during and after the war Irène Némirovsky

33. Daughters survived the war, kept a suitcase containing their mothers’ last writings. 30 years later, daughter Dianne spent 20 years transcribing Irène’s novel Suite Francaise, working with a magnifying glass. Book was published in France in 2004, and a tremendous success – though Némirovsky had been long forgotten. Published in Canada in April 2007 Irène Némirovsky

34. So, which helps you understand the horror of the Holocaust better – the statistic, “6 million murdered” – or the story of Irène Némirovsky? Both are important, but in different ways So, vividness can be an important aid to thought And if vivid examples are filtered through many people’s evaluations, perhaps that process produces a useful conclusion Vividness

35. Basic point: Correlation does not imply causation Goldberger example: cause of pellagra Goldberger used a manipulation to test causal accounts He put the germ theory in jeopardy – and showed that it was wrong He also put his diet theory in jeopardy – and showed that it was right. Correlation and Causation

36. Correlation and causation • Third variable problem • Directionality • Selection bias

37. Relationship between two variables A and B might be produced by a third variable which influences both. E.g., correlation between # of births in German cities and # of storks in German cities, in 19th century. # Families  # houses  # chimneys  # storks Third variable problem

38. Correlation and Causation • But note the example Stanovich uses: the relation between SES and going to university. • Correlation is still there even when academic aptitude effects have been removed statistically. • S says this finding “strikes at the heart of the merit-based goals of our society.” • So, he’s concluding that SES controls university attendance, even though he admits that we don’t know the reason for the correlation.

39. Poor reading example: researchers once thought that poor reading caused by improper eye-movements. We now know that causation works in the other direction: poor reading skill produces impaired eye-movements. Directionality

40. Self-esteem and academic achievement – lots people have claimed that low self-esteem leads to low academic achievement. In self-esteem case, there is evidence that poor academic achievement leads to low SE and good academic achievement leads to high SE (other things being equal) Directionality

41. relationship between subject and environmental variables arises because people with different characteristics select different types of environments. E.g., Arizona and respiratory illness. SAT scores & teacher salaries example Selection bias