9 – Judgment Most people do not understand chance and risk, often with dire consequences.

1. 9 – Judgment Most people do not understand chance and risk, often with dire consequences. These errors reflect misconceptions about probability. These misconceptions are revealed by people’s response to the following questions:

2. The Banker Question Linda is 31, single, outspoken, and highly educated. Which one of the following statements is more likely to be true?  She is a banker.  She is a banker and a democrat. Correct Answer Banker. chance of just one  chance of both conjunction fallacy chance of both  chance of just one (Tversky & Kahneman, 1983)

3. Hospital Question At Big City Hospital, about 50 babies are born daily. At Small Town Hospital, about 10 babies are born daily. At each hospital, someone counts the number of girls and boys born each day. If at least 60% of babies born that day are girls, a pink sticker is placed on the calendar. In a typical year, which hospital will have more pink stickers? ● Big City Hospital ● Small Town Hospital ● About the same Modal Answer About the same Correct Answer Small Town Hospital

4. Explanation smaller sample  less accurate survey Example # of births Probability of at least 60% girls 10 38% 50 10% (Kahneman & Tversky, 1972)

5. Coin Question A fair coin turns up Heads5 times in a row. Which is more likely to happen on the next toss? ● Heads ● Tails ● Neither – chance is 50-50 Correct Answer Neither. A coin has no memory! Common Error Tails gambler’s fallacyevent is less likely if it just occurred

6. More gambler’s fallacy When I was a kid, my best friend was 1 of 3 kids in the family. All 3 kids were boys. Then their mom got pregnant. The mom said, “Since I have 3 boys, I’ll probably have a girl. ” (gambler’s fallacy) Truth = 50% (Though she had a girl.)

7. blank

8. Demo: Imagine rolling a fair die 25 times. Write the outcomes below (1, 2, 3, 4, 5, or 6). ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ ___ Step 2 Write check by each outcome that matches previous outcome 4 4  6 3 3  3  2 On average, there should be 4 repetitions Typical subject < 4 repeitions Why? Gambler’s Fallacy

9. More Gambler’s Fallacy When teachers write multiple-choice tests, repeats are too rare.

10. More gambler’s fallacy May 2012 My insurance company phone rep tells me that I need more hurricane coverage. I point out that hurricane hasn’t hit Florida since 2005. She says that’s why I need more coverage – the chances are getting higher! By her logic, Iowa is really in danger. Note: Cost of insurance coverage has climbed every year since 2005. Weather Facts No hurricane has hit Tampa since they started keeping records in 1851 en.wikipedia.org/wiki/List_of_Florida_hurricanes Data show Florida hurricanes come in clusters. Example. 4 in 2004 and 3 in 2005 That is, chance of hurricane is LESS if it’s been a long time since last one.

11. Hot Hand Fact: In basketball, players often have streaks. Example. A 50% shooter might hit 4 in a row or she might miss 4 in a row. Widespread Belief: Chance of hit after a hit > Chance of hit after a miss False! Basketball statistics  Chance of hit after a hit = Chance of hit after a miss hot hand fallacy = success is more likely after a success than after a failure This fallacy occurs commonly in games of skill and sometimes in games of luck. (Gilovich, Vallone, & Tversky, 1985)

12. Data from 9 players from Philadelphia 76ers Data show that chance of hit is NOT greater after a hit (Gilovich, Vallone & Tversky, 1985)

13. blank

14. Poetry Lover Question Tom is a randomly selected U.S. citizen. He is asked if he likes poetry. He answers “yes.” Which is more likely to be his profession? ● Trucker ● Latin Professor Correct Answer Trucker  Explanation Even if only 0.3% of truckers like poetry, they exceed 1000. Yet fewer than 1000 people are Latin Professors. Thus, answer depends partly on base rate (percent of population with trait X) Wrong answer is example of base rate neglect

15. Disease Question On average, 1 in 191 people has Disease X. There is a blood test for X. Of people who have X, 100% test positive. (True Positive rate = 100%) Of people who don’t have X, 10% test positive. (False Positive rate = 10%) During a routine physical, Homer tests positive for X. Intuitively, what is the chance that Homer has X? ● less than 10% ● about 50% ● higher than 80% Correct Answer 5%

16. Explanation base rate TP FP Chance that Homer has X Disease X 1 in 191 100% 10% 5% Disease Y 1 in 11 100% 10% 50% Disease X 191 1 has X 1 TP 190 do not 19 FPs 1 / 20 = 5% Disease Y 11 1 has X 1 TP 10 do not 1 FP 1 / 2 = 50%

17. Examples blood test base rate TP FP Disease Q 1 in 91 100% 10% Disease R 21 in 91 100% 10% Disease S 1 in 11 100% 0% 1. Quinn tests positive for Q, and Rick tests positive for R. Which event is less likely: Quinn has Q or Rich has R (smaller base rate) 2. Reece has R. Find the chance that he will test positive for R. 100%. True positive rate = chance that person with R will test positive. 3. Stan tests positive for S. What is chance that Stan has S? 100%. The test is perfect.

18. Study Harvard Med students and faculty given Disease Question (with slightly different data). Results AnswerPercent of Subjects 2% (correct) 18% 95% 45% Troubling Implication Patient tests positive for X Doctor tells patient his chance of having X is 95% but truth is 2% Patient chooses risky surgery and dies. (Casscells, Schoenberger, & Graboys, 1978, NEJM)

19. Base rate neglect in academia … What is the chance that a manuscript will be accepted? Academics rely heavily on their judgment of manuscript’s quality. They tend to ignore the journal’s acceptance rate (i.e., base rate). Journal acceptance rates vary dramatically. For APA journals, acceptance rate is between 9% and 69% (2009 data) For all psych journals, range is even greater.

20. blank

21. Conditional Probabilities Disease question requires understanding of conditional probabilities. conditional probability = Chance that A is true if B is true In Disease X question, Chance that Homer has disease X if he tested positive) = 5% Chance that Homer will test positive if he has disease X) = 100% Conditional probabilities are hard to understand. (e.g., Cosmides & Tooby, 1996; Gigerenzer, 2002)

22. Example In 1995, celebrity O. J. Simpson was put on trial for murdering his ex-wife. Simpson had previously battered her. His attorney: “Only 0.04% of male batterers later murder the battered woman.” Translation: If Mr. X once battered Ms. Y, the chance that X will murder Y = 0.04% Relevant data: If Mr. X once battered Ms. Y, the chance that X is murderer if Y was murdered = 93% Neither statistic was presented to jury. (I. J. Good, Nature, 1995, 1996; Strogatz, New York Times, 2010)

23. People better understand risk if they are presented with frequency data. Study Doctors given two versions of the same question Conditional Probability Version The probability that a woman has breast cancer is 1%. If she has breast cancer, the probability of a positive mammogram is 80%. If a woman does not have breast cancer, the probability of a positive result is 10%. Take, for example, a woman who has a positive result. What is the probability that she actually has breast cancer? Natural Frequency Version 10 out of every 1000 women have breast cancer. Of the 10 women with breast cancer, 8 will have a positive result on mammography. Of the 990 women who do not have breast cancer, 99 will still have a positive result. Take, for example, a sample of women who have positive mammograms. How many of these women actually have breast cancer?

24. Results (Hoffrage & Gigerenzer, 1998)

25. Hungry Sailors 10 starving men in a lifeboat agree to sacrifice one man so that the others will live. They put 9 green marbles and 1 red marble in a bag. One at a time, each man chooses a marble. After each marble is chosen, the marble is NOT placed back in the bag. The first player to choose red is the loser. Is it better to choose first or tenth, or does it not matter? Answer: It does not matter

26. Hungry Sailors Version 2 Same as above, except each man replaces his marble before next man chooses. Is it better to choose first or tenth, or does it not matter?   Answer: Tenth

27. The whale ship Essex was sunk by a whale on November 20, 1820. Twenty men escaped in three small craft and wandered the Pacific. After being unable to catch fish, they ultimately resorted to cannibalism by common consent. Three months later there were two boats and five men left. Rescuers found bug-eyed stick figures hunkered over a pile of human ribs, with finger bones stashed in their pockets. (Mark Shone, 2000) A similar scenario took place in Edgar Allen Poe novel The Narrative of Arthur Gordon Pym of Nantucket

28. blank

29. Two Girls Question A woman moves into the neighborhood. When you meet her, you find out that her name is Beth and that she has two kids. Later, you see one of Beth’s kids playing in her yard, and the kid is a girl. What is the chance that Beth’s other child is also a girl? Answer 1/3 Explanation Random sample of 400 women with exactly 2 kids. first bornsecond born~ N_ B B 100 B G 100 G B 100 1/3 of these have 2 girls G G 100

30. The Birthday Question A group of 41 people is randomly selected from the general population. What is the chance that at least two of them will have the same birthday? 10% 30% 50% 70% 90% Answer 90% Explanation The question is: does any person share a birthday with any other person. 41 people  820 different pairs. (e.g., Voracek et al., 2008)

31. # of people# of possible matchesProbability of a match • 23 253 50% • 32 496 75% • 41 820 90% • 50 1225 97% • 366 66795 100% • n n(n-1)/2 • - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - • # of peopleP(match) = 1 – P(no match) • 2 1 – (364/365) = .003 • person 2 • 3 1 – (364/365)(363/365) = .008 • person 2 person 3 • Calculation assumes 1) all birthdates are equally common (not quite true) • 2) no Feb 29 • Correcting for these assumptions has no discernible effect on answer.

33. Monty Hall Dilemma A game show contestant is shown 3 curtains. One curtain has a prize behind it. (Stagehands randomly choose prize location.) The emcee knows which curtain has the prize behind it. Player chooses a curtain. Emcee then opens a non-chosen losing curtain. Player is then allowed to stay with original choice or switch. Should player switch or stay, or does it not matter? Modal Response It doesn’t matter. Correct Answer Switch  2/3 chance of winning

34. 10-Curtain Version of Monty Hall Dilemma 1 of 10 curtains has a prize behind it. (Stagehands randomly choose prize location.) The emcee knows which curtain has the prize behind it. Player chooses curtain. Emcee then opens 8 non-chosen losing curtains. Player is then allowed to stay or switch. Should player switch or stay, or does it not matter? Chance of winning if player switches = 9/10

35. The End

36. Bankers Banker-Democrats Correct Answer Banker. chance of just one  chance of both conjunction fallacy chance of both  chance of just one (Tversky & Kahneman, 1983)

37. Disease Question On average, 1 in 91 people has Disease X. There is a blood test for X. Of people who have X, 100% test positive. (TP = True Positive rate = 100%) Of people who don’t have X, 10% test positive. (FP = False Positive rate = 10%) During a routine physical, Homer tests positive for X. What is the chance that Homer has X? 10% 30% 50% 70% 90% Correct Answer 10% Modal Answer 90% base rate neglect (continued)

38. Explanation base rate TP FP Chance that Homer has X Disease X 1 in 91 100% 10% 10% Disease Y 21 in 91 100% 10% 75% Disease X 91 1 has X 1 TP 90 do not 9 FPs 1 / 10 = 10% Disease Y 91 21 have X 21 TPs 70 do not 7 FPs 21 / 28 = 75%

39. Real Medical Study Success rate for two kidney stone treatments How can this be? Patients were NOT randomly assigned. Instead, doctors usually chose A for large stones, which are harder to treat. Simpson’s Paradox trend reverses when samples combined (Simpson, 1951)

40. More Simpson’s Paradox - UC Berkeley gender bias UC Berkeley sued by because of these 1973 admission data Then data were partitioned by department.

41. Question Scientists compute p in order to test the null hypothesis (H0). What does p equal? P(data given H0 is true) ORP(H0 is true given data) correct common belief (if true, p would be useful)

42. The Disease X question is formally answered by using: Bayes’ Theorem where P(A | B) = Probability of A given that B has happened and P(B | A) = Probability of B given that A has happened (Bayes, 1763) Solution to Disease X question: Let A = disease X is present and let B = positive test P(A) = 1/91, which means that P(not A) = 90/91 P(B | A) = P(positive test given presence of X) = TP = 100% = 1 P(B | not A) = P (positive test given absence of X ) = FP = 10% = 0.10 P(X | positive test) = P(B | A) = (1)(1/91) / [(1)(1/91) + (0.1)(90/91)] = 0.1

43. Based on studies described thus far today, few people understand chance. However… others argue that these studies are misleading because: The questions require algorithms, but people use “fast-and-frugal” heuristics. The questions were contrived to exploit fallibility of heuristics. The heuristics work well in everyday life. (e.g., Gigerenzer, 2002) (e.g., Gigerenzer, 2002)

44. Two versions of mammogram data for women over 50 Relative Risk version If woman has mammogram, her chance of cancer death in next 10 years drops 25% Frequency version If 1000 women skip mammogram, 4 will die of breast cancer in next decade. If all 1000 have mammogram, the number who die will decrease from 4 to 3. Frequency Data are easier to understand and more informative. (e.g., Gigerenzer & Edwards, 2003)

45. Hungry Sailors Six men in a lifeboat are starving, so they agree that one must be sacrificed. They put 5 green marbles and 1 red marble in a bag. One at a time, each man chooses a marble. The first player to choose red is the loser. After each marble is chosen, the marble is NOT placed back in the bag. Is it better to choose first or sixth, or does it not matter? Going first has obvious disadvantage: You’re the only player who must choose! But going first has an advantage: Only 1 in 6 is red. For others, 1 in 5, 1 in 4, … Answer: It does not matter Optional P(Man A gets red) = 1/6 P(Man B gets red) = P (A doesn’t choose red AND B does) = (5/6)(1/5) = 1/6

46. Hungry Sailors Version 2 Six men in a lifeboat are starving, so they agree that one must be sacrificed. They put 5 green marbles and 1 red marble in a bag. One at a time, each man chooses a marble. The first player to choose red is the loser. After each marble is chosen, the marble is placed back in the bag. Is it better to choose first or sixth, or does it not matter? Answer: Sixth Optional P(Man A gets red) = 1/6 = 6/36 P(Man B gets red) = P(A gets green AND B gets red)= (5/6)(1/6) = 5/36