Integrated Uncertainty Modelling for Decision Making Workshop

1. Jan NoyesProfessor of Human Factors PsychologyUniversity of Bristolhttp://human-factors.psy.bris.ac.uk/ Integrated Uncertainty Modelling for Decision Making Workshop

2. Outline • Humans as information processors • Decision making as explained by reasoning • Risk perception • The study of human decision making over the decades • Influences on decision making • Summing up – how do humans make decisions • Conclusions especially with regard to uncertainty

3. Humans as ‘information processors’

4. Our strengths PerceptionThe process of receiving information from the outside world. Note: humans are ‘furious’ pattern-matchers Memory - an interesting paradox … Humans have amazing memories but also severe limitations on the amount of information they can process at any one time, for example, short term memory is fragile and limited, and effort is required to retrieve information.

6. Our weaknesses AttentionWe are poor at monitoring. “There is no such thing as voluntary attention sustained for more than a few seconds at a time.” Quote from William James (1890) Higher order (cognitive) processes We are not good at some types of decision making, for example, deductive reasoning.

7. Deductive reasoning The process of reasoning from one or more premises with regard to what is known, and upon which a logically specific conclusion can be reached. All A are B, All B are C, therefore, all A are C.

8. Inductive reasoning Reasoning from specific facts or observations to a general conclusion that may explain the facts, that is, it is not possible to reach a logically certain conclusion. All men are mortal, therefore, Fred Bloggs is mortal.

9. Deductive versus Inductive reasoning Deductive - logic, while Inductive - particular conclusions are drawn from more general principles. Inductive reasoning involves an element of doubt/uncertainty. If Alan is taller than Bill, and Bill is shorter than Chris, is Alan taller than Chris?

10. Common mistakes • Draw incorrect conclusions, for example, if I take my umbrella, then it will rain. • Reverse the propositions in the conditional statement, for example, if the ‘plane will be diverted, then there is fog. • Confirmation bias (seek to confirm).

12. Wason’s 4-card problem You have four cards with a letter on one side and a number on the other. E F 4 7 “If a card has a vowel on one side, it will have an even number on the other.” Which two cards do you want to turn over to check whether this rule is valid or not?

13. Drinking age rule Drinking Drinking 22 years 16 years a beer a coke of age of age Which card or cards do you want to turn over to check whether the ‘drinking age’ rule is valid or not?

14. Entering the country Entering In transit Inoculated Inoculated against against cholera typhoid & hepatitis The rule ‘people entering the country have been inoculated against cholera’. Is this valid or not?

15. Explanations • People do not reason logically – they simply apply their knowledge of the world – helps cope with uncertainty. • Meaningful content – limits of working memory. If correct, not an issue,if incorrect, can lead to problems. • Difficulties – representation, negation, language. • Confirmation bias – search for evidence that confirms our beliefs. For example, Three Mile Island.

16. Three Mile Island, 1979

17. Decision making Early models – based on Normative decision making Break down decision problems. • Do this by identifying the alternatives. • Weigh the outcomes in terms of their usefulness (utility). • Select the ‘best’ outcome. Goal - to seek pleasure and avoid pain. When decision making, we: • seek to maximise pleasure (+ve utility) • seek to minimise pain (-ve utility).

18. From Normative to Descriptive In the 1970s, the basic UTs ‘failed’, because of the subjective element in the observed behaviour of decision makers. There was seen a need to take into account human irrationality and biases. Hence, the Descriptive models appeared.

19. The Prisoner’s Dilemma Suppose that two men have been arrested and they are charged with a bank robbery. Each man is found to have an unregistered firearm at the time of the arrest. The police do not have conclusive evidence that the two men actually robbed the bank, and need a confession from at least one of the men in order to make a successful prosecution.

20. The Dilemma If neither confesses, they will be charged with possessing an illegal firearm and jailed for a year. If they both confess, they will each be given an intermediate-length term of 10 years. Further, the prisoner who confesses will be let off, while the other will get the maximum sentence of 20 years.

21. Would you confess? Prisoner A Confess Not Confess Prisoner B Confess Both get A=20 years 10 years B=0 years Not A=0 years Both get 1 year Confess B=20 years

22. Game theory Suggests that much decision making is like playing a game with the following features: Minimax loss rule - minimise possibility of maximum loss (20 years) - confess (possibility of 0 [or 10] years). Maximin gain rule - maximise possibility of minimum gain (least favourable outcome = 20 years) - don’t confess. Maximax gain rule - maximise possibility of maximum gain - confess.

23. Normative and Descriptive Criticism of early (normative and descriptive) theories - they often do not use problems and scenarios that are realistic. Solution – the Prescriptive models. • These focus on how people should make decisions to conform to the normative model. • Still idealised, but taking into account, the real situation often characterised by ambiguous and incomplete information, limited time resources, and high stakes. • This led to the development of naturalistic decision making (NDM) prescriptive models in the 1990s.

24. Insights into NDM (complex decision making) Klein (1993) and Rasmussen (1993) • Focus on finding a course of action which works. • Assess situation and select course of action (experience helps - become quicker and more accurate). • Act without considering all contingencies. Course of action is not necessarily the best one - best one at that point in time. • Choice is related to evaluation and mental construction of problem. • Focus on relevant options rather than filtering out unacceptable ones.

25. Putting the pressure on … In a high workload/panic-type situation, what happens? • Resort to automatic (skilled) behaviour, for example, use of checklists will help to reduce memory load. • High levels of attention can be sustained. • Will become ‘tunnel-visioned’, for example, fixation in terms of problem-solving.

26. Historical perspective - decision making • Early decision theories based on (a) humans operate in ideal circumstances, and (b) make optimal decisions. • Recent work - we strive to make the ‘best’ decisions in any given situation. • Naturalistic theories are a mixture of different models and strategies, and more suited to explaining decision making in operational environments such as the flight deck, fire-fighting/emergency services.

27. How do humans make decisions? 1. Satisficing We do not consider all the options, but opt for the first choice that ‘satisfies’ us, for example, working through a long menu.

28. How do humans make decisions? 1. Satisficing We do not consider all the options, but opt for the first choice that ‘satisfies’ us, for example, working through a long menu. 2. Heuristics and Biases We do not apply logic but tend to resort to feelings … For example, All the families having exactly six children in a particular city were surveyed. In 72 of the families, the exact order of births of boys (B) and girls (G) was G B G B B G. What is your estimate of the number of families surveyed in which the exact order of births was B G B B B B?

29. 3. Availability (Anchoring) The extent to which information is available to us can determine our decision making. Influence of media and how available information is to us. Evidence when there is a health scare, for example, about the contraceptive pill. Example: What percentage of African countries are in the United Nations?

30. 3. Availability (Anchoring) The extent to which information is available to us can determine our decision making. Influence of media and how available information is to us. Odds of dying ... In a car crash: 5,000 to 1 From surgical complications: 80,000 to 1 In a ‘plane crash: 250,000 to 1 By falling out of bed: 2 million to 1 WINNING THE LOTTERY: 80 million to 1

31. 4. Overconfidence We tend to overvalue our judgement skills. Example: “Absinthe is a liqueur or a precious stone.”

32. 4. Overconfidence We tend to overvalue our judgement skills. Example: “Absinthe is a liqueur or a precious stone.” 5. The Gambler’s Fallacy Luck will change, for example, have lost money on five horse races, therefore, will win on the sixth race. But, probability has no memory …

33. 4. Overconfidence We tend to overvalue our judgement skills. Example: “Absinthe is a liqueur or a precious stone.” 5. The Gambler’s Fallacy Luck will change, for example, have lost money on five horse races, therefore, will win on the sixth race. But, probability has no memory … 6. Fallacy of Composition Parts of the whole make up the whole, for example, the ‘horns/halo’ effect.

34. More evidence of plasticity Order effects How questions affect answers? Q.1. Do you think a Communist country like Russia should let American newspaper reporters come in and send back to America the news as they see it? Q.2. Do you think the USA should let Communist newspaper reporters from their countries come in and send back to their papers the news as they see it?

35. Results Q.1. Do you think a Communist country like Russia should let American newspaper reporters come in and send back to America the news as they see it? Q.2. Do you think the USA should let Communist newspaper reporters from their countries come in and send back to their papers the news as they see it? Counterbalanced design - 50% had Q.1 first. Q.1: 82% agreed when 1 first, 64% agreed when 2 first. Q.2: 55% agreed when 1 first, 75% agreed when 2 first.

36. Pseudo-opinions Which of the following statements most closely coincides with your opinion of the Metallic Metals Act? 1. It would be a good move on the part of the USA. 2. It would be a good thing, but should be left to individual states. 3. It is all right for foreign countries, but should not be required here. 4. It is of no value at all.

37. Results 1. It would be a good move on the part of the USA. (15%) 2. It would be a good thing, but should be left to individual states. (41%) 3. It is all right for foreign countries, but should not be required here. (11%) 4. It is of no value at all. (3%) No opinion (30%).

38. Inconsistency Principles: • Public officials should be chosen by majority vote. • Every citizen should have an equal chance to influence government policy. • The minority should be free to criticize majority decisions. Derived statement:“In a city referendum, only people who are well informed about the problem being voted on should be allowed to vote.”

39. Results It was found that: • 51% agreed with anti-democratic idea that only well-informed should vote. Similar study with taxpayers. It was found that: • 79% agreed that only taxpayers should vote.

40. Problems … • Integration of information – problematic for us (attention deficit). • Resource limitations (memory) – perhaps cannot cope with all the aspects involved. • Prediction outcomes – create uncertainty, possibly because of difficulties associated with integration. • Language, for example, negatives, conditional clauses, use of probability – poor understanding. • Simple manipulations can change our decision making, for example, switching the order of questions can influence the answers. • People when asked will often give answers. • Generally, people exhibit inconsistency.

41. Summary: Making decisions • Construction of a mental representation (story) – an integration of available information, knowledge and personal beliefs about the world. • Assessment of the situation. • Realistic settings – uncertain situations, incomplete, and often, contradictory information, time pressure, delay in feedback, shifting, and competing, goals, and changing conditions. • In terms of strategy, experiments at Bristol have shown differences in how people perceive levels of risk according to how information is presented.

42. Conclusions We excel in situations which demand the use of inductive reasoning (unlike machines/computers that can programmed to carry out deductive reasoning). It could be concluded that we do not use ‘knowledge’ optimally - we use ‘short cuts’ which guarantee solutions (which are not necessarily the best) – the fast but frugal heuristic. In this sense, uncertainty is not a problem, because we are not very efficient decision makers anyway.

43. Finally … “Many people would sooner die than think. In fact they do.” Bertrand Russell