The influence of hierarchy on probability judgment

1. The influence of hierarchy on probability judgment David A. Lagnado David R. Shanks University College London

2. Level of hierarchy can modulate judgment • Consider two statements about the next World Cup • It is most likely that Brazil will win • It is most likely that a European team will win • These appear to support opposing predictions, but both may be true • Shows the importance of the level at which probabilistic information is represented

3. Hierarchical structure • Pervasive feature of how we represent the world • Reflects pre-existing physical and social hierarchies • Readily generated through conceptual combination • Category hierarchies serve both to organize our knowledge, and to structure our inferences

4. Tabloid Broadsheet Sun Guardian Times Mirror Inference using a hierarchy • One powerful feature of a category hierarchy is that given information about categories at one level, you can make inferences about categories at another level. • This allows you to exclude alternatives, or reduce the number you need to consider

5. Tabloid Broadsheet Sun Guardian Times Mirror Probabilistic Inference using a hierarchy • In many real-world situations we must base our initial category judgments on imperfect cues, degraded stimuli, or statistical data. • What effect do such probabilistic contexts have on the hierarchical inferences that we are licensed to make?

6. Tabloid Broadsheet Sun Guardian Times Mirror Commitment heuristic • Commitment heuristic - When people select the most probable category at the superordinate level, they assume that it contains the most probable subordinate category. • This leads to the neglect of subordinates from the less probable superordinate.

7. How adaptive is this heuristic? • The efficacy of such a heuristic depends on the precise structure of the environment. • In certain environments it confers considerable advantages • increases inferential power by focus on appropriate subcategories • reduces computational demands by avoiding complex Bayesian calculations. • But in some environments it can lead to anomalous judgments and inferences.

8. Non-aligned hierarchy • In the above sample the most frequently read type of paper is a Tabloid, but the most frequently read paper is a Broadsheet (the Guardian). • Non-aligned hierarchy: the most probable superordinate category does not contain the most probable subordinate category. Tabloid 60 Broadsheet 40 Sun 30 Times 5 Guardian 35 Mirror 30

9. Real world examples • Word frequencies: the superordinate BE- is more frequent than BU-, but the subordinate BUT is more frequent than any of the other subordinates (BET, BED…etc.) • NHS statistics on survival rate for operations for different areas & sub-areas • You are more likely to survive a hip operation in Surrey rather than Essex, but the best sub-area for survival is Colchester (in Essex).

10. Experiments 1 and 2 • Learning phase - participants exposed to a non-aligned hierarchical environment in which they learn to predict voting behavior from newspaper readership. • 100 trials ‘reading/voting profiles’

11. Screen during learning phase Tabloid Broadsheet Chronicle Herald Reporter Globe ○ Liberal ○ Progressive

12. Screen during learning phase Reading profile for J. K. Tabloid Broadsheet Chronicle Herald Reporter Globe ○ Liberal ○ Progressive

13. Screen during learning phase Reading profile for J. K. Tabloid Broadsheet Chronicle Herald Reporter Globe ○ Liberal Outcome feedback ○ Progressive

14. Structure of environment Tabloid 60 Broadsheet 40 Sun 30 Mirror 30 Guardian 35 Times 5 Party A Party B 50 50

15. Judgment phase What is the probability that X votes for one party rather than the other? Baseline X is selected at random Which type of paper is X most likely to read? Type Which paper is X most likely to read? Paper

16. Results of Experiment 1 • Probability ratings for Party B rather than Party A with judgments divided into those based on aligned and non-aligned choices

17. Experiment 2 • Replication of Experiment 1, with frequency as well as probability response formats • Frequentist hypothesis that probability biases reduced with frequency format

18. Results of Experiment 2 • Mean ratings for Party B rather than Party A collapsed across probability and frequency ratings

19. Summary of Results • Participants allow their initial probability judgment about category membership (newspaper readership) to shift their rating of the probability of a related outcome (voting preference), even though all judgments are made on the basis of the same statistical data. • When their prior choices were non-aligned this led to a switch in predictions about the outcome category

20. Conclusions • These biases are explicable by the Commitment heuristic: • The priming question commits people to just one inferential path, leading them to compute an erroneous estimate for the final probability. • This is understandable given the complexity of the normative Bayesian computation.

21. Comparison of Bayesian and commitment heuristic computations (just type level inference) • Type of paper? Type of paper? 0.6 0.4 0.6 Tabloid Broadsheet Tabloid 0.23 0.1 0.9 0.77 0.77 Party A Party B Party A • P(A) = (0.6 . 0.77) + (0.4 . 0.1) • = 0.46 + 0.04 • = 0.5 P(A) = 0.77 Bayesian computation Simplified heuristic computation

22. Conclusions • Simplifying heuristic that assumes that environment is aligned • Empowers inference when hierarchical structure is aligned, otherwise can lead to error • Suggests tendency to reason as if a probable conclusion is true

23. Process level accounts • Associative model • People learn predictive relations between category options (at both levels of hierarchy) and outcome. At test responses to category questions prime the appropriate associations and lead to a biased rating of the outcome. • Frequency-based model • People encode event frequencies in the learning phase. At test responses to the category question serves as the reference class for subsequent conditional probability judgments about voting preferences.

24. Implications • Importance of the level at which probabilistic data is represented to (or by) a decision maker • E.g., using NHS statistics to decide on hospital • How do people search through hierarchical statistical data? • People’s judgments can be manipulated by the level at which statistical information is represented • More generally, in multi-step inferences people are susceptible to biased probability judgments