UNCERTAINTY. Pastiche from 2 day course on Expert Judgment. AMBIGUITY. Roger Cooke Resources for the Future Dept. Math, Delft Univ. of Technology Dec 14, 2009 http://dutiosc.twi.tudelft.nl/~risk/. INDECISION. UNCERTAINTY. AMBIGUITY. INDECISION.
2 day course on Expert Judgment
Resources for the Future
Dept. Math, Delft Univ. of Technology
Dec 14, 2009
INDECISION“Uncertainty from random sampling ...omits important sources of uncertainty” NRC(2003)
All cause mortality, percent increase per 1 μg/m3 increase in PM2.5 (RESS-PM25.pdf)
Very Different Guidelines:
The story you hear today is NOT the only story
INDECISIONHistory Structured Expert Judgment in Risk Analysis
INDECISIONEJ for RATIONAL CONSENSUS:RESS-TUDdatabase.pdf
Parties pre-commit to a method which satisfies necessary conditions for scientific method:
Neutrality (don’t encourage untruthfulness)
Fairness (ab initio, all experts equal)
Empirical control (performance meas’t)
Withdrawal post hoc incurs burden of proof.
Goal: comply with principals and combine experts’ judgments to get a Good Probability Assessor
“Classical Model for EJ”
INDECISIONWhat is a GOOD subjective probability assessor?
INDECISIONCombined ScoreLong run strictly proper scoring rule
Requires that experts assess uncertainty for variables from their field for which we (will) know the true values:
Calibration / performance / seed variables
any expert, or combination of experts, can be regarded as a statistical hypothesis
INDECISIONCalibration questions for PM2.5RESS-PM25.pdf
In London 2000, weekly average PM10 was18.4 μg/m3. What is the ratio:
# non-accidental deaths in the week with the highest average PM10 concentration (33.4 μg/m3)
Weekly average # non-accidental deaths.
5% :_______ 25%:_______ 50% :_______ 75%:________95%:________
INDECISIONExperts CAN quantify uncertainty as subjective probability
P-Value Too Low for Acceptance
INDECISIONWe CAN do better than equal weightingRESS-TUDdatabase.pdf
13 studies with 14 seed vbls, split, initialize on one half, predict other half
Now its all about Dependence
Thank YouWilly AspinallTim BedfordJan van NoortwijkTom MazzuchiDorota KurowickaDavid LodgeRamanan LaxminarayanAbby ColsonHarry Joemany studentsgod knows who else
“ In the first few weeks of the Montserrat crisis there was perhaps, at times, some unwarranted scientific dogmatism about what might or might not happen at the volcano, especially in terms of it turning magmatic and explosive. The confounding effects of these diverging, categorical stances were then compounded for a short while by an overall diminution in communication between scientists and the various civil authorities. The result was a dip in the confidence of the authorities in the MVO team and, with it, some loss of public credibility; this was not fully restored until later, when a consensual approach was achieved. “
Aspinall et al The Montserrat Volcano Observatory: its evolution, organization, rôle and activities.
1Using Uncertainty to Manage Vulcano risk responseAspinall et al Geol Soc _.pdf
British Airways Pilots study