Ameliorating Mental Mistakes in Tradeoff Studies. Terry Bahill Systems and Industrial Engineering University of Arizona [email protected] ©, 1993-2010, Bahill This file is located at http://www.sie.arizona.edu/sysengr/slides/. Acknowledgement. This research was supported by
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
This research was supported by
Smith, E. D., Son, Y. J., Piattelli-Palmarini, M. and Bahill, A. T., Ameliorating mental mistakes in tradeoff studies, Systems Engineering, 10:3, 222-240, 2007.
All of the material in this presentation is based on peer-reviewed journal papers. None of it comes from the Internet.
CMMI and recommended as a Decision Analysis and Resolution (DAR) method for simultaneously considering multiple alternatives with many criteria.
have a large body of research on human biases and errors in considering numerical judgments and criteria-based choices.
Daniel Kahneman won the Nobel Prize in Economics in 2002 "for having integrated insights from psychological research into economic science, especially concerning human judgment and decision-making under uncertainty."
Eric Smith studied hundreds of experimental papers and isolated seven dozen biases that could affect the components of tradeoff studies. His results are summarized in this Excel spreadsheet.
$ bet isolated seven dozen biases that could affect the components of tradeoff studies. His results are summarized in this Excel spreadsheet.
Has higher dollar value
Has higher probabilityPreference reversals*
Although the expected values are the same,
most people preferred to play the P bet, however
most people wanted a higher selling price for the $ bet.
Choice, 67% chose Program X
Calculation, 4% calculated $55M or more
Different people will give different weights of importance because of their perceptions of
Recommendation: Intersubject variability can be reduced with education, peer review of the assigned weights and group discussions. Keep a broad view of the whole organization, so that criteria in one area are considered in light of all others.
subject to elimination only after
comparison to all alternative solutions.
“You can either select one of these gambles or you can pay $1 to add one more gamble to the choice set. The added gamble will be selected at random from the list you reviewed.”
Gambler’s fallacy isolated seven dozen biases that could affect the components of tradeoff studies. His results are summarized in this Excel spreadsheet.
Law of small numbers
Mis-Estimation of probabilities
Ease of Representation: Typicality
Aversion to sequences of chance events
Base-Rate NeglectProbabilistic illusions
reading, either record both
measurements or the average
of the two readings.
Recommendation: isolated seven dozen biases that could affect the components of tradeoff studies. His results are summarized in this Excel spreadsheet. Scoring functions in a tradeoff study
should express gains rather than losses.
should be referenced in company engineering processes.
Discussion isolated seven dozen biases that could affect the components of tradeoff studies. His results are summarized in this Excel spreadsheet.
is to help a decision maker make valid decisions that he or she (and other stakeholders) will have confidence in.
for ensuring success of tradeoff studies include
quadrant to the ideal
The original design for the RMS Titanic called for 64 lifeboats, but this was reduced to 20 before its maiden voyage: this might have been a mistake. The Chief Designer (CD) wanted 64 lifeboats. But the Program Manager (PM) reduced it to 20 after his advisors told him only 16 were required by law. The CD resigned over this decision. The British Board of Trade regulations of 1894 specified the lifeboat capacity. For ships over 10,000 tons, this lifeboat capacity was specified by volume (5,500 cubic feet), which could be converted into passenger seats (about 1000) or the number of lifeboats (about 16). So, even though the Titanic displaced 46,000 tons and was certified to carry 3,500 passengers, its 20 lifeboats complied with the regulations of the time. But let us go back to the design decision to reduce the number of lifeboats from 64 to 20. What if they had performed the following hypothetical tradeoff study? In this table, the weights of importance range from 0 to 10, with 10 being the most important and the evaluation data (scores) also range from 0 to 10, with 10 being the best. For simplicity, we have not used scoring functions, so the evaluation data are also the scores.