Download
1 / 22
220 likes | 347 Views
Psychology of Estimating. North Florida AACE: March 2012. Issue. Many lurking variables influence estimates Some are more obvious than others We will discuss some of the psychology that affects estimates. Things are not always as they appear. What is This a Picture Of?.
E N D
Psychology of Estimating North Florida AACE: March 2012
Issue • Many lurking variables influence estimates • Some are more obvious than others • We will discuss some of the psychology that affects estimates
What is This a Picture Of? All three pictures are of the same thing – Spiders Due to massive flood in WaggaWagga Australia all the spiders congregated in small area
Confirmation Bias • Philosopher Francis Bacon in 1620 • “the human understanding, once it has adopted an opinion, collects any instances that confirm it, and though the contrary instances may be more numerous & more weighty, it either does not notice them or else rejects them, in order that this opinion will remain unshaken” • People have an unjustified bias in favor of their opinion • Much tougher when critiquing validity of information that undercuts their currently held theories than they are in supporting information that apparently endorses one of their own tenets • Employing this double standard, the tendency is to dismiss information that doesn’t fit with what they already believe • A 1999 study of 27,000 expert predictions • Study did not vindicate their "expertness" as was anticipated • Error rates were many times what these experts had predicted • No differential advantage for those experts holding graduate degrees versus those with only undergraduate degrees
What is This a Picture Of? • Two groups shown blurry image • One group resolution increased in 10 steps • Other group resolution increased in 5 steps • Stop on same image for both • Group members who saw fewer intermediate steps likely to recognize hydrant faster • Conclusions more information is not always better! • Delay forming opinions as long as possible Result of Confirmation Bias and Belief Perseverance tendency not to reverse your existing opinions
Everyone Thinks They Are Smarter Than Everyone Else! • Brilliant ideas are hard for people to adopt, because most people don't have the sophistication to recognize how good an idea really is • Research has shown demonstrated again and again that people are self-delusional when it comes to their own intellectual skills • Researchers have tested people's ability to rate • Funniness of jokes, the correctness of grammar, or even their own performance in a game of chess, • People assess their own performance as 'above average' — even people who, when tested, actually perform at the very bottom of the pile • We're just as undiscerning about the skills of others as about ourselves. • To extent that you are incompetent, you are a worse judge of incompetence in other people • Reason is simple: • If you have gaps in your knowledge in a area, then you're not in a position to assess your own gaps or gaps of others • Strangely though, people tend to readily & accurately agree on the worst ideas or performers, while failing to recognize the best
Anchoring Any logical person knows that there is no relationship, but it still influenced the results. • Anchoring is human tendency of latching on a concept or a number & consciously or subconsciously assuming it is correct. • There have been many studies on the effect(s) of anchoring. • One such study had subjects spin a numbered “wheel of fortune” • Then told to estimate number of African countries in the United Nations • Those shown a low number on wheel estimated a low number • Those shown a high number on wheel estimated a high number • Caution in mentioning any numbers is advised when questioning experts or briefing management to avoid inadvertently skewing their opinions
Harvard Study • Harvard Business School students asked to answer a series of questions in such a way that they believed they had a 98% chance of being correct, and less than 2% chance of being wrong. • EG, “I am 98% confident that population of Vermont is between 2 million & 20 million.” • Students free to set their range as wide as they wanted. • Experiment was not designed to test their subject matter knowledge, but rather to test their confidence levels of their own knowledge (e.g. to what extent they feel certain that they know something). • Students failed miserably with error rate on close to 45%. • Experiment has been replicated dozens of times, across various populations, professions, & cultures. • Results regarding self-supporting gravitational thinking are always the same. • Expected error rate of 2% is actually on average between 15%-30%! • Conclusion: we over-estimate what we really know & under-estimate the possibility of our being wrong. • Our tendency to think well of our own opinions can have dramatic results in the negative.
Are You Smarter Than A Harvard Business Student? 120 5,000 3:53 47 443 19.8 179,500 • Your series of questions • How many countries have McDonalds? • What is the range of a Minuteman Missile in miles? • How long in minutes & seconds is the song Stop in the Name of Love? • How many rulers has England had in the last 1,000 years? • How tall is the Sears Tower in meters? • What is the combined average MPG of cars & trucks? • What was the average home price in 2001? • Provide large enough range to provide 90% certainty of a correct answer • Example what is the wingspan of a Boeing 747 • Answer: It depends… Which model? • 747-100 =195 feet 8 inches • 747-400 = 211 feet 5 inches • 747-8I = 224 feet 9 inches • Real Answer: 150 feet to 250 feet
Separate MIT Study On Same Subject • 83% of Systems Engineers got 50% or less answers correct. • NOTE: In 221 instances respondents said they were 100% certain of their answers… however, they were correct only 73% of the time • Why is this important? • If a system engineer tells estimator they are 90% certain something won’t be required… they will be WRONG much of the time • So the estimate will be wrong much of the time! Honest mistakes coupled with management desire for “affordable” = overruns! MIT Study by Dr. Ricardo Valerdi Over ½ were only 20-40% accurate when they were asked to give their 90% confidence answer
Is There Hope? Desired Accuracy Achieved Accuracy MIT Study by Dr. Ricardo Valerdi • So it’s hopeless…? • No… Studies have shown that if given feedback, rapid improvement can occur • The sooner the feedback the better • Calibration feedback shown to be substitute for experience • Everyone is not trainable • Better calibrated people don’t have better information or possess superior guessing skills, they are more in tune with their cognitive abilities & more realistic about their judgments – a skill that requires an understanding of the connection between subjective probabilities & objective outcomes
Group Think • "Reaching consensus in a group is often confused with finding the right answer". • - Norman Mailer • In an effort to make decisions without conflict, the various viewpoints held by individual members of the group are conformed to general viewpoint, thus stifling debate. • Silence is viewed as agreement • Provides illusions of invulnerability; creates optimism and encouraging risk taking. • Group rationalizes warnings that might challenge the group's assumptions. • Evidence suggests that those who are meek, ignorant or naïve are subject to manipulation by a loud, opinionated minority, or a strong leader. • If this is true, uninformed individuals are detrimental to democratic decision-making, since they can turn over power to a minority. • Classic example was the decision to launch the space shuttle Challenger. A dissenter presented the team with objections and advised to postpone launch. • Facing another embarrassing delay, the decision makers succumbed to Group Think and seven astronauts lost their lives.
Group ThinkContinued “Camel is a horse designed by committee” Sir Alec Issigonis – Designer of the Mini Cooper “Elephant is a mouse designed by a government committee”
The BossLeft Brained or Right Brained? • Some people are left brained, some right brained • Estimate presentation to them should vary accordingly
Why Does Any of This Matter? “In God we trust; all others must bring data” - Edward Deming • Bad estimates by “experts” • Good assumptions are critical for good estimates • Assumptions are often provided by experts • Anchoring & Confirmation Bias • Management if initially shown a low initial number will assume its correct • Don’t offer initial numbers, but if you must, make it a high number • It can be refined & go down if conditions merit • Much easier to reduce number than increase the initial estimate • Confirmation Bias, & Belief Perseverance • If contrary evidence is presented, experts will argue that they are right • Even though many studies & history prove them wrong • Trust valid data not experts • Consider “padding estimate” to compensate - must be done with caution • Group Think • Study result biased by opinionated group minority
Analytical MethodsPossible Solution? • What are analytical methods? • Extensive use of data, statistical and quantative analysis, explanatory and predictive models, and fact based management to drive decisions and actions • Why use “unproven” analytical methods? • Methods are far from unproven, in fact majority of highly successful companies use them. • Constrained by shortage of people who understand them • Managers who don’t understand, therefore don’t trust models • Analysts who can construct and use models • Netflix offered $1M prize to anyone who can improve Cinematch algorithm by 10% or more • Here is a partial list of companies who reportedly rely heavily on analytic’s. List Source - Competing on Analytics The New Science of Winning, Thomas Davenport
