Uncertainty in Cost Estimates

1 / 21

# Uncertainty in Cost Estimates - PowerPoint PPT Presentation

Uncertainty in Cost Estimates. Joseph B. Kadane Carnegie Mellon University kadane@stat.cmu.edu. “Prediction is difficult, especially about the future” Niels Bohr Robert Storm Peterson Yogi Berra Samuel Goldwyn Mark Twain Outline: Uncertainty in general An example in another field

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about 'Uncertainty in Cost Estimates' - jess

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

### Uncertainty in Cost Estimates

Carnegie Mellon University

“Prediction is difficult, especially about the future”

Niels Bohr

Robert Storm Peterson

Yogi Berra

Samuel Goldwyn

Mark Twain

Outline:

• Uncertainty in general
• An example in another field
• Uncertainty in cost estimation
• Conclusion
Uncertainty in General

Suppose you are uncertain about the weather in Pittsburgh tomorrow and in particular about whether it will rain, and whether the high temperature will exceed 80° F. Then there are 4 uncertain events:

• rain and temp > 80°F
• rain and temp ≤ 80°F
• no rain and temp > 80°F
• no rain and temp ≤ 80°F

One and only one of the will occur, i.e. they are mutually exclusive and disjoint.

Imagine that you can buy or sell tickets that pay \$1 if occurs, and \$0 if does not occur. At what price would you be willing to buy or sell such a ticket?

• If you will lose for sure.
• If you will lose for sure.
• If you will lose for sure.
• If you will lose for sure.
• for all events A.
• where S is the sure event.
• if A and B are disjoint.

Hence you avoid being a sure loser if and only if your prices obey the laws of probability.

Notes:
• Your prices need not be those of someone else. These prices – probabilities – are personal, or subjective.
• Avoiding sure loss does not make you a winner. (Indeed, the most absurd beliefs can be expressed using such prices.
• This interpretation of probability does not require an infinite (or long) sequence of independent and identically distributed repetitions.

Suppose there are tickets that pay \$1 if A and B both occur, \$0 if A occurs but B does not, and the price is returned if A does not occur. Suppose you would pay for such a ticket.

Notes:
• This is the same as the usual definition of conditional probability, except here it is a consequence of a desire to avoid sure loss.
• When is unconstrained.
2. Example from another field: physics

Physics is touted as the most exact of sciences, one in which uncertainty is best controlled. However, as we’ll see, experimental physicists routinely assume that the measurements they make have no systematic errors, and are routinely wrong, by hindsight, in that assessment.

Systematic errors can arise from:

• Undetected bias in experimental procedures
• Computational approximations
• Errors in auxiliary variables
• Deficiencies in theoretical assumptions

None of these are apparent to the experimenter at the time.

Repeating an experiment many times can reduce the “random error” associated with an average, but does nothing for systematic error. The culture of physics has not traditionally encouraged a discussion of the likely magnitude of systematic error.

3. Uncertainty in Cost Estimates

“A reported value whose accuracy is entirely unknown is worthless”

Churchill Eisenhart

Why is it that cost estimates of almost anything are so rarely too high, and very often too low? One simple correction is to multiply the cost estimate by .

If an interval is required, multiply by where

Perhaps it is possible to do better than this with a more thoughtful approach.

What are some of the sources of uncertainly in estimating the cost of large, long-term projects?

• Scope creep: things change. New capabilities are requested. Is the cost estimate supposed to estimate
• The hypothetical cost of the project if no changes in scope are required.
• The actual cost, including possible future (now unforeseen) changes in scope?

Technology risk: the technology to do the project may not currently exist. How much may it cost to produce the technology? What happens if the technological issues cannot be resolved?

Economic and political changes: If cost estimates are in current dollars, what effect may future inflation have? What may be the effects of legislative or regulatory changes? What assumptions underlie the project? What are the consequences if those assumptions are wrong?

4. Conclusions
• Give up the idea that cost estimates are anything but somebody’s opinion.
• Best practice is to express your opinion in terms of a probability distribution.
• Be clear about exactly what you are costing out, particularly whether changes in scope are included.
• Think about all the ways you might be wrong, and what the costs under those scenarios look like.
References:
• For a fast read, introduction:

D.V. Lindley, Understanding Uncertainty, J. Wiley & Sons.

• For proof of theorems 1 and 2, and a more technical discussion, my book

Principles of Uncertainty, Chapman and Hall, 2011, also free on the web at my website http://www.uncertainty.com.

• For the physics:

M. Henrion and B. Fischoff, “Assessing uncertainty in physical constants,” American Journal of Physics, 54(9), September, 1986.