Download
1 / 30
310 likes | 570 Views
Uncertainty surrounding the Cone of Uncertainty. Todd Little. “It’s tough to make predictions, especially about the future.” – Yogi Berra. IEEE Software, May/June 2006. When will we get the requirements?. All in good time, my little pretty, all in good time.
E N D
Uncertainty surrounding the Cone of Uncertainty Todd Little “It’s tough to make predictions, especially about the future.” – Yogi Berra
When will we get the requirements? All in good time, my little pretty, all in good time But I guess it doesn't matter anyway Just give me your estimates by this afternoon Not so fast! Not so fast! ... I'll have to give the matter a little thought. Go away and come back tomorrow No, we need something today! Ok then, it will take 2 years. No, we need it sooner. Doesn't anybody believe me? I already promised the customer it will be out in 6 months You're a very bad man! Managing the Coming Storm Inside the Tornado Project Kickoff Team Unity
I may not come out alive, but I'm goin' in there! The Great and Powerful Oz has got matters well in hand. My! People come and go so quickly here! "Hee hee hee ha ha! Going so soon? I wouldn't hear of it! Why, my little party's just beginning! We’re not in Kansas Anymore Developer Hero Reorg Testing
About Landmark • Commercial Supplier of Oil and Gas Exploration and Production Software • Users are Geophysicists, Geologists, Engineers • Subsidiary of Halliburton Energy Services • Integrated suite of ~60 Products • ~50 Million lines of code • Some products 20 years old
Reservoir / Fluid data Seismic data Production data Structural / Stratigraphic data Common Model Representation Common Model Representation Velocity data Well data Landmark Product Suite Geophysics Engineering Geology
Data in the Portfolio • 3 years of data (1999-2002) • 570 projects • 106 valid (Shipped commercial product) • Remainder: Currently active, placeholder projects, internal projects, non-commercial releases, deferred projects, etc. • Relatively Unbiased. • Each week the Program Manager recorded the state of the project and the current release estimate. • No “improvement goal” bias
Data from LGC Developing Products in Twice the Time
Data from Tom DeMarco It’s déjà vu all over again
How does Estimation Accuracy Improve Over Time? • At the “end” of each phase, compare the most current estimate with the resulting end date. • Envisioning • Planning • Developing
But is Uncertainty Really Reduced? “Take away an ordinary person’s illusions and you take away happiness at the same time.” Henrik Ibsen--Villanden
The Pipe of Uncertainty 2 0.5
Does Landmark Suck at Estimation? A severe depression like that of 1920-21 is outside the range of probability. Harvard Economic Society, Weekly Letter, November 16, 1929. I think there is a world market for about five computers. Thomas J. Watson, chairman of IBM, 1943. They couldn't hit an elephant at this dist… General John B. Sedgwick, Union Army Civil War officer's last words, uttered during the Battle of Spotsylvania, 1864
Estimation Quality Factor (EQF) Actual Value Blue Area Red Area Value to be Estimated EQF = Initial Estimate Actual End Date Elapsed Time Link to article by Tim Lister
EQF from Lister/DeMarco • An EQF of 5 is pretty good (i.e. averaging about 1/5 or 20 percent off.) • The median for schedule estimating is about a 4, with the highest sustained scores at 8 to 9. • Lister and DeMarco have never known anybody to sustain a 10 (just 10 percent off). • Typical disaster project is 1.8
EQF Distribution Curve (LGC) EQF for duration has a theoretical minimum of 2.0
We slip one day at a time, EQF=2 Actual Value Blue Area Red Area Value to be Estimated EQF = Initial Estimate Actual End Date Elapsed Time
LGC Estimation Quality • LGC’s EQF measurement is pretty good. • Our p(50) is 4.8, versus an industry average around 4 and a best sustained in the ~8-10. • Our p(10) is 2.8, which is not bad.
