1 / 23

Evaluating Prediction Intervals: Trusting Regression Estimates in Project Planning

In this comprehensive review by Dr. Mark L. Hornick, we delve into the mechanics of regression calculations and their significance for projecting project sizes and timelines. By examining Prediction Intervals, we assess the accuracy of these estimates: can they be off by as much as 500%? We explore real-world scenarios where decision-making hinges on these projections and consider how to effectively communicate uncertainty to management. This analysis highlights the importance of understanding the reliability of estimates to safeguard your reputation, job, and future performance.

gisela-fields
Download Presentation

Evaluating Prediction Intervals: Trusting Regression Estimates in Project Planning

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Prediction Intervals SE-280Dr. Mark L. Hornick

  2. Review: PROBE’s regression calculations give us estimates (projections) for size (A+M LOC)… SE-280Dr. Mark L. Hornick

  3. …and time SE-280Dr. Mark L. Hornick

  4. But just how good are these estimates??? Off by 5%, 10%, 50%, 100%, 500%? Does it matter? Do you want to bet: • Your weekends? • Your reputation? • Your JOB? SE-280Dr. Mark L. Hornick

  5. Which of the following regression projections would you trust more? SE-280Dr. Mark L. Hornick

  6. Example A10 data pointsCorrelation = 0.75 800 700 600 500 Actual Total LOC 400 300 200 100 0 0 100 200 300 400 Estimated Object LOC SE-280Dr. Mark L. Hornick

  7. Example B25 data pointsCorrelation = 0.75 800 700 600 500 Actual Total LOC 400 300 200 100 0 0 100 200 300 400 Estimated Object LOC SE-280Dr. Mark L. Hornick

  8. A Prediction Interval calculation computes the bounds on the likely error of an estimate UPI = estimated A+M LOC + Range LPI = estimated A+M LOC - Range UPI Range Projection (Estimate) Range LPI Strictly speaking, the UPI/LPI "lines" are parabolas, and Range varies.

  9. If you had this kind of information about your estimates, how would you use it? 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 Suppose your time projection said that a project would take 8 weeks. But, your prediction interval has a range of 3 weeks. How should you make your plan? What should you tell management? SE-280Dr. Mark L. Hornick

  10. 3 3 If you had this kind of information about your estimates, how would you use it? 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 Suppose your time projection said that a project would take 8 weeks. But, your prediction interval has a range of 3 weeks. How should you make your plan? What should you tell management? SE-280Dr. Mark L. Hornick

  11. 6 6 If you had this kind of information about your estimates, how would you use it? 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 Suppose your time projection said that a project would take 8 weeks. What if the range was 6 weeks? How should you make your plan? What should you tell management? SE-280Dr. Mark L. Hornick

  12. 70% limits(area) The prediction interval is based on the t distribution. Regression-projected value Range Lower prediction interval limit (LPI) Upper prediction interval limit (UPI) SE-280Dr. Mark L. Hornick

  13. Prediction Interval Usage • Range within which data is likely to fall • Assuming variation is this estimate is similar to that in prior estimates • PSP uses 70% and 90% limits • Computes range in which actual value will likely fall • 70% of the time • 90% of the time • Helps to assess planning quality SE-280Dr. Mark L. Hornick

  14. To get the prediction interval, we must calculate the range: Text, page 128; may have error in formula (n instead of d), depending on textbook revision. Note: this is for one independent variable.

  15. For multiple regression, the range calculation is just extended a little.

  16. The s ("sigma") value is computed in the following way. xi,j = previous independent variable values yi = previous dependent variable (estimate) values n = number of previous estimates m = number of independent variables d = n-(m+1) [degrees of freedom] bj = regression coefficients calculated from previous data Same for one independent variable. Alternate form:

  17. -t t The range formula requires us to find the integration limit that yields the correct integral value. Two-sided integral value = p For a 70% interval,we want p = 0.70 Question: what integration limit “t” gives this value? For a 90% interval,we want p = 0.90 We have to search (try t values) in order to find out.

  18. When thinking about searching for the desired integral value, it may be helpful to plot the integral of the t-distribution function. Hint: create a "function object" that calculates the two-sided p integral, given a specified t value.

  19. The calculation needed is the reverse of that used in the significance calculation, since we are seeking "t" instead of "p". p For a specified "p" (integral) value, we want to find the corresponding "t" (integration limit). t How should we do the search? SE-280Dr. Mark L. Hornick

  20. In the significance calculation, we calculated "p" for a given "t"; now we are seeking "t" that will give us a desired "p". p For a specified "p" (2-sided integral) value, we want to find the corresponding "t" (integration limit). t How should we do the search?

  21. The textbook's suggests a state-machine approach that requires the function to be monotonic (pg. 246). p t The sign of the error (desired versus actual "p") tells you whether to increase or decrease the trial "t" value to get closer to the desired answer. The increase/decrease step size is halved when changing search direction.

  22. An alternative search method brackets the answer and bisects the interval.

  23. How does the interval bisection method work? error (+) p error (-) t

More Related