1 / 20

BRACE: B ootst R ap based A nalogy C ost E stimation

BRACE: B ootst R ap based A nalogy C ost E stimation. Automated support for an enhanced effort prediction method I. Stamelos, L. Angelis Aristotle Univ. Thessaloniki E. Sakellaris Singular International. Cost Estimation Methods. Expert judgement (experience – based estimation)

kelton
Download Presentation

BRACE: B ootst R ap based A nalogy C ost E stimation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. BRACE: BootstRap based Analogy Cost Estimation Automated support for an enhanced effort prediction method I. Stamelos, L. Angelis Aristotle Univ. Thessaloniki E. Sakellaris Singular International Aristotle University/Singular Int’l

  2. Cost Estimation Methods • Expert judgement (experience – based estimation) • Algorithmic cost estimation (statistical models such as regression) • Estimation by analogy (Case Based Reasoning-comparison) Aristotle University/Singular Int’l

  3. Estimation by Analogy (EbA) • Characterise new project by certain attributes and place it into a historical data set Aristotle University/Singular Int’l

  4. Estimation by Analogy (cont’d) • Calculate distances of the new project from the completed ones and find few "neighbour" projects. • Estimate the unknown effort by a location statistic (mean, median) of the efforts of the neighbour projects Aristotle University/Singular Int’l

  5. Distance Metrics Aristotle University/Singular Int’l

  6. Efficiency of EbA • Shepperd and Schofield (1997): Superiority of EbA when compared to OLS regression models with 9 industrial datasets (AngelTool used for EbA calibration) • Other researchers (Myrtveit-Stensrud, Briand et al., Jeffery et al.) have recently observed contradicting results Aristotle University/Singular Int’l

  7. Research Issues in EbA • Method Accuracy • Extend calibration options: choice of distance metric • Application of EbA with proper historical data • Generation of Interval Estimates • Calculation of Confidence Intervals (%CI) • Other measures of accuracy (bias, etc) • Angelis, Stamelos, ‘A Simulation Tool for ...’, EMSE, March 2000 Aristotle University/Singular Int’l

  8. Bootstrap • Non parametric bootstrap: • Draw with replacement from the sample, a large number of new samples (of same size) • Estimate each time the effort of the new project • Use the empirical distribution (or an estimation) of the bootstrap samples in order to obtain confidence intervals • Parametric bootstrap:based on the multivariate distribution of the original dataset Aristotle University/Singular Int’l

  9. EbA calibration with BootstrapMMRE - PRED(25) Distributions(Albrecht data set) Aristotle University/Singular Int’l

  10. Confidence Interval Estimationwith Bootstrap Aristotle University/Singular Int’l

  11. Comparison of EbA and Regression Confidence Intervals(Abran-Robilland Data Set) Aristotle University/Singular Int’l

  12. BRACE Functions • Definition of attributes and project characterization • Project/attribute management (e.g. exclusion of projects/attributes from calculations) • Calibration of EbA (with and without bootstrap) including the various distance metric options • Generation of estimations for a single project (with and without bootstrap) • Typical utility functions and file management facilities Aristotle University/Singular Int’l

  13. Aristotle University/Singular Int’l

  14. Aristotle University/Singular Int’l

  15. A case-study on software projectsfor the industry • The ISBSG Cost Data Base • International Software Benchmarking Standards Group (Australia) • Non profit organization collecting software project data from around the world • Release 6 contains 789 software projects from 20 countries Aristotle University/Singular Int’l

  16. ISBSG Project Data • Project Nature (Organisation Type, Business Area Type, Application Type, …) • Project Work Effort Data (man-hours) • Project Size Data (Function Points) • Project Quality Data (defects) • ... Aristotle University/Singular Int’l

  17. Supply Chain ISBSG Project Subset • 59 projects implementing information systems for manufacturing, logistics, warehouse management, … • characterized through effort, size, elapsed time, team size, project nature attributes • accurate project attribute measurement • average productivity ~ 190 FP/ 1000 mh Aristotle University/Singular Int’l

  18. BRACE Application • Various strategies were tried because of missing values in project characterisation • Best strategy pursued a trade-off between number of projects and attributes • Precision was measured through project jackniving • Different treatment for elapsed time and max team size Aristotle University/Singular Int’l

  19. EbA Precision Results • Best parameter configuration: 30 projects, Canberra distance, one analogy, size adjustment: MMRE = 28.84%, PRED(25) = 46.67% • When using elapsed time and team size Minkowski, λ=3 distance MMRE = 23.84%, PRED(25) = 70.37% Aristotle University/Singular Int’l

  20. Future work • Project portfolio estimation • Clustering of the cost dataset • Implementation of Parametric Bootstrap • Optimisation techniques in calibration • Replication of the study with new ISBSG release (~1000 projects) Aristotle University/Singular Int’l

More Related