COCOMO II Maintenance Model Upgrade

1. COCOMO II Maintenance Model Upgrade Vu Nguyen, Barry Boehm Center for Systems and Software Engineering (CSSE) CSSE Annual Research Review 2010 Mar 8th, 2010

2. Outline Motivation Research Problem and A Solution Major Software Maintenance Estimation Models COCOMO II for Maintenance Research Validation and Preliminary Results Next Steps Expected Main Contributions References

3. Magnitude of Software Maintenance Majority of software costs incur after the first operational release [Boehm 1981] Maintenance vs. Total Software Cost 100 90 Others 80 70 Maintenance 60 50 40 % of Software Cost 30 20 10 0 Zelkowitz et McKee Moad Erlikh al. (1979) (1984) (1990) (2000) Studies Fig. 1. Software maintenance cost versus total software cost

4. Importance of Software Estimation in Managing Software Projects • Estimation is a key success factor of software projects • Two out of three most-cited project failures are related to resource estimation, according to a CompTIA survey in 2007 [Rosencrance 2007] • Cost estimate is the key information for investment, project planning and control, etc. • Many software estimation approaches have been proposed and used in industry • E.g., COCOMO, SEER-SEM, SLIM, PRICE-S, Function Point Analysis

5. The Problem • These models are built on the assumptions of new development projects • Problem is that these assumptions do not always hold in software maintenance due to differences between new development and maintenance • Low estimation accuracies achieved

6. A Solution Improved COCOMO models that allow estimators to better determine the equivalent size of maintained software and estimate maintenance effort can improve the accuracy of effort estimation of software maintenance.

7. Major Software Maintenance Estimation Models • Estimate cost for the maintenance phase after the software is delivered • All types of regular maintenance tasks are included • Most of the models use SLOC as a size input • Lack of empirical evaluation of these models

8. Outline Motivation Research Problem and A Solution Major Software Maintenance Estimation Models COCOMO II for Maintenance Modeling Process A Software Maintenance Sizing Method Effort Model Calibration Techniques Research Validation and Preliminary Results Next Steps Expected Main Contributions References

9. The Modeling Process 1 Analyze existing literature 2 • Perform maintenance experiment to validate some size measures 3A 3B • Perform Behavioral Analysis • Identify relative significance of factors • Determine sizing method for maintenance 4 • Determine form of effort model 5A 5B • Perform Expert-judgment and Delphi assessment Gather project data 6 • Test hypotheses about impact of parameters 7 • Calibrate model parameters • Determine parameter variability 8 Evaluate model performance Fig. 2. The Modeling Process, Adapted from the COCOMO Modeling Process [Boehm 2000]

10. COCOMO II for Maintenance – A Glance • An extension of COCOMO II • COCOMO is the non-proprietary most popular model • COCOMO has attracted many independent validations and extensions • Two Components • A Unified Reuse and Maintenance Model • Determining equivalent SLOC for reuse and maintenance • COCOMO Effort Model for Maintenance • Using a different set of parameters and constants

11. Preexisting Code Delivered Code Reused Modules External Modules Manually develop and maintain Adapted Modules New Modules Automatically translate Existing System Modules Automatically Translated Modules Software Maintenance Sizing • Size is a key determinant of effort • Sizing method has to take into account different types of code Fig.3. Types of Code

12. A Unified Reuse and Maintenance Model – 1/3 • Objectives • Provide a consistent size measure for both reuse and maintenance • Better account for different types of code • Changes • Use a single model for both reuse and maintenance • Redefine the reuse model equation and parameters to • account for code expansion • allow equivalent SLOC to be determined from completed code • include deleted SLOC in modified modules, but excluded SLOC in deleted modules • smooth the curve representing nonlinear effects

13. Major changes A Unified Reuse and Maintenance Model – 2/3 • New AAF and AAM equations AAF – Adaptation Adjustment Factor (% of modification) AAM – Adaptation Adjustment Multiplier DM – Percentage of Design Modified, accounting for only design changes made to the preexisting modules IM – Percentage of Integration changed, relative to the integration of the preexisting modules CM – Percentage of Code Modified, including added, modified, deleted

14. A Unified Reuse and Maintenance Model – 3/3 • Compute Equivalent SLOC (ESLOC): • New Modules: • Adapted Modules: • Reused Modules: • Total Equivalent KSLOC: : KSLOC of the adapted modules before changes : KSLOC of the reused modules

15. COCOMO Effort Model for Maintenance • Follows the same COCOMO II non-linear form Where, PM – project effort measured in person-month A – a multiplicative constant, calibrated using data sample B – an exponent constant, calibrated using data sample Size – software size measured in SLOC EM – effort multipliers, cost drivers that have an multiplicative effect on effort SF – scale factors, cost drivers that have an exponential effect on effort • Linearize the model using log-transformation log(PM) = 0 + 1 log(Size) + iSFi log(Size) + j log(EMj)

16. Calibration • Process of fitting data to the model to adjust its parameters and constants Rating scales for cost drivers New rating scales for cost drivers and Constants Delphi survey of experts (Expert-judgment estimates) Model Calibration • Calibration Techniques: • Ordinary Least Squares Regression (OLS) • Bayesian Analysis • Constrained Regression Technique [Nguyen 2008] Sample data Fig.4. Calibration Process

17. Outline Motivation Research Problem and A Solution Major Software Maintenance Estimation Models COCOMO II for Maintenance Research Validation and Preliminary Results Next Steps Expected Main Contributions References

18. Project starts for Release N+1 Release N Release N+1 Project starts for Release N+2 Timeline Baseline 2 Baseline 1 Maintenance project N+1 Data Collection – 1/2 • Collect data of completed maintenance projects from industry • Maintenance type: error corrections, enhancements, etc. Excluding reengineering and language-migration projects • CodeCount tool (UCC) is used for size collection [Nguyen 2007] Fig.5. Release Period to be Collected

19. Data Collection – 2/2 • Metrics

20. Current Status • Completed the controlled experiment • Collected 83 projects/releases • 64 from a large organization member of CSSE Affiliates • 14 from a CMMI-Level 5 company in Vietnam • 4 from a CMMI-Level 3 company in Thailand • Generated preliminary results using Bayesian and constrained regression techniques • Use cost-driver rating scales from Delphi exercise for COCOMOII.2000 (Delphi COCOMOII.2000) • Delphi survey has yet to be completed

21. +18% +16% -17% -37% Preliminary Results – 1/2 • Relative Impact of Cost Drivers on Effort • Rating scales: Delphi COCOMOII.2000 Sample data: 161 projects Sample data: 83 projects Values Generated by COCOMO II for Maintenance Using the Bayesian Analysis COCOMO II.2000 Values [Boehm 2000] Fig. 6. Productivity Ranges

22. Preliminary Results – 2/2 • Estimation Accuracies • Estimated 83 projects using • COCOMO II.2000 • COCOMO II for Maintenance: Bayesian analysis • COCOMO II for Maintenance: Constrained regression • Rating scales: Delphi COCOMOII.2000 • Computed MMRE and PRED(0.3) values:

23. Next Steps • Perform Delphi survey to obtain expert-judgmentrating scales for maintenance • Continue data collection • COCOMO data for both new development and maintenance projects • Validate the research hypotheses • Analyze and validate the models

24. Expected Main Contributions A model for sizing maintenance and reuse An extended COCOMO model for maintenance A set of cost drivers and levels of their impact on maintenance cost Empirical validations on the impact of cost drivers for software maintenance

25. References Boehm B.W. (1981), “Software Engineering Economics”, Prentice-Hall, Englewood Cliffs, NJ, 1981. Erlikh L. (2000). “Leveraging legacy system dollars for E-business”. (IEEE) IT Pro, May/June, 17-23. McKee J. (1984). “Maintenance as a function of design”. Proceedings of the AFIPS National Computer Conference, 187-193. Moad J. (1990). “Maintaining the competitive edge”. Datamation 61-62, 64, 66. Nguyen V., Deeds-Rubin S., Tan T., Boehm B.W. (2007), “A SLOC Counting Standard,” The 22nd International Annual Forum on COCOMO and Systems/Software Cost Modeling. Nguyen V., Steece B., Boehm B.W. (2008), “A constrained regression technique for COCOMO calibration”, Proceedings of the 2nd ACM-IEEE international symposium on Empirical software engineering and measurement (ESEM), pp. 213-222 Rosencrance L. (2007), "Survey: Poor communication causes most IT project failures," Computerworld Zelkowitz M.V., Shaw A.C., Gannon J.D. (1979). “Principles of Software Engineering and Design”. Prentice-Hall

26. Backup Slides