COSATMO/COSYSMO Workshop

1. COSATMO/COSYSMO Workshop Jim Alstad, USC-CSSE Gan Wang, BAE Systems Garry J Roedler, Lockheed Martin April 30, 2014

2. Workshop Agenda • Reminder: general directions for COSYSMO 3.0 development • Delphi for SEFR and SEWR category parameter values • Overview of SEFR and SEWR • Instructions for Delphi questionnaires • Discussion of extending COSYSMO to estimate development costs

3. COSATMO/COSYSMO 3.0 Direction • Several factors affecting the COSYSMO cost model have been shown to be valuable in increasing estimation accuracy: • Reuse (simple model—SEWR) • Reuse (with SEFR) • Requirements volatility (SERV) The rating scales for these could be integrated into a comprehensive COSYSMO model. • Which should provide more accurate estimates than any of these alone • Add additional data points exhibiting a range of values for SEWR, SEFR, SERV • Fit a COSYSMO III model to the overall dataset • Add variables and/or subset the data as needed

4. COSATMO/COSYSMO Extension to Total Development Costs • Explore a model for total development cost based primarily on the COSYSMO parameters (Roedler) • Can such a model be improved by dividing development cost into three parts: system engineering, hardware engineering, software engineering? (Alstad)

5. Workshop Agenda • Reminder: general directions for COSYSMO 3.0 development • Delphi for SEFR and SEWR category parameter values • Overview of SEFR and SEWR • Instructions for Delphi questionnaires • Discussion of extending COSYSMO to estimate development costs

6. What is this Delphi all about? • A Generalized Reuse Framework implemented in COSYSMO • To assess not only the effort to leverage but also the effort to investin a development project • The goal is an industry-wide model for design sensitivity analysis and product line investment decisions • To bring back to organization to further tailor or use • Invite your participation to calibrate the model coefficients, through a Delphi • The Delphi process today…

7. Two Fundamental Reuse Processes

8. CONOPS – Reuse in a Project Total Project Effort = SEWR Effort + SEFR Effort

9. Generalized Reuse Framework:System Engineering for Reuse • System Engineering for Reuse produces artifacts intended for later reuse on projects. A completed SEFR artifact may (intentionally) not be completely developed, so that it will be in one of these SEFR states: • Conceptualized for Reuse (e.g., Concept of Operations document) • Designed for Reuse (e.g., component detailed design) • Constructed for Reuse (e.g., integrated component) • Validated for Reuse (e.g., validated component)

10. Generalized Reuse Framework:System Engineering for Reuse

11. Generalized Reuse Framework:System Engineering with Reuse • System Engineering with Reuse is project System Engineering, with reusable artifacts being brought into the product • A special case: zero reusable artifacts • Each reusable artifact is included in one of these SEWR states of maturity: • New (i.e., not reused) • Re-implemented (through requirements & architecture) • Adapted (through detailed design) • Adopted (through implementation) • Managed (through system verification & validation)

12. Generalized Reuse Framework:System Engineering with Reuse

13. COSYSMO – Context • COSYSMO • COnstructiveSYStems Engineering Cost MOdel • Parametric Estimate of the Systems Engineering Effort • Covers full systems engineering lifecycle • Originally developed by Dr. Ricardo Valerdi and Dr. Barry Boehm at USC • Inception of COSYSMO 1.0 • Valerdi, R., The Constructive Systems Engineering Cost Model (COSYSMO), PhD Dissertation, University of Southern California, May 2005. • Introduced the Reuse Model Extension to COSYSMO 2.0 • Wang, G., Valerdi, R., Ankrum, A., Millar, C., and Roedler, G., “COSYSMO Reuse Extension,” Proceedings of the 18th INCOSE International Symposium, June 2008. • Fortune, J. Estimating Systems Engineering Reuse with the Constructive Systems Engineering Cost Model (COSYSMO 2.0). Ph.D. Dissertation. University of Southern California. December 2009 • Wang, G., Valerdi, R., Fortune, J., “Reuse in Systems Engineering,” IEEE System Journal, v4, No.3, 2010. • Marching to COSYSMO 3.0(work in progress…) • Fortune, J. and Valerdi, R., “Considerations for Successful Reuse in Systems Engineering,” AIAA Space 2008, San Diego, CA, September 2008. • Wang, G. and Rice, J., “Considerations for a Generalized Reuse Framework for System Development,” Proceedings of the 21st INCOSE International Symposium, June 2011. • Peña, M. Quantifying the Impact of Requirements Volatility on Systems Engineering Effort. Ph.D. Dissertation. University of Southern California. August 2012. • Fortune, J. and Valerdi, R., “A Framework for Systems Engineering Reuse,” Systems Engineering, 16(2), 2013.

14. COSYSMO Overview (1/2) • COSYSMO starts by computing the “size” of a system engineering project, in units of eReq (“equivalent nominal requirements”) • These artifacts are considered in the size: system requirements, system interfaces, system-critical algorithms, and operational scenarios. • Each artifact is evaluated as being easy, nominal, or difficult. • Each artifact is looked up in this size table to get its number of eReq, and then these are summed to get the system size:

15. COSYSMO Overview (2/2) • Size is raised to an exponent, representing diseconomy of scale, and then multiplied by factors for 14 effort multipliers and a calibration constant. • This results in the following equation for a COSYSMO estimate of effort in person-months:

16. Generalized Reuse Framework:SEFR Model Equations • A SEFR estimate adjusts each artifact’s size contribution by considering its SEFR state according to this table (from the first Delphi):

17. Generalized Reuse Framework:SEWR Model Equations • A SEWR estimate adjusts each artifact’s size contribution by considering its SEWR state according to this table (from the first Delphi):

18. The Exercise – Mapping Activities to Reuse Categories X • Format of Questionnaire Spreadsheet: • 1st tab: definition of reuse categories • 2nd tab: DWR questionnaire • 3rd tab: DFR questionnaire • 4th tab: definition of EIA-632 activities • You do: • Fill out the cells intersecting EIA-632 activities (rows) and the reuse categories (columns) • Enter the percentage you believe that particular activity applies or is relevant to a particular reuse category • Fill in a few words of your rationale for each selection

19. Helpful Hints • When evaluate the questionnaire, frame your thinking around the COSYSMO construct, i.e., a group of • Requirements • Interfaces • Algorithms • Scenarios • Based on your overall professional experience, not a single or a particular program experience • Seek nominal behavior, biased toward desirable/more successful reuse situations • The handouts have the definitions of reuse categories and EIA-632 activities • For definition of general (engineering) terms, use INCOSE SE Handbook 19

20. Questions and Comments on Delphi Contacts:

21. Workshop Agenda • Reminder: general directions for COSYSMO 3.0 development • Delphi for SEFR and SEWR category parameter values • Overview of SEFR and SEWR • Instructions for Delphi questionnaires • Discussion of extending COSYSMO to estimate development costs

22. Extending COSYSMO To Estimate Development Cost • The general goal: Use a model that is very similar to COSYSMO to estimate development cost • There is some evidence that system engineering cost has some correlation with development cost • Where to go from here? • Especially, who has data from actual projects that they are willing to contribute?