Agenda

1. 21st International Forum on COCOMOand Software Cost ModelingSystems Engineering Cost Estimation: System-of-SystemsJon K. IlsengPrincipal Systems EngineerRaytheon Network Centric SystemsNovember 8, 2006

2. Agenda • Introduction • Systems Engineering Definitions • SECOST History • SECOST Capabilities and Functionality • SECOST Cost Estimation Mode: System-of-Interest • System-of-Systems Definition • System-of-Systems Examples • SECOST Cost Estimation Mode: System-of-Systems • SECOST Recommended Modifications • Summary and Conclusions

3. Introduction • Systems Engineering Cost Estimates • Our customers require believable and accurate estimates • OSD AT&L encouraging defense contractors to find “most accurate and consistent systems engineering cost estimation method” • Critical that all 5 Raytheon Business Units (IIS, NCS, SAS, IDS, RMS) submit accurate, consistent, and believable cost estimates

4. Introduction • Systems Engineering Cost Estimates • Various cost estimates used past 20 years for a “system-of-interest” • Heuristic and rule of thumb – Systems Engineer use knowledge & experience to prepare cost estimates; no documented written evidence • Expert Opinion – From SE Domain Expert; no scientific or historical basis • Case Studies – Provide vital information; no scientific basis for how cost estimates actually prepared • Top down and design-to-cost – Top-down approach starts at defined system level; tries to capture SE Tasks but not scientific or historical basis; DTC is designing a system to meet cost targets • Bottoms-up – Common approach beginning with lowest level cost component and rolls up to highest level for total estimate; resource intensive effort which is time-consuming and many times no actual historical data to justify estimate • Parametric – Employs cost estimating relationships (CERs); most accurate cost estimation method; provides repeatable and most credible estimation framework; less time-consuming than bottoms-up

5. Introduction • Parametric Cost Estimation Method • October 2003 OSD SE Summit • OSD’s position is parametric-based estimates are recommended technique for preparing SE Cost Estimates • Why Parametric Cost Estimation Method • Provides a credible source • Shortens cost estimating cycle times • Creates more easily defended negotiation position with customer • Reduces customer-approval cycle times • Uses historical data to improve quality of cost estimates • Establishes greater consistency in cost estimating process • Consortium developed Constructive Systems Engineering Cost Model (COSYSMO) parametric model • U.S. Defense Contractors including Raytheon • International Council of Systems Engineering • University of Southern California Center for Software Engineering

6. Introduction • COSYSMO Cost Model • Accurately estimates time & effort for SE tasks • Parametric-based cost model • Successfully defended August 2005 doctoral disseration • Open public domain model • SECOST Tool • Developed by Raytheon IIS-Garland for SE Cost Estimates • Uses COSYSMO as its embedded estimation engine • Prepares SE Cost Estimates for System-of-Interest • Raytheon proprietary • Could SECOST Tool prepare SE Cost Estimates for System-of-System • Yes, if SECOST additions & modifications are implemented

7. Systems Engineering Definitions • Systems Engineering • Various definitions across SE Domain • Applying scientific & engineering efforts to integrate related technical parameters • System solution which satisfies customers’ expectations • Interdisciplinary approach which involves integrating various engineering disciplines (i.e., electrical design, software design, hardware design • Notice something missing from these various definitions! • No mention of cost estimating, however, does not diminish importance of it • Pro-active aggressive SE process during program life-cycle • Lower life cycle costs • High system quality & enhanced technical solution • Minimizes cost & schedule overruns

8. System • A system • is completely composed of • a set of interacting • system elements • System Element • System Element • System Element System Structure Systems Engineering Definitions • System-of-Interest • Defined by ISO/IEC 15288 architectural structures • Comprised of interacting system elements • Each system element independent of each other; can operate on their own • Only provides real value when connected together to provide system functions

9. SECOST History • Suite of MS Excel spreadsheets • Generates, documents, and archives SE cost estimates within single process-focused framework • Uses open COSYSMO parametric model • Developed in early 2004 from MyCOSYSMO • MyCOSYSMO leveraged off SWCOST software engineering estimation model • Proprietary version of MyCOSYSMO • Raytheon IIS and NCS Business Units currently collecting historical program data for local calibration

10. SECOST Capabilities and Functionality • Supports ROMs, budgetary estimates, formal proposal bids • Supports multiple levels of estimate formality & complexity • Consists of SECOST Framework • USC COSYSMO is embedded engine • Interfaces with standard Raytheon Pricing Systems • Supports Cost Volume & generates Basis-of-Estimates • In-Process & Historical Data Collection • Results used for local COSYSMO model calibrations • Local COSYSMO model calibrations feeds USC COSYSMO • Current Raytheon NCS Systems Engineering Cost Enabler • Using SECOST Size Drivers (Requirements, Interfaces, Algorithms, Operational Scenarios) • Using SECOST Size Drivers Complexity Criteria • Using SECOST EREQ Conversion and Reuse Factors

11. SECOST Cost Estimation Mode: System-of-Interest • Cost estimation mode prepare SE cost estimates for future pursuits • Data collection mode collects SE labor hours expended during program execution • Cost Estimation Mode 15 steps • 1) Initialize Project Parameters (e.g. project name, period of performance, type of estimate) • 2) Enter SE Contractor Work Breakdown Structure (e.g. Technical Management, IV&V, Requirements Definition & Validation) • 3) Document Project Assumptions • Assumptions always associated with SE cost estimates • 4) Document and Register Project Risks • Program risks always associated with SE cost estimates

12. SECOST Cost Estimation Mode: System-of-Interest (continued) • Cost Estimation Mode 15 steps (continued) • 5) Set COSYSMO Effort Multipliers (continued) • Application Effort Multiplier – evaluates specific COSYSMO application factors on scale from Very Low to Extremely High • Requirements Understanding • Architecture Understanding • Level of Service Requirements • Migration Complexity • Number & Diversity of Installations/Platforms • Number of Recursive Levels in Design • Documentation to match lifecycle needs • Technology Risk

13. SECOST Cost Estimation Mode: System-of-Interest (continued) • Cost Estimation Mode 15 steps (continued) • 5) Set COSYSMO Effort Multipliers (continued) • Team Effort Multiplier – evaluates specific COSYSMO team factors on scale from Very Low to Extremely High • Stakeholder Team Cohesion • Personnel/Team Capability • Personnel Experience/Continuity • Process Capability • Multisite Coordination • Tool Support • 6) Determine Labor Distributions among Raytheon Salary Labor Grades • 7) Estimate Four SE Size Drivers • System-Level Requirements • Decompose system-of-interest objectives & capabilities into requirements that can be tested, verified, or designed • Count number of requirements (“shalls”) in system specification; only requirements managed by SE – not HW or SW

14. SECOST Cost Estimation Mode: System-of-Interest (continued) • Cost Estimation Mode 15 steps (continued) • 7) Estimate Four SE Size Drivers (continued) • System-Level External & Internal Interfaces • Functional interfaces (e.g. protocols or timing requirements) not physical interfaces (e.g. number of wires) • Interfaces that involve SE for your defined system-of-interest • Only count number of unique interface types – not every interface • System-Level Algorithms • Algorithm sources are functional block diagram, mode description document, system specification, etc. • System-Level Operational Scenarios • Typically quantified by number of system test thread packages, unique end-to-end tests, number of use cases • 8) Determine Effort Hours • Outputs total SE hours and equivalent requirements (EREQs)

15. SECOST Cost Estimation Mode: System-of-Interest (continued) • Cost Estimation Mode 15 steps (continued) • 9) Time Phase SE Estimate • Spread total SE hours among CWBS • 10) Submit to Pricing Group • Pricing analyst processes SECOST file (e.g. adds appropriate Raytheon Business Unit, correct CLIN, other pricing variables) • 11) Process Pricing Group Data • After pricing analyst processes SECOST file, sent back to SE estimator to copy and paste process SECOST file into SECOST Worksheet • 12) Conduct Internal Estimate Review • Internal review among SE Estimator, Lead SE, Program Manager • 13) Determine and Signoff Final Bid • After internal SE Review completed & approved by SE Center Director, final cost estimate presented to Raytheon Senior Management

16. SECOST Cost Estimation Mode: System-of-Interest (continued) • Cost Estimation Mode 15 steps (continued) • 14) Finalize Management Bid Review Charts • SECOST provides four management review package charts • SE Labor Cost Summary Chart • Past Program SE Sizing and Unit Cost • Monte Carlo Output Sample Distribution • Monte Carlo Output Cost “Probability of Success” • 15) Archive the Estimate • Most important step; provides rationale and data if questions or issues are raised during cost estimation phase

17. System-of-Systems Definition • System-of-Systems (SoS) is not the same as a Family-of-Systems (FoS) • FoS do not create capability beyond additive sum of member systems’ individual capabilities • FoS belong to domain or product lines (e.g. family of missiles, family of aircraft) • FoS lacks synergy of a SoS • FoS do not acquire qualitatively new properties as result of its grouping • U.S Department of Defense (DoD) SoS Definition • “A SoS is a set or arrangement of interdependent systems that are related or connected to provide a given capability. The loss or any part of the system will significantly degrade the performance or capabilities of the whole. The development of a system of systems solution will involve trade space between the systems as well within an individual system’s performance.”

18. System-of-Systems Definition • My thesis used the U.S. DoD SoS definition • SoS Characteristics • Researched five main sources which truly defined SoS • Addressing the System of Systems Challenge Paper • Purdue University School of Aeronautics & Astronautics • “Systems of System Approaches in U.S. Department of Defense” presentation at 1st Annual SoS Engineering Conference • SoS Engineering Center of Excellence • “System of Systems Engineering” presentation at 1st Annual SoS Engineering Conference • Four SoS common characteristics shared by five SoS definitions • Emergence – Whole is greater than the sum of its parts; SoS behave as collective whole, interacting with its environment to adapt and respond • Independence – Each system within SoS can operate on their own • Lack of Ownership – SoS does not have an identified owner at SoS level • Evolutionary – SoS is never completely formed; continues to be a living system

19. System-of-Systems Examples • U.S. DoD Programs • SoS examples in commercial world (e.g. internet) • Focus on U.S. DoD Programs • Raytheon’s primary customer is U.S. DoD • DoD driving towards mandating “jointness for services (i.e Air Force, Army, Navy, Marine Corps) • Current GWOT & OIF campaigns have Airmen, Sailors, Soldiers, Marines fighting together as joint units • Emphasis on jointness forced services to develop warfighting strategies to support joint warfighting • Current C2ISR, communications & computers, COE capabilities need integration to support joint warfighting • Integration of these capabilities provides integrated capability-centric jointness system; in other words a SoS

20. System-of-Systems Examples • Future Combat Systems SoS • SoSCOE • Software that allows various systems to operate seamlessly • Approximately 35 million lines of code • Battle Command Software • Consists of four software packages • Mission Planning & Preparation, Situation Understanding, Battle Command & Mission Execution, Warfighter-Machine Interface • Communications & Computers • FCS SoS connected to C4ISR network by multilayered Communications & Computer network • Network provides secure access to information sources over extended distances & complex terrain • ISR • Distributed & networked array of ISR Sensors • Networked Logistics Systems • Integrates logistics into C4ISR network

21. System-of-Systems Examples

22. System-of-Systems Examples • DoD Distributed Common Ground System SoS • Combination of U.S. Air Force, Army, Navy, and Marine Corps ground and surface systems • Each service’s DCGS consists of elements and processes, exploits, and posts ISR sensor data • Each service’s DCGS consists of legacy systems • DoD currently preparing migration plans to integrate all service DCGS elements • Achieves a net-centric DCGS • Integration of services’ DCGS is referred to as DoD DCGS SoS • DoD mandated DCGS SoS migrate to net-centric warfare & net-centric DCGS Enterprise

23. System-of-Systems Examples Improves accuracy and timeliness of intelligence provided to warfighter Promotes standards-based ISR infrastructure

24. System-of-Systems Examples • Land Warrior SoS • High-tech SoS which provides U.S. Army soldier enhanced capabilities • Integrated fighting system which helps increase soldier’s • Lethality • Battle Command Compatibility • Survivability • Mobility • Awareness • Situational Awareness • Combat effectiveness

25. System-of-Systems Examples • Land Warrior SoS • Consists of following subsystems • Weapon Subsystem • Integrates weapon-mounted sensors (multifunction laser, daylight video sight, thermal weapon sight) • Soldier Control Unit • Provides primary user interface to system functions • Personal Area Network Cables • Distributes power & data through the system • Personal Clothing & Individual Equipment • Consists of utility belt & subsystem pouches • Computer/Master Hub Subsystem • Provides control of system functions

26. System-of-Systems Examples • Land Warrior SoS • Consists of following subsystems • Power Source Subsystem • Provides centralized power from dual disposable or rechargeable batteries • CommsNet Radio Subsystem • Provides transmit/receive voice & data capability • Navigation Subsystem • Provides position location data to the soldier & time reference to system • Helmet Subsystem • Provides full-color display for computer interface

27. System-of-Systems Examples • NASA Exploration SoS • Represents U.S. President’s vision for U.S. space exploration • New capabilities & systems enabling safe & successful human & robotic missions

28. System-of-Systems Examples • NASA Exploration SoS • Consists of following subsystems • Crew Transportation System • Flight elements which deliver human crew from Earth to mission destination & return crew safely to Earth • Cargo Delivery System • Delivers all non-crew exploration vehicle flight elements to accomplish human exploration objectives • Ground Support System • Provides all common ground-based capabilities needed to execute exploration missions • Robotic Precursor System • Provides measurements, technology, & demonstrations in advance of human missions • In-Space Support System • Encompasses capabilities provided by space-based infrastructure elements (e.g. communications, navigation, surveillance) • Destination Surface System • Encompasses all elements necessary to enable long-duration human exploration mission

29. SECOST Cost Estimation Mode: System-of-Systems • Can SECOST be used for SoS • Yes, with recommended modifications & additions • SECOST does not account for SoS characteristics • Emergence, Independence, Lack of Ownership, Evolutionary • Complex Integration Efforts • Many domains involved • Design Optimization • Approach not feasible for SoS; have to evaluate each system within SoS to determine best optimal design • Complex interface design & management • Management of SoS interfaces more difficult than for system-of-interest • Decomposing FoS functional requirements & allocating to SoS • No SoS hierarchy; results in stovepipe solutions • SoS IV&V • More complex than system-of-interest IV&V

30. SECOST Cost Recommended Modifications • Add Software Requirements Size Driver • SoS have significant costs associated with software integration & numerous software requirements • DoD DCGS SoS has over 2,000 unique software requirements • Previously mentioned SoS programs have extensive software-level requirements

31. SECOST Cost Recommended Modifications • Add Software Modules Size Driver • Software module defined as CSCI • Newly developed, COTS, GFE • Previous mentioned SoS Programs contained numerous CSCIs • DoD DCGS SoS has over 3,000 unique CSCIs

32. SECOST Cost Recommended Modifications • Modify Interfaces Complexity Criteria • Managing SoS interfaces complex (e.g. FCS SoS has over 2,000 unique interfaces)

33. SECOST Cost Recommended Modifications • Modify COSYSMO Application Effort Multiplier • Requirements Understanding • Rates the level of understanding of the SoS system and software requirements • Migration Complexity, Number of Diversity of Installations/Platforms, Number of Recursive Levels in the Design, Documentation to Match Lifecycle Needs • Add word “SoS” to each of these COSYSMO Application Factors • Add IV&V COSYSMO Application Factor • Rates maturity & experience of performing SoS IV&V tasks • Add SoS Integration COSYSMO Application Factor • Rates maturity & experience of performing as a SoS integrator on previous SoS programs

34. SECOST Cost Recommended Modifications • Modify COSYSMO Team Effort Multiplier • Stakeholder Team Cohesion • Add the following viewpoints “What is the number of stakeholders involved?” and “Is there a defined and agreed to list of stakeholder responsibilities?” • Add word “SoS” to Personnel Experience/Continuity COSYSMO Team Factors • Add words “and experience of systems engineers who have worked on SoS programs”

35. Summary and Conclusions • 2003 OSD SE Summit • OSD’s position statement is parametric-based estimates recommended technique for preparing SE cost estimates • Raytheon IIS Garland developed SECOST • Uses COSYSMO parametric model • Currently predicts SE cost estimates for system-of-interest • DoD SoS Programs are increasing in importance and come with challenges • No credible method for performing SoS SE cost estimates • SECOST can predict SoS SE cost estimates • With additions & modifications to SECOST • SECOST can provide accurate, credible & believable SoS SE cost estimates