DoD Software Engineering and System Assurance New Organization – New Vision

1. DoD Software Engineering and System AssuranceNew Organization – New Vision Kristen Baldwin Office of the Under Secretary of Defense Deputy Director, Software Engineering and System Assurance February 14, 2007

2. Recent Acquisition & Technology Reorganization Under Secretary of Defense Acquisition Technology & Logistics as of June 1, 2006 Deputy Under Secretary of Defense Acquisition & Technology Director Systems & Software Engineering Systems Engineering is well positioned in DoD

3. Director, Systems & Software Engineering Mark Schaeffer SES Deputy Director Enterprise Development Deputy Director Developmental Test & Evaluation Deputy Director Software Engineering & System Assurance Deputy Director Assessments & Support Bob Skalamera SES Chris DiPetto SES Mark Schaeffer (Acting) SES Dave Castellano SES CORE COMPETENCIES CORE COMPETENCIES CORE COMPETENCIES CORE COMPETENCIES • SE Policy • SE Guidance • SE in Defense Acquisition Guidebook • Technical Planning • Risk Management • Reliability & Maintainability • Contracting for SE • SoS SE Guide • SE Education and Training • DAU SE Curriculum • SPRDE Certification Rqmt • Corrosion • R-TOC • Value Engineering • DT&E Policy • DT&E Guidance • T&E in Defense Acquisition Guidebook • TEMP Development Process • DT&E Education and Training • DAU DT&E Curriculum • DT&E Certification Rqmt • Joint Testing, Capabilities & Infrastructure • Targets Oversight • Acq Modeling & Simulation • Energy • DSOC/Acq Tech Task Force • SWE and SA Policy • SWE and SA Guidance • SoS, SA Guides • SWE and SA Education and Training • DAU SW Acq Curriculum • Continuous Learning Modules for SWE, SoS, SA • Software Engineering • Acquisition Support • Software Engineering Institute (SEI) • Process Improvement • CMMI Sponsor • DoD/National Software Investment Strategy • Support of ACAT I and Other Special Interest Programs (MDAP, MAIS) • Assessment Methodology (Program Support Reviews - PSRs) • T&E Oversight and Assessment of Operational Test Readiness (AOTR) • Systems Engineering and Developmental Test Planning and Support • Lean/6-Sigma Training/Cert Acquisition program excellence through sound systems and software engineering Systems and Software EngineeringOrganizational Core Competencies Est. Aug 06

4. Establishing a DoD Software Center of Excellence Industry • DoD SoftwareCenter of Excellence • Support Acquisition Success • Improve State-of-the-Practice of Software Engineering • Leadership, Outreach and Advocacy • Foster Resources to Meet DoD Needs DoD-Wide Partnerships National Partnerships DoD Software Center of Excellence International/Global Alliances University Consortia

5. Elements of a DoD Strategy for Software • Support Acquisition Success • Ensure effective and efficient software solutions across the acquisition spectrum of systems, SoS and capability portfolios • Improve the State-of-the-Practice of Software Engineering • Advocate and lead software initiatives to improve the state-of-the-practices through transition of tools, techniques, etc. • Leadership, Outreach and Advocacy • Implement at Department and National levels, a strategic plan for meeting Defense software requirements • Foster Software Resources to meet DoD needs • Enable the US and global capability to meet Department software needs, in an assured and responsive manner Promote World-Class Leadership for Defense Software Engineering

6. DoD Software Performance:What We’re Seeing* • Software systemic issues are significant contributors to poor program execution • Software requirements not well defined, traceable, testable • Immature architectures, COTS integration, interoperability, obsolescence (electronics/hardware refresh) • Software development processes not institutionalized, planning documents missing or incomplete, reuse strategies inconsistent • Software test/evaluation lacking rigor and breadth • Schedule realism (compressed, overlapping) • Lessons learned not incorporated into successive builds • Software risks/metrics not well defined, managed *Based on ~40 program reviews in the past 2 ½ years

7. Getting Started – What are we Doing? • Identify software issues, needs • Software Industrial Base Study • NDIA Top Software Issues Workshop • Defense Software Strategy Summit • Creating opportunities, partnerships • Established network of Government software POCs • Chartered the NDIA Software Committee • Information exchanges with Government, Academia, and Industry • Planning the Systems & Software Technology Conference, TAMPA! • Executing focused initiatives • CMMI Integrity, CMMI-ACQ, CMMI Guidebook • Handbook on Engineering for System Assurance • SoS Systems Engineering Guide • Providing software support to acquisition programs • Transparent data vision for Acquisition Reform

8. The impact of system requirements upon software is not consistently quantified and managed. Fundamental system engineering decisions are made without full participation of software engineers. Software life cycle planning and management by acquirers and suppliers is ineffective. The quantity and quality of software engineering expertise is insufficient for dealing with the scale of complexity of modern systems. 5. Software verification techniques are costly and ineffective for dealing with the scale of complexity of modern systems. 6. There is a failure to assure correct, predictable, safe, secure execution of complex software in distributed environments. 7. Inadequate attention is given to total life cycle issues for COTS/NDI impacts on life cycle cost and risk. NDIA Top Software Issues COTS = commercial off-the-shelf NDI = non-developmental item

9. Enforce effective software requirements development and management practices. Institutionalize the integration and participation of software engineering in all system activities. Establish a culture of quantitative planning and management Collaborate on approaches attract, develop, and retain qualified talent to meet current and future needs in government and industry. 5. Develop guidance and training to improve effectiveness in assuring product quality across the life cycle. 6. Develop approaches, standards, and tools, addressing system assurance issues throughout the acquisition life cycle and supply chain. 7. Improve and expand guidelines for addressing total life cycle COTS/NDI issues NDIA Top Software Recommendations

10. Software Strategy Summit EventsOctober 18-19 • Keynote Address, the Honorable Dr. James I. Finley, Deputy Under Secretary of Defense (Acquisition and Technology) • Program Executive Officer and Service/Defense Agency panels on software related acquisition issues and initiatives • Plenary Session Topics -- NDIA Top Software Issues, Software Industrial Base Study, and Software Producibility • Workshops • Software Acquisition and Sustainment • Mr. Mike Nicol, Air Force Aeronautical Systems Center • Mr. Lawrence T. Osiecki, US Army, Armament Software Engineering Center • Policy • Mr. Jim Clausen, DoD CIO, Office of Commercial IT Policy • Col Peter Sefcik, Jr., USAF Chief, Air Force Engineering Policy and Guidance Team • Lt Col Mark Wilson, SAF/AQR Systems & Software Engineering • Human Capital • Dr. Kenneth E. Nidiffer, Fellow, Systems and Software Consortium • Mr. George Prosnik, Defense Acquisition University E&T Center • Software Engineering Practices • Mr. Grady Campbell, Software Engineering Institute • Mr. Paul R. Croll, CSC, Industry Co-Chair NDIA Software Committee

11. Software Acquisition and Sustainment Software issues not addressed early in lifecycle Software requirements not well defined at program start Management has limited visibility into software development processes and status Risk areas – single point failures not adequately addressed, e.g., single software providers, incomplete data rights, key personnel stability, life cycle support of COTS Acquirers do not adequately address software sustainment and total life cycle early in the program Some agencies contract before engineering is complete, prior to system design and development Policy PMs need assistance with software policy and analysis Arbitrary separation of weapon and information technology software policies Policy implementation guidance and follow-up monitoring is limited Department needs software group with good expertise to oversee and implement policy Need capability to share policy and guidance information Summary of Workshop Issues* Software Engineering • Weak linkage between software requirements and capabilities/portfolios • System development methods do not properly leverage software ability to rapidly field new capability • Systems and software engineering lifecycles not always consistent or harmonized • Software considerations not consistently addressed in architectures • Inadequate software estimating methods, e.g., COTS/NDI; best practices not applied Human Capital • Experienced system & software engineers seem missing from key DoD leadership positions • Shortage of highly experienced software managers, architects, domain and technical experts • Eroding depth and breath of experience for personnel in DoD • Young people may consider system and software engineering as a career dead end • Emerging skill set may be needed for future complex DoD systems, e.g., systems of systems *Reaffirmed NDIA Top Software Issues (See NDIA issue slide)

12. Software Acquisition and Sustainment Require earned value reporting for software Require software sustainment planning at all milestones, e.g., in Systems Engineering Plan Improve architecting of systems and software Develop software work force competency maturity plan; balance software acquisition needs against capabilities Improve software metrics reporting and risk management Emphasize acquisition “-ilities” for software Policy Set up portal to share policy & guidance information Analyze policy, cross reference results, define commonality, evaluate policy quantitatively, update guidance, e.g., information technology and weapon systems software polices Update guidance for GOTS, COTS, software best practices, portfolio and risk mgmt, as a minimum Establish DoD software group to oversee and implement policy Require program offices to have software engineering expertise Summary of Workshop Recommendations* Software Engineering • Break-up large programs/architectures • Set up product lines • Raise software as a major factor within the work breakdown structure • Revise contracting methods to allow additional detail for software requirements • Provide leadership training to include a software appreciation • Provide cadre of no-cost experts to support PMs • Encourage independent program assessments Human Capital • Build experience bases, e.g., software communities of practice, preferred processes, tools • Consider establishing a software engineering specialty track for DoD acquisition personnel • Emphasize software acquisition management skill for DoD System Engineers and Project Managers • Ensure teams have both systems engineering and software development expertise • Assess the results of on-going service and agency human capital efforts • Encourage internal process improvement for DoD acquisition offices *Reaffirmed NDIA Top Software Recommendations (See NDIA recommendation slide)

13. Preliminary Analysis – Gap Identification • Many issues and recommendations require further review and analysis • Inventory of Service and Defense Agency on-going initiatives not complete • Analysis of existing guidance, such as, directives, instructions, plans, and standards, that are related to software management is not complete • Where possible, need to quantify software acquisition issues, establish benchmarks, and identify problem causes Need to prioritize issues and determine collective best path forward

14. Our Challenge • Given the shortage of software resources and critical software reliance • We cannot afford to be stovepiped • We must integrate across cross-functional perspectives to improve our software capability • Next Steps… • Analyze Assess summit issues and recommendations • Prioritize and begin to implement Summit recommendations • Contribute to ongoing initiatives: Systems of Systems, System Assurance, CMMI, more • Explore new areas: Agile methods for DoD, SW estimation, SW Test, more • Follow up • Track and monitor the above • Conduct a 2007 Software Summit to review these actions and recommend adjustments for 2008 and beyond.