Reliability Improvement Working Group (RIWG) Status Report

1. Reliability Improvement Working Group (RIWG) Status Report Patrick Dallosta Performance Learning Director – Life Cycle Logistics (Sustainment) LCL FIPT 14 Nov 08

2. Discussion Path • Defense Science Board Task Force on Developmental Test & Evaluation • Reliability Improvement Working Group (RIWG) Activities • O-FIPT Briefing 4 Aug 08 • AT&L/SSE Systems Engineering Forum / RIWG Continuance • LCL Career Field Implications / Path Ahead

3. Defense Science Board (DSB) Problem StatementOur Systems Do Not Meet Requirements "Operational effectivenessis the overall degree of mission accomplishment of a system when used by representative personnel in the environment planned or expected for operational employment of the system considering organization, doctrine, survivability, tactics, vulnerability and threat. "Operational suitabilityis the degree to which a system can be satisfactorily placed in field use, with consideration given to reliability, availability, compatibility, transportability, interoperability, wartime usage rates, maintainability, safety, human factors, manpower supportability, logistics supportability, documentation, training requirements, and natural environmental effects and impacts.

4. DSB Findings • Acquisition Management / Processes • Strong OSD/Service leadership commitment is vital to solving the major acquisition problems, to include widespread suitability deficiencies • Acquisition Reform improved flexibility, but sometimes resulted in less discipline in program formulation and execution • High Suitability failure rates were caused by lack of a disciplined systems engineering process, including a robust reliability growth program, during system development. • DT&E needs improvement, but changes in test processes will not remedy systemic program formulation and execution deficiencies • Additional emphasis on integrated testing will improve T&E process efficiency and allow program cost reductions. • Governance • RAM shortfalls are frequently identified in DT, but program constraints (schedule and funding) often preclude incorporating fixes and delaying IOT&E. • Workforce Erosion • Workforce cuts have had a significant adverse impact of DoD Acquisition capability. • Acquisition personnel reductions combined with loss of guidance documents and retirement of experienced personnel have exacerbated the adverse impact.

5. DSB Recommendations • Requirements / Processes • Identify and define RAM requirements during the JCIDS process, and incorporate them in the Request for Proposal (RFP) as a mandatory contractual requirement • Develop a military standard for RAM development and testing that can be readily referenced in future DoD contracts • Governance • Strengthen Program Manager accountability for RAM-related achievements • During source selection, evaluate the bidders’ approaches to satisfying RAM requirements, to include flow-down of RAM requirements to subcontractors, and development of leading indicators to evaluate RAM achievement • Make RAM, to include a robust reliability growth program, a mandatory contractual requirement and document progress as part of every major program review • Ensure that a credible reliability assessment is conducted during the various stages of the technical review process and that reliability criteria are achievable in an operational environment • Workforce Reconstitution • Ensure an adequate cadre of experienced RAM personnel is part of the Service acquisition and engineering office staffs

6. DUSD(A&T) / DOT&E Joint Memorandum 15 Feb 2008 • Noted Defense Science Board Task Force on Developmental T&E Findings “…poor operational suitability primarily as a result of RAM deficiencies” “…appropriate time to change how we do business with regard to RAM” • Established Reliability Improvement Working Group (RIWG) • WG IEnsure programs are formulated to execute a viable systems engineering strategy from the beginning, including a reliability growth program, as an integral part of design and development • WG IIEnsure government organizations reconstitute a cadre of experienced T&E and RAM personnel. • WG IIIImplement mandated integrated DT and OT, including the sharing and access to all appropriate contractor and government data and the use of operationally representative environments in early testing.

7. Working Group Accomplishments • WG I Ensure programs are formulated to execute SE/RAM • USD(AT&L) Memorandum, Reliability, Availability and Maintainability Policy (21 Jul 08) • WG II Ensure the Government Reconstitutes a Cadre of Trained RAM/T&E Personnel • USD(AT&L) Memorandum, Workforce Improvements in Test & Evaluation and RAM (Proposed to RIWG but not Staffed) • AT&L/SSE Memorandum to the President of DAU, Addition of Reliability, Availability and Maintainability (RAM) Courses to Selected DAWIA Certification Career Paths (29 July 08)

8. Directs Services/Agencies To Establish A Reliability Improvement Acquisition Policy Ensure effective collaboration between requirements and acquisition communities in the establishment of RAM requirements that balance funding and schedule while ensuring system suitability and effectiveness in the anticipated operating environment. Ensure development contracts and acquisition plans evaluate RAM during system design Evaluate the maturation of RAM through each phase of the acquisition life cycle Evaluate the appropriate use of contract incentives to achieve RAM objectives Provides Framework For The Policy Integrated within the Systems Engineering processes Documented in the Program’s Systems Engineering Plan and Life Cycle Sustainment Plan Assessed in Technical Reviews, Test & Evaluation and Program Support Reviews USD(AT&L) Memo Reliability, Availability and Maintainability Policy

9. Outlines OUSD Policy Support Develop RAM-C Rationale Report Manual Develop supporting guidance in the Defense Acquisition Guidebook, DoD/Service Directives and Instructions Defines Desired Outcome “… establishing reliability improvement policy in each Service, with appropriate oversight, will better implement the mandatory Materiel Availability Key Performance Parameter (KPP) and Materiel Reliability and Ownership Cost Key System Attributes (KSAs).” USD(AT&L) Memo Reliability, Availability and Maintainability Policy

10. Would have directed the Services/Components to execute human resource plans to develop, retain, and situate RAM expertise across the Acquisition Enterprise Recruit and hire personnel for T&E and RAM related positions Ensure Key Leadership Positions (KLP) include T&E and RAM personnel Retain and recognize RAM expertise Develop and train RAM expertise through Defense Acquisition University (DAU) curriculum , Defense Acquisition Workforce Improvement Act (DAWIA) career field certifications, and advanced training and education programs Strengthen and apply RAM expertise throughout the workforce, with emphasis at the systems command level, test organizations and requirements development organizations The RIWG concluded the USD(AT&L) RAM policy memo was a sufficient initial step for workforce reconstitution activity across DoD. The RIWG recommended that activity to assure workforce sufficiency be a focus area in future reviews of DoD implementation of this policy, USD(AT&L) Memo (Proposed but not staffed)Workforce Improvements in Test & Evaluation (T&E) and RAM

11. AT&L/SSE Memorandum for President, DAU“Addition of RAM Courses to Selected DAWIA Certification Career Paths” “The RIWG identified a critical need for improved knowledge within the DAWIA workforce…and recommends…” “…adding courses…to selected career fields…” (Per Attachment) “…consider the establishment of a Learning Center of Excellence to support…RAM education and training via mechanisms such at Rapid Deployment Training, Targeted Training, and CoP/SIA products…” “Request…the O-FIPT consider these recommendations and provide a strategy to address them within 30 days…”

12. Memorandum AttachmentDAWIA Career Field Recommended Changes POTENTIAL NEW COURSE (Section 852) LOG 211 Supportability Analysis 36 hours classroom instruction – builds on and expands LOG 201 and LOG 203’s focus on Supportability Analysis, Maintenance Planning,RAM, and Life Cycle Cost – includes rigorous mathematical analysis.

13. O-FIPT Briefing / Action Items Action Due Status RIWG will provide recommendations for strengthening LOG 203 29 Aug 08 Completed and CLE 301 to DAU DAU (LCIC) will review the feasibility of establishing of a 12 Sep 08 Open pending Learning Center of Excellence to support extended RAM staffing additions education and training ECD 31 Dec 08 Executive Secretaries will review the RIWG recommendations with 15 Dec 08 In progress their Functional Leader and supporting FIPT(s) as appropriate provide a response back to the President DAU, Attn: Director LCIC, regarding their decision. Executive Secretaries confirm with Mark Camporini when and how 25 Aug 08 Unknown Ms Baldwin’s recommendations will be addressed so that a response can be provided for Ms Baldwin regarding the development of a strategy to address the recommendations.

14. AT&L/SSE Systems Engineering Forum / RIWG Continuance • The RIWG published its Final Report 29 Sep 2008 • Service/Agency implementation is on-going • AT&L Systems and Software Engineering (SSE) is charged with continuing the RIWG initiatives via its Systems Engineering (SE) Forum • Point of Contact is Mr. Don Gelosh SSE/Enterprise Development Lead for Education and Training donald.gelosh.ctr@osd.mil 703.602.0851 x 194

15. LCL Career Field Implications / Path Ahead • Human Strategic Capital (HCS) Competencies • Logistics Design Influence • Integrated Logistics Support Planning • Product Support & Sustainment • Configuration Management • Reliability & Maintainability Analysis • Technical/Product Data Management • Supportability Analysis Path Ahead Insert / Strengthen “R” “A” & “M” in the Career Field Architecture and Curriculum via updates and course revisions • Workforce Reconstitution • Recruit and hire personnel for T&E and RAM related positions • Ensure Key Leadership Positions (KLP) include T&E and RAM personnel • Retain and recognize RAM expertise • Develop and train RAM expertise • Strengthen and apply RAM expertise Path Ahead Establish “Workforce Improvement” Memo tenants as focus areas for Implementation. Ensure consistency with other Career Fields

16. Questions & Answers