Product Assurance Revitalization Presented By: John Deas Product Assurance Division

1. Product Assurance Revitalization Presented By: John Deas Product Assurance Division Logistics FIPT July 14, 2006

2. Agenda • Background • Policy & Guidance • Tools & Initiatives • Program Support/Systemic Analysis • Education & Training Initiatives • Conclusion

3. Overview • Mission – DCMA Provides Customer-Focused Acquisition Life Cycle and Combat Support to Ensure Readiness, Worldwide 24/7 • Represents the Military Services, other Federal Agencies and Allied Government Buying Agencies at Defense Contractors Worldwide • Prior to contract award • After contract award • Agency Goals • Transform the Agency into a customer-focused organization • Embrace a performance-based culture • Equip our employees with the competencies needed to provide extraordinary support to our customers • Ensure fiscal responsibility

4. The Way We Were Total 44 CONUS Major Contract Management Offices (CMO) DCMAE HQ DCMA HQ DCMAW HQ DCMA East (23) DCMA West (21) CMOs District HQ Agency HQ

5. Product Alignment • Product Divisions: • Aeronautical Systems - Boston • Naval Sea Systems - Boston • Space and Missile Systems- Carson • Ground Systems and Munitions - Carson • Sustainment activities align along product divisions

6. Key Technical Capabilities • Product Assurance (PA) • Enterprise process that provides confidence to our customers • Used to ensure that program/contract technical requirements are fulfilled • Focuses on those supplier requirements that poses the greatest risk – Value added oversight • Early adoption of Performance Based Management • Supplier Management • Prime Contractor Control of Subcontractors • Subcontract delegations • Earned Value Management (EVM) • Ensure contractors use EVM as a management tool instead of a reporting requirement

7. PA Policy & Guidance • PA Policy & Guidance approved for Agency use • Uses a customer outcome-based approach • Outcomes are translated to DCMA performance measures • Clearly defined Agency role in different acquisition phases • System Development & Demonstration • Production & Deployment • Sustainment • Covers Systems Engineering (to include Software Acquisition Management, Manufacturing & Production), Quality Assurance, etc. • Provides an integrated approach to oversight • Utilizes all technical personnel to get deeper insight

8. Customer Outcomes • Understanding customer outcomes allows us to: • Focus on what the customer needs • Develop surveillance strategy based on customer priorities • Fine tune technical expertise in areas most important for meeting customer requirements • In the Program Managed environment: • Outcomes are determined through continuing dialog with Program Management Offices • Customer outcome Product Assurance support meetings • PA personnel are a proactive partner in this dialog by understanding the: • Contract/technical requirements/deliverables • Supplier’s performance history and capabilities

9. LRIP FRP Production and Deployment Program Managed PA • PA strategy focuses surveillance on: • Areas of moderate and high risk to customer outcomes • Supplier’s systems – System Audits • Supplier’s processes – Process Reviews • Product characteristics – Product examination • Criteria for Technical Reviews • Identify level of oversight/mix of personnel • CMMI based Risk Management (CBRM) methodology • Determine process capability • Based on continuous version of the model • Predict programmatic outcomes based on process capability, EV data, Technical Performance Measures (TPMs), Technology Readiness Levels (TRLs) System Integration System Demonstration System Development & Demonstration

10. Program Managed Contract Complete QMS (Formal Audit) During SDD, ~3 Year Intervals, or Customer Request Program Managed PA Customer Outcome PA Support Meeting Risk Assessment - Customer Outcomes - Identification & Analysis of System, - Outcome Performance Measures Process, and Product Characteristic - Risk Information Effecting the Outcomes Posing Risks to Achieving Outcomes Formulate PA Surveillance Strategy (Interdisciplinary Teaming Approach) - Systems, Processes, and Characteristics Requiring Monitoring to Ensure customer outcomes are achieved - Frequency and Intensity Dependent Upon Risk to Customer Outcome System Audit Product Examination Process Review - - Initial Reviews SDD to FRP - Inspect - Recurring Surveillance of Product - Witness - Recurring Surveillance of System Realization Planning Process and Control of - Test Elements as Risk Warrant Purchased Products - Verification ( Using Objective Evidence) - Product Realization Planning Process and - Other Processes as Risk Warrant - Analysis of Supplier Data / Information Control of Purchased Products - Focused on Process Outputs / Products - Independent Where Possible - Independent Audits - Independent Where Possible - Reverse Trail into Process Review - Reverse Trail into Systems Audit / Review - Document Surveillance Strategy Plan - Perform Surveillance and Document Results - Analyze Surveillance Data - Issue Corrective Action Requests as Necessary - Adjust Strategy based on Analysis of Surveillance Results and Changes to Customer Outcomes PDOS W / G

11. DCMA Use of CMMI • Not the “typical” CMMI user • Not seeking a benchmark “Maturity Level” • Tool to aid program surveillance • Risk identification • Prioritize activities in support of customer outcomes • Helps determine contractor performance strengths and weaknesses • Impact on the program and customer outcomes • Determine whether the supplier is maintaining processes as stated • Integrated surveillance for Engineering, Software & Quality Assurance

12. DCMA PBL Support • Contract administration • Contracting functions (other than production surveillance) are not diminished by performance-based contracts • For system-level PBL contracts, DCMA provides award fee board support by: • ensuring management controls are effective • Monitoring of performance requirements • Verification of compliance with Terms & Conditions • cost savings are being realized • Payment through WAWF facilitates rapid processing • DCMA offers WAWF training and support to contractors • Standardized SOW improves PALT and administration

13. DCMA PBL Support • Pricing / business case analysis (BCA) • DCMA supports by providing input to • Cost-benefit analysis • Price negotiation • Technical evaluation of costs • Evaluation of capital investments to increase productivity • Risk assessment at contractor facilities • DCMA evaluation of cost reasonableness requires specific requirements established by contract or MOA

14. DCMA PBL Support • Product Assurance • It is essential that PBL contractors have effective quality management systems • Enables minimal oversight • Allow for alternate release procedures • DCMA assures that contractor quality systems meet requirements • Normally, DCMA Product Assurance monitors the effectiveness of a contractor’s quality system through independent analysis of data • Quality system audits are conducted by specific request • Credible third-party certification should be relied upon unless evidence exists indicating that the system is inadequate • CM becomes a critical process as responsibility is transferred to the contractor • Increased surveillance of contractor CM processes is essential • Production surveillance is limited to verification of performance

15. DCMA PBL Support • Property administration • Essential for PBL contractors to have effective property management systems • Ideally, establish a rotable pool of spares in contractor custody • Monitoring of GFP Induction for repairs distinct from recording receipt in Government property management system • NAVICP CAV system provides data points for reduction in RTAT metrics • PHS&T • Ensure contractor does not use least cost methods for shipment of high value items

16. CMMI Implementation • More prevalent at sites that have a CMMI contract requirement • 20 + sites have been trained • Training includes: • One day of model training • One day of CMMI Based Risk Management (CBRM) method with case studies • ½ day of tool training • Maintain a core group of trained personnel to support SCAMPIs • Full voting member of DoD/NDIA CMMI Steering Group

17. Software Intensive Programs • New IPT chartered by the Acting Agency Director • Final recommendations due on July 20, 2006 • Looking very closely how the Agency conducts Program Level Analysis/Program Level Oversight • Areas of review include: • Integrated Master Plan/Integrated Master Schedule (IMP/IMS) • Basis of Estimate (BoE) • Contractor’s Latest Revised Estimate (LRE) • Independent Estimate At Completion (IEAC) • Competencies the Agency needs especially with regard to Software Cost Estimating

18. Education & Training Initiatives • Intern and Journeymen Training • ISO9001/AS9100 Quality Management Systems Auditing Training • Software Professional Development Program (SPDP) • Professional Society Certification • INCOSE Systems Engineering • ASQ Certified Quality Engineer (CQE) • ASQ Certified Software Quality Engineer (CSQE) • APICS Certified – Production & Inventory Management (CPIM) • Tuition Reimbursement • Sabbatical Programs • Specialized training from outside sources

19. Other Support Activities • Support to Program Status Reviews (PSRs) • Completed/Scheduled this year - 10 • Seven more reviews are being scheduled • Areas normally covered include: • Earned Value Management • Manufacturing & Production • Software Acquisition Management • Visits to DoD components • Understand priorities, concerns and expectations • Support to Air Force Acquisition Transformation Advisory Council (ATAC) • Probability of Program Success • Will be involved in the pilot activities

20. Conclusion • Product Assurance encompasses all technical functions in DCMA • Product Assurance focus on customer outcomes will improve the quality of service provided • Resource realignment towards higher risk requirements • Implementation of systems engineering activities continuing across the Agency • Professional, mature workforce knowledgeable and embedded in the supplier base • Timely acceptance of products and services meeting customer expectations is the key