Summary of PARCA Root Cause Findings after One Year April 18th, 2011

2. Agenda Root Cause Framework Findings PARCA Actions going forward

3. Root Cause: Framework

4. Step One: Clarify What Happened Define the cost and/or schedule growth Gather documentation Program content and guidance Acquisition documents General background Program chronology Change analysis Prepare for fact finding

5. Fact Finding & Analysis Interviews Program Office and Contractors Organizations providing functional support to program Analysis of performance reporting data EVMS Cost reporting Performance metrics Program documentation, studies, reports Exact analytical methods are case dependent Blend of quantitative and qualitative (but fact based) methodologies Decision-maker concerns Actionability Exogenous vs endogenous Effectiveness

6. Products One-page memorandum for Nunn-McCurdy certification process Describes breach and its root causes Supporting document in the program certification package sent to Congress Briefing summarizing results Lessons for acquisition community Reports Analytical tools Contribution for PARCA�s annual report to Congress

7. Summary of Findings

8. Unrealistic Cost and Schedule Estimates Mainly, the source of these problems is not un-professional or incomplete cost estimates, but rather bad programmatic assumptions The criteria that PARCA uses is: �do the facts and DoD experience at the time of the estimate support the assumptions made?� The F-35 is an illustrative example; bad postulates include: That the state of design was far more mature than average; That vertical lift, with CAD/CATIA / simulation tools, would be easier; and That arbitrating multi-Service configurations requirements would be straightforward. Challenging � and even recognizing � deeply held programmatic assumptions is difficult, but must be done at the milestone.

9. Improving EVM: Better Situational Awareness Tailored metrics Relevant metrics e.g. identification AND tracking of �bets� Effective metrics e.g. avoiding integration effects Earned Value Management Compliance Effectiveness Integrated with PARCA�s program assessments Semi-annual, statutory reviews after critical Nunn-McCurdy breach As part of DAES process

10. Systems Engineering & Interface Management are a Major Challenge Complex process which demands SME input Two key questions Does Government know what it wants/able to define it ? Is schedule based upon engineering realities?

11. Effective Contracting Strategy Two elements Blocking and tackling Incentive strategy effectiveness Incentive evaluation Aligned with program goals and challenges Demanding yet achievable Sufficient to motivate No perverse effects Consistent with overall incentives Signal sent and received

12. Going Forward: Quantity-Related Sources of Variance Every quantity change is in response to something else. The criteria that PARCA uses is: �is the underlying principal animating source of the quantity change endogenous or exogenous to the operation of the acquisition system?� Changes induced from doctrinal or �pure� fiscal constraints are outside However, changes that are in recognition to, for example, escalating unit costs or schedule slips, are properly chargeable to the acquisition system Findings to date: half of the six Root Causes in which quantity was a principal factor stem from �internal� reasons, and half �external.� There will always be some debate about this, but PARCA is prepared to stick by its assessments

13. Earned Value Management (EVM) Status Reconstituted OSD EVM IPT Membership from: OSD, Services, Intel Community, DCMA, DCAA, DAU. Chartered to develop and coordinate EVM items common across DoD, and coordinate and provide service level concurrence on OSD EVM positions on policy and guidance. Created PARCA EVM Stakeholders Initiative Forum for Industry Stakeholders to provide comments and recommendations to PARCA on EVM issues which impact the wider EVM community. Current issues: CPR Data Reporting, Integrated Master Schedule, Access to EVM Data (NDA Process). Transitioning EVM Central Repository from CAPE to PARCA.

14. PARCA Evolution in 2011 Work related to 2009 WSARA Further evolution of Performance Assessment institutions and guidance implementation Bringing on-line innovative tools for the acquisition enterprise to use Feeding back Root Cause lessons into routine acquisition practice Work related to the 2011 NDAA section 2548 Prepare detailed guidance and detailed instructions for the conduct of Performance Assessments within DoD Reflect Performance Assessment activities, findings, and recommendations during the year EVM Complete migration of EVM duties to PARCA New PARCA division: �Acquisition Policy and Institutional Analysis� Purpose is to analyze the performance of DoD�s institutions and policies A re-organization of USD(AT&L) will be announced shortly that will create and resource this division

15. Backup

16. Performance Assessment in 2010 PARCA is using the DAES process to execute its performance assessment responsibilities Makes use of existing infrastructure Provides venue for PARCA analysis and observations to be included in decision-making PARCA has improved the DAES process in two ways: Improving the quality of the analysis by actively participating in all parts of the process Assessing individual programs Choosing programs that brief to the DAES meetings Ensuring that critical issues are addressed at DAES meetings Briefing PARCA analyses and insights at DAES meetings Suggesting specific changes to the DAES mechanism Responded to PUSD(AT&L) request for recommendations to ��assure that PARCA�s functions comply with the statutory provisions of 2009 WSARA.� Recommendations are now being refined and tested

17. Performance Assessment in 2010 PARCA has completed six statutorily required post-Nunn McCurdy performance assessments F-35 RMS DDG-1000 ATIRCM/CMWS Apache Block III WGS PARCA completed two statutorily required Full Rate Production assessments SSN-774 H-1 Upgrades

18. Root Cause Analyses in 2010 Performed seven root cause analyses as part of Nunn-McCurdy breach process F-35 DDG-1000 ATIRCM/CMWS Excalibur Six root cause analyses are ongoing Some �on request� Variety of programs RMS Apache Block III WGS

19. Ongoing Development Analytical methodologies Impact of funding constraints in development Evaluation of incentive strategies Systems engineering assessment EV metrics and data presentation Relationships SME sources for specific issues Knowledge-sharing of analytical approaches e.g. UK MoD Effective use of root cause analysis results Effective mechanisms for sharing lessons for benefit of acquisition community

20. Policies, Procedures, and Guidance State of play WSARA 09 requires PARCA to address Virtually no current guidance for root cause analysis Other organizations have done some root cause analysis Key questions exist e.g. is it enough to say that estimate was unrealistic (WSARA 09 category) or should we ask why? Three different purposes Documentation of PARCA processes Service conduct of root cause analyses Program Office conduct of root cause analyses

21. PARCA Root Cause Objectives Independence and even-handedness Analysis transparently based on facts Clear and accurate definition of the predominant issues Actionable Relevant trends Timely results