Improving Program Performance at FAA Contractor Feed-Back

1. Improving Program Performance at FAAContractor Feed-Back

2. Agenda – Adjourn at 11:00 am –

3. Welcoming Remarks Mr. James H. WashingtonFAA VP for ATO Acquisition and Business Services • To make a major difference in the ability of FAA programs to stay on track and within budget • To achieve high EVM scores on Exhibit 300s to protect the Agency’s budget • To contribute to the goal of having FAA’s “NAS Modernization Program” removed from the GAO High Risk List FAA EVM transformation focus is to improve program performance

4. Welcoming Remarks • FAA EVM policies and practices will leverage Contractor “Lessons Learned” and “Tips and Traps” • Continuing two-way feed back required to ensure we are achieving the desired results effectively and efficiently • We aim for dialogue and invite your opinions and feedback on our EVM process improvements. FAA’s going forward goal – to partner with its suppliers in using proven Program Management and Contracting “Best Practices” to make us a recognized “Center of Excellence” in Program Management

5. FAA’s EVM Transformation Mr. Robert RovinskyOffice of the FAA’s CIO • Background: Objectives, Drivers and History to date • Recent Activities and Accomplishments • Our new Objectives • Forward Plan and Challenge • An invitation to partner with us

6. Background – History and Drivers for EVM FY 03-04 FAA Business Case Failure and Recovery: The Original Driver for this Initiative Summer 2005: Assessed our major investments (80% of FAA capital expenditures) against the industry EVM Standard Winter 2005: Committed FAA to Full EVM implementation against the Industry standard Spring 2006: Meetings with major prime and support contractors on industry best practices Summer 2006: Gained significant traction toward our compliance targets and increased use of EVM data by executives.

7. Recent EVM Transformation Activities (since 2/06) • Conducted Checkpoint Reviews/EVM assessments – well on the way to full EVM by Dec 07 • Managed program baselines by aligning the business cases, internal finance documents, and project work products and updating baselines as needed • Revised FAAs Acquisition Policy to require Standard Program Milestones and EVM metrics plan • Created an active FAA EVM Council to lead surveillance • Conducting Executive EVM awareness briefings • Supporting Federal EVM Community of Interest • Analyzed costs and benefits of EVM

8. Recent EVM Transformation Activities (since 2/06) • From our Contractors we heard: • “Most program problems are caused by poor planning, not the EVM control system” • “EVMS is Program Management Process” • EVM is tailored based on program needs and risk” • “With proper tool set, effort to maintain the system is minimized allowing more time for analysis” • “All levels of the organization must be held accountable to “walk the walk, talk the talk” ” • “Earned Value is the key element in determining program cost and schedule performance” • “Baseline reviews are key to the successful start up of a program and are required internally”

9. A New Objective Emerges • Original Objectives of the EVM Initiative • Meet commitments to the Office of Management and Budget and the General Accounting Office • Improve the Business Case for our Major Programs • Improve Program Management and Reduce Variances • Remove all FAA programs from the OMB high-risk list by the end of 2007 • As a result of the input from contractor meetings our new objective is become the leading program performance organization – a center of excellence for Program Management!

10. A New Objective Emerges • To be the leading program management organization means: • Excellence in Program Planning and Execution • Ongoing Executive Use and Involvement in Program Management tracking and decision making • Predictable (early warning) variances in cost and schedule • Few if any negative audits of our programs • FAA Recognized by stakeholders as an outstanding steward of public funds • FAA Stands with its suppliers in the use of best practices, consistent data, and state of the art tools

11. EVM Transformation Summary • The FAA has committed to OMB and GAO that we will implement full EVM – prime contracts, support contracts, and government FTE’s - by the end of FY 07 and is on schedule to do so. • The FAA sees EVM as a major tool to help meet its goals to manage its acquisitions against cost and schedule and performance targets. • EVM Transformation will be measurable and transparent – we expect to “get worse” before we get better. • EVM Transformation is a major evolution of the Acquisition Management System, in the areas of procurement, risk, cost estimation, work breakdown structure, and system engineering, and affects program planning through execution • EVM data and analysis is being built into the reporting systems used to brief key executives on the planning, progress, and evaluation of programs • FAA’s Contractors are partners with and essential to FAA’s success in implementing EVM on all its programs – we will be adopting their best practices and sharing ours with them • Existing contracts will most likely not be modified but data from contractors will be used to build EVM in at the program level. • EVM will be used for Screening Information Requests (SIRs) and New Contracts • EVM will be a discriminator in contract awards and past performance evaluations • The use – benefits and costs - of EVM will be tracked as evidence of excellent good contract management and not as an add-on cost to the government • FAA has set the goal to be a center of excellence in Program Management

12. EVM TransformationForward Plan Goals for FY 07 • All major programs assessed in FY 05 will have implemented full EVM by December 2007. • All new major programs entering development will be assessed against the ANSI standard and make progress towards green • The EVM Council will measure and enforce compliance with our EVM policies and standards via a surveillance process • Most program managers will be using EVM as an integral part of the way they manage programs by December 2007 • FAA Executives will use EVM to support their critical investment decisions including rebaselining, Air Traffic quarterly reviews, Integrated Baseline and Service Level reviews, etc. • OMB, other stakeholders will continue to recognize FAAs use of EVM and increase their confidence in our Program management. • FAA will baseline and measure its progress towards its new objective. • We will make the required continuous process improvement.

13. EVM TransformationForward Plan Activities for FY 07 • EVM Council will track our commitment to OMB and assure it is met • the FAA EVM Flag is improving, Red Guidelines assessments reduced from 43% to 27% • We will assess FAA financial processes against the ANSI standard & update our flag – we expect yellows and reds in this area. • We will continue Technical Training and expand it to all programs under active management oversight. • We will select and procure an EVM toolkit. • We have set the goal that 100% of our OMB “EVM” scores on our exhibit 300s be Green. • We will support other civilian agencies with our best practices and templates to help them implement EVM. • We will establish a Contractor EVM Working Group • Review FAA EVM policy and practices • Advise FAA on EVM Best Practices • Coordinate with other Civilian Agencies • We will participate in a PMI sponsored study on the value of PM.

14. EVM Transformation - Challenges • FAAs Financial systems may need modification. • Budgets are tight and this affects new initiatives. • This effort requires continuous process improvement and updating the FAAs Acquisition Management System. • We must reduce the burden on program managers. • We must show the impact on bottom line results. • We must meet our aggressive targets. • Awareness and Acceptance of EVM remains “lumpy”. • We must encourage our stakeholders and suppliers to participate and support us and continue to keep senior executives involved and engaged. • We need a partnership environment with our contractors

15. FAA’s Proposed EVM Processes Mr. Keith KratzertFAA EVM Focal Point • Review of EVM Best Practices from Contractors • Review of FAA EVM Governance Processes

16. Review of EVM Best Practices from Contractors = Current Practice P = Planned X = Current Practice P = Planned Practice

17. Review of EVM Best Practices from Contractors Summary of Responses: Contractor EVM Policy • All major contractors: • EVM is certified by DoD to the industry EVM Standard • EVM surveillance is conducted with or without Government participation • Contractors have a Corporate or Business Unit EVM policy

18. Review of EVM Best Practices from Contractors Summary of Responses: Using EVM on FFP • Two major contractors use EVM on FFP contracts • Use of performance-based payment (PBP) milestones on a fixed price contract “benefits both the Government and the Contractor”. • PBP provides common EVM schedule status metrics • Avoids risk of interpreting schedule differently for EVM reporting • Provides common basis to ensure both parties are working toward common goals through PBP definitions and acceptance criteria • EVM reporting examples using PBPs were provided

19. Review of EVM Best Practices from Contractors Summary of Responses: Program Start Up • Four major contractors have “Program Start-Up” processes • One major contractor has plans for Start-Up support • Use of dedicated SME resources is common • Support ranges from “assist” to “business system design” • Use of checklist is common • Inclusion of the Risk Assessment is common • EVM Guide example was provided

20. Review of EVM Best Practices from Contractors Summary of Responses: Program Performance Reporting • All major contractors use the Classis EVM “S” curves for Executive and/or PM program performance reporting • Examples include: • Below the program level, e.g. SW development • Significant events, WJHTC Acceptance, Key Site Acceptance • Included with technical status briefings, Test & Integration

21. Review of EVM Best Practices from Contractors Contractor Suggestions: • Include the entire SOW and related CLINs in the Performance Measurement Baseline • Allow the contractor to structure WBS the way programs are managed and let the contractor restructure the WBS to meet the FAA Standard WBS reporting requirement • Avoid variance reporting thresholds with “or” conditions, i.e. 10% or $25K • Encourage frequent interaction between Agency and Contractor SMEs • Common issues include: • Reconciling EV performance data with estimated cost to invoices • Reconciling EV metrics associated with activities ahead of schedule, i.e., activities with a challenge plan earlier than the contract baseline plan • Understanding EVM toolset capabilities and limitations

22. Review of EVM Best Practices from Contractors Contractor Suggestions: • Establish joint Government and Contractor working team early in the planning phase • Reduces rework, reduces the IBR effort, and results in a better project plans. • Performance Metrics: • Avoid using SV and CV as award fee metrics. • Avoid raw mathematical EVM performance metrics. • Use multiple metrics and performance data • Establish common performance metrics and report formats across the enterprise programs. • Categorize schedule and cost variances by “Cause”, i.e., Staffing, Contractor Redirection, Technical Problems/complexity, Government redirection, and GFE/GFI.

23. Review of EVM Best Practices from Contractors: Contractor Suggestions: • Use the Program risk assessment to plan the program • Balance project detail with risk • Tailor the EVM implementation and reporting • Common for major contractors tailor EVM based on Risk • Include Government participation with contractor Risk meetings • Establish enterprise business rules for program baseline change management

24. Review of EVM Best Practices from Contractors: Contractor Suggestions: • Integrated Baseline Reviews • Include Government participation with Contractor internal IBRs • Internal IBR without a Government requirement is a common practice • Consolidated Government and Contractor IBRs reduces cost • Two contractors specifically support IBRs prior to definitization • EVM Surveillance • Joint surveillance supported by all major contractors • Surveillance practices vary from a 90 day review team activity to a sampling of data • All major contractors willing to work with the FAA to develop EVM surveillance process • Use of a checklist is common

25. Review of EVM Best Practices from Contractors: Contractor Suggestions: Contracting for EVM • Define EVM surveillance requirements. • 1.13-1 Notice of Earned Value Management System (EVMS) (March 2006) and 1.13-2 Earned Value Management System (March 2006) regarding subcontract requirements related to EVMS and CO approval are redundant with subcontract statements of work • Recognize and reward early detection of risks and problems

26. Use EVM across the business Corp or Bus Unit EVM policy Regardless of contract type Sub contractors Tailored implementation Standard business systems design Industry EVM Standard compliance Early and Joint (Gov ‘t & Ctr) Planning Risk Assessment Risk Management Risk level influences EVM implementation Joint Gov’t / Ctr risk meetings Structured Program “Start Up” Dedicated SME resources Use of checklists Focus includes WBS, Schedule, performance metrics, performance budgets, and analysis Key Themes “Early Warning” is a significant and consistent EVM objective

27. FAA EVM Governance Plan • FAA EVM Governance Plan • Structured to assess the use of EVM during system Development/Modernization/Enhancement (D/M/E) for all program phases (Acquisition and O&M) • Addresses program and contractor EVM usage • Results will be measured • Average program schedule and cost baseline deviations • Number of program re-baselines • OMB Business Case R/Y/G Ratings • Average program funding request OMB approvals • Cost of EVM

28. Integrated Baseline Reviews Contactor Practices National Defense Industry Association (NDIA) IBR Guide A-11: Capital Programming Guide AMS EVM Clause EVM Certification NDIA EVM Certification Guide A-11: Capital Programming Guide AMS EVM Clause EVM Surveillance Contactor Practices NDIA Surveillance Guide A-11: Capital Programming Guide AMS EVM Clause Management Program Performance Reviews Contactor Practices OMB Exhibit 300 Reports FAA EVM Governance Plan: Sources Industry and Contractor Best Practices used to formulate FAA EVM governance approach

29. FAA EVM Governance Plan FAA EVM Governance approach is metrics-based (NOT audit and inspection) linked to existing oversight processes EVM Surveillance Acquisition Planning Program Performance Management Bus Mgmt Processes JRC Acq Prog B/L Approval Cap Investm’t Team Review Re-Baseline Request Quarterly Prog Perf Reviews OMB Exhibit 300 Budget Justification EVM Certification SIR Review IBRs

30. Forward Plan • Draft high-level process definition • Included in Back-Up • Work with our contractors to obtain Stakeholder “Buy-In” for the detailed processes and procedures • Engage NDIA and other Civilian Agencies

31. Summary • Significant positive response from our Contractors • Best Practices • Tips and Traps • EVM Governance • Metrics based • Engage Stakeholders to define processes • Contribute to the FAA becoming the center of excellence for Program Management • Business metrics will be measured

32. FAA Management Program Performance Reporting Mr. Tom KellyFAA ATO-F • Expectations • EVM will enable the FAA to deliver our Capital Acquisitions on schedule and within cost baselines • We don’t anticipate EVM in contracts to be a cost driver • EVM will be the accepted practice for all Capital programs involving HW/SW development; however consistent with contractor best practices, tailored EVM may be used for NDI/COTS

33. FAA Management Program Performance Reporting • EVM Usage • EVM is not a stick to punish. We won’t automatically cut a program’s budget based on EVM data and our owners shouldn’t either – Congress, OMB, OST • EVM is meant to provide early warning indicators of program performance. It is a management tool, to provide improved oversight. • We will continue reporting against overall (total) program cost and schedule targets, and use EVM as performance indicators on the phase –or segment.

34. FAA Management Program Performance Reporting EVM Metrics • Program EVM metrics briefed quarterly to ATO-F and ATO VP responsible for the programs • Until we get all programs to “green”, management reports on EVM data will include a qualifier, to identify data accuracy , quality (Green, Yellow, Red). • For programs with > 10% variances, reported to Executive Council and Joint Resources Council quarterly as an indicator that management review should occur • Twice a year EVM reporting included in JRC Service Level Reviews

35. Program Example – ATO-F Quarterly Review

36. FAA Management Program Performance Reporting • EVM Metrics • CPI and SPI are used as key indicators • Less than 1.0 would generate discussion

37. FAA Management Program Performance Reporting • EVM Metrics • EVM used to monitor Program Management annual goals: Last Site Government Acceptance

38. Summary • The FAA is focused on improving program performance • The FAA is making progress • We are partnering with our Contractors to: • Implement proven and efficient “Best Practices” • Obtain improved business results • Show significant progress by the end of 2007 • We need Contractors feed back to FAA program management and executives

39. Back-UP

40. FAA EVM Governance Plan: Draft IBR Process Flow Chart Post Award & Pre- Definitization IBRs

41. Determination of Existing Certification Determination of Need for Certification FAA EVM Governance Plan: Draft Contractor EVM Certification Process Flow Planning Assessment Acceptance Agency Supplier / Company Issue Advance Agreement Approved Plan Certified Pre-review and Conduct Certification Assessment Report Findings & Conclusions Contract with EVMS Cert requirement Contractor Plan for Certification Corrective Action Request Closure

42. FAA EVM Governance Plan: Draft EVM Surveillance Flow Chart Program System Contractor / In-House Agency Program Establish Surveillance Policy / Plan Action Item Closure Conducts Metrics Based Surveillance / Action Items Cross Program Surveillance Business Metrics Assessment Establish Surveillance Policy / Plan Action Item Closure