1 / 53

Earned Value Management, a Management Tool

Breakout Session # Michael Thompson, Senior Consultant, MCR, LLC Brian Evans, Principal Consultant, MCR, LLC. Earned Value Management, a Management Tool. 5 November 2009 2:15 PM. What is Earned Value Management and why do we use it?. The EVM Requirement.

erasto
Download Presentation

Earned Value Management, a Management Tool

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Breakout Session # Michael Thompson, Senior Consultant, MCR, LLC Brian Evans, Principal Consultant, MCR, LLC Earned Value Management, a Management Tool 5 November 2009 2:15 PM

  2. What is Earned Value Management and why do we use it?

  3. The EVM Requirement • Federal Agency Earned Value Management (EVM) Requirements, are “Guidelines” Contained in ANSI/EIA-748-B, July 9, 2007 “Earned Value Management Systems” • Contract Requirements for an EVM System (EVMS) are Contained in Federal Acquisition Regulation (FAR) Clauses that Were Adopted by DoD and Most Federal Civilian Agencies • FAR/DFAR Contract Clauses Refer to: • DI-MGMT-81466 “Contract Performance Report (CPR)” • DI-MGMT-81650 “Integrated Master Schedule (IMS)” • DoD Defines Many EVM Requirements Because EVM Started in DoD and has been Adopted by the Rest of the Federal Government

  4. Where is the “Requirement”? • It’s the Law – Federal Acquisition Streamlining Act of 1994, Title V “Acquisition Management” • The President Enacted the Law, Through… • The White House Office of Management and Budget (OMB), Through… • OMB Circular No. A-11 Which Provides Guidance on Preparing and Instructions on Executing the Federal Budget, Specifically… • Part 7 - Planning, Budgeting, Acquisition, and Management of Capital Assets (Section 300), which States that • The policy and budget justification and reporting requirements in this section apply to all agencies of the Executive Branch of the Government • An exhibit 300 must be submitted for all major investments • Using Performance-based Acquisition Management

  5. What is a “Capital Asset?” • Capital Assets are Land, Structures, Equipment, Intellectual Property, and Software Used by the Federal Government and Have Useful Life of Two Years or More • The Cost of a Capital Asset is its Full Life-Cycle Cost, Including all Direct and Indirect Costs for Planning, Procurement, Operations and Maintenance and Disposal • Capital Assets may be Acquired in Different Ways: • Purchase, construction, or manufacture • Lease-purchase or other capital lease • Operating lease for an asset with an estimated useful life of two years or more

  6. What is a “Major” Investment? • A Major Investment is: • A system or acquisition requiring special management attention because of its importance agency mission or function • Obligates more than $500,000 annually • Has significant program or policy implications • Has high executive visibility • Has high development, operating, or maintenance costs • Is funded through other than direct appropriations • Is defined as major by the agency’s capital planning and investment control process

  7. Performance based Acquisition Management

  8. What is Performance-Based Acquisition Management? • Performance-Based Acquisition Management Means: • A documented, systematic process for program management • Including integration of program scope, schedule and cost objectives • Establishing a baseline plan for accomplishment of program objectives • The use of earned value techniques for performance measurement during execution of the program • EVM is required for those parts of the investment where developmental effort is required • EVM is to be applied to both Government and contractor efforts and regardless of contract type

  9. An “Oversight” Mindset • The Issue • Federal Enterprises tend to view acquisition programs from an oversight perspective • They typically hire industry “primes” to perform the work • How it Differs from Industry • Industry tends to have a “product” or “supplier” view • They typically are the ones contracted to do the work and deliver the end-product • The Challenge • Change the mindset

  10. A “Short Term” Mindset • The Issue • Personnel within Federal Enterprises tend to have a fiscal year focus • “Crisis management” and “issue management” tend to be more prevalent than disciplined “risk management” • “Now” is more critical than “next week” vs. “next month” or “next year” • How it Differs from Industry • To some degree this is no different than industry • Clearly this depends on a variety of factors, the degree of matrixing (and thus multiple “bosses” with competing deadlines), the level of planning discipline, management needs, etc. • The Challenge • Change the mindset

  11. Work breakdown structure (wBs)

  12. Work Breakdown Structure (WBS) Level One- Product- entire product being built or delivered Level Two- Major Element (Segment) and Sub-Systems Level Three- Subordinate Components (Prime Items), individual components or assemblies subordinate to Level Two Levels 4 to X- Subproducts at lower levels until material/work effort is found Elements numbered in a logical and consistent manner

  13. Establish Baseline and Track • Baseline is set, then an Intergraded Baseline Review (IBR) is conducted • Cost, Schedule, Performance along with risks and opportunities are tracked • Variances are analyzed, they are neither good or bad, but variations that indicate areas that might need management action • Changes are managed through a configuration control process • PM may Replan as necessary within scope or Reprogram to new scope

  14. Example: WBS element 1.1.3 • 1.2.3 Represents Level 1 • 1.2.3 Represents Level 2 • 1.2.3 Represents Level 3

  15. Organizational Breakdown Structures (OBS) Level One-Performing Organization- entire organization responsible for performing the work Level Two- Major organization elements, sub contractors, or IPTs Level Three- Subordinate organizations responsible for primary items within the WBS or sub Integrated Product/Process Teams (IPTs) Levels 4 to X- Sub organizations responsible for work packages

  16. Terminology NCC- Negotiated Contract Cost Contract Price less profit/fee(s) AUW-Authorized Unpriced Work Work contractually approved, but not yet negotiated/definitized CBB-Contract Budget Base Sum of NCC and AJW OTB-Over Target Baseline Sum of CBB and recognized overrun TAB-Total Allocated Budget Sum of all budgets for work on contract=NCC, CBB, or OTB BAC-Budget AT Completion Total budget for total contract thru any given level PMB-Performance Measurement Baseline Contract time-phased budget plan MR-Management Reserve Budget witheld by KTr PM for unkowns/risk management UB-Undistributed Budget Broadly defined activities not yet distributed to CAs CA-Control Account Lowest CWBS element assigned to a single focal point to plan and control scope/schedule/budget WP-Work Package Near-term, detail-planned activities within a CA PP-Planned Package Far-term CA activities not yet defined into WPs BCWS-Budgeted Cost for Work Scheduled Value of work planned to be accomplished= Planned Value BCWP-Budgeted Cost for Work Performed Value of work accomplished = Earned Value ACWP-Actual Cost of Work Performed Cost of work accomplished =Actual Cost EAC-Estimate at Completion Estimate of total cost for total contract thru any given level; may be generated by Ktr, PMO, DCMA, etc. =EAC LRE-Latest Revised Estimate Ktr’s EAC or EACktr SLPP-Summary Level Planning Package Far Term activities not yet defined into CAs TCPI-To complete Performance Index Efficiency needed from time now to achieve an EAC

  17. Earned Value Analysis • Analysis of Past Performance • Variance Reports • Projection of Future Performance • Earned Value Review Process

  18. Performance Measurement • Value is earned (BCWP) when corresponding work is accomplished • When value will be earned is determined before beginning work using common performance measurement methods • Assignment of method should strive to reduce subjectivity

  19. Performance Measurement-EV Methods

  20. Analysis of Past Performance • CV, SV, and VAC: Most common and simplest derived earned value data • Any variance over 10% is serious and should be examined • Examine current and cumulative data points and trends • Cumulative data points good for determining average performance • Current data points good for assessing current performance and for highlighting anomalies, errors in data, and error corrections • Trend lines good for assessing performance over time-Sudden trend changes should be examined Tip: Sudden “healing” spikes in cum CV/SV typically mean a rebaseline has occurred

  21. Past Performance- Percentages • Other useful data: • - % planned= BCWS/BAC • -% completed= BCWP/BAC • % spent= ACWP/BAC • Conduct analysis such as • % planned>> % complete? Problems! • % spent>> % complete? Problem! • % of MR used>>% complete? Problem! (unless % complete close to 100%) • Master schedule shows % complete>> or << Earned Value % complete? Possible problem- an indication that Schedule and EV are not in sync

  22. Past Performance- Anomalies • Finally, look for anomalies: • BCWS= BCWP= ACWP (and <> 0) • - Indicates not really using Earned Value- May be OK if task does not adhere to use of EV • - If task 100% complete, indicates a possible rebaseline which effectively erased all variances • BCWP> 0, ACWP=0 • - Indicates work performed was free • - See more often in current month data • - Could be reflective of a delay in reporting of actuals or due to a correction in performance • BCWS, BCWP, ACWP negative • - Should never see in cumulative data • - See in current data only if the result of a corrective action

  23. Variance Reports • Variance reports (CPR Format 5 or similar report) are critical to earned value analysis process • Variance often explained at WBS level lower than training reporting level • Key elements in a variance report include: • - Where is variance (WBS, control account, labor or material, etc.) • - Cause • - Impact (CV or SV, and impact on VAC (Cost) and impact on VAC (Cost) and/ or impact on completion date (Schedule) • - Action plan (what and when)

  24. Variance Reports • Typical variance reporting thresholds: • - (+) / - 10% and /or $X • - %/$ thresholds that increase with % complete • - PM’s “Top 10” or “Critical” tasks • Focus tends to be on negative (-%/%) • Important to look at “good performers” (+%/$) • - Areas of opportunity • - Can reprogram resources to support problems and/or to accelerate schedule • Variance reports actions should be captured and tracked in risk mitigation process

  25. Future Performance- LRE • Can use Earned Value data to assess the reasonableness of the Contractor’s LRE • Contractor typically uses estimating methods outside of EVMS to estimate the LRE (e.g., bottom-up analysis of remaining work) • When analyzing LRE reasonableness, be aware that LRE also typically includes human input regarding future performance • - Good: can account for special situations • - Bad: incorporates optimism that may be unintentionally exaggerated

  26. Future Performance- EAC • Objective, mathematical Estimates at Complete (EACs) can be calculated • Most common are CPI and CPI * SPI • BAC/ CPI • Assumes even performance across the entire project equal to performance experience to date • CPI* SPI Forecast • ACWP+ BCWR/ (CPI*SPI) • Also assumed even cost performance across the entire project equal to performance experienced to date • Adjusts estimate to account for schedule performance experienced to date

  27. Future Performance- EAC • If done correctly, the EAC: • - Generates a credible story for time phased budget submissions and for providing direction to the contractor • - Helps avoid FY funding surprises • - Allows team to focus on corrective actions • If EAC calculated outside of the EVMS, can use Earned Value data to assess the reasonableness of the EAC

  28. Risk/Opportunity Indexes • Steps for Cost or Schedule related Index at CA/Summary level • Determine Range of estimate (can defined as 20% to 80% on cumulative probability distribution (CPD) Risk Range=CPD 80% -CPD 20% Range is in dollars for Cost and time (days) for Schedule The wider the range, the riskier (likely more uncertain) the program • Determine where CBB falls in the range (distribution) Assuming CBB (PMB+MR) equal or less than 80% confidence level PM decides how much opportunity to take in baseline and sets MR • Determine Risk Liability given the selected PMB and CPD Risk Liability Baseline (RLB)= CPD 80%- CBB, should decrease over time • Determine Risk Exposure Index (REI)= ratio of 1- (RLB/CBB) - REI=.75 indicates CBB only covers 75% of estimate program value at 80% confidence values -Contract Budget Base for Schedule is contractual end of contract • Determine Risk Susceptibility Index (RSI)= ratio of 1-(RLB/CBB) - RSI=.75 indicated MR can cover 75% of contract liability. Index goes to 1 at 80% confidence in Cost and/or Schedule • Similar Index can be calculated for Opportunity (other side of range) - Opportunity Range=CBB-CPD 20%

  29. Planning, Scheduling, and Budgeting • All tasks have unique names, with milestones representing key events • Work Packages average less then <90 days • Planning Packages are usually < one year and most often are tied to a Rolling Wave; the Rolling Waves are tied to significant project events • Methods selected to measure work accomplished against plan must be objective and meaningful • Schedule tasks are derived from a product-based WBS • Schedule networks are created by linking lowest level tasks only

  30. Each schedule task has at least one predecessor and at least one successor (Exceptions: first and last task or external milestones) • Use of schedule lags/leads is minimal; a lag/lead may be injected into the schedule network only for a specific purpose, not as shortcuts to identifying logical sequences of events or to override event sequencing • Use of task constraints (Must Start On, etc.) is minimal, usually only for key contractual milestones • Stakeholders validate Schedule Scope, Task Logic, and Durations • Network critical path (+some float) focus of schedule/SPI analysis • Track schedule metrics to assess goodness of schedule, include start, finish slips, tasks near critical path, float/slack, etc • Pessimistic/Most Likely/ Optimistic estimates are made for costs and time • MR is based on risks and opportunities

  31. Tech Performance Measures (TPM) and EV Techniques • Baseline and Track TPMs and relate to measurement technique • Establish tolerance bands for TPMs, track and turn into indexes • Assign Discrete Effort (DE) that produces a product using: • Weighted Start/Finish Milestones on short tasks • 0/100, 50/50 or other breakout • % Complete on longer tasks with subjective inch stones • Document what constitutes % completion • Interim Milestones internal to work packages • Value taken based on established Standards • Relate Apportioned Effort (AE) to related discrete tasks • Minimize Level of Effort (LOE), which has no tangible product • Breakup LOE by Rolling Wave and tie to project milestone

  32. Observations and suggestions

  33. Limited EVMS Legacy • The Issue • Federal Enterprises do not tend to have an earned value management system in place • How it Differs from industry • Generally speaking, significant acquisition contract requirements in the RFP will require either: • A validated EVMS implementation, or • A clearly-defined plan to establish an EVMS implementation before contract award • The Challenge • Create an earned value capable MANAGEMENT SYSTEM while executing a program

  34. “Permanent” Non-Compliance • The Issue • Federal Enterprise accounting systems are typically not standardized • How it Differs from Industry • GAAP compliance is required • SEC compliance is required (if publicly held) • DCAA compliance is required • The Challenge • Establish a “reasonable” accounting approach that meets the intent of the EVMS guidelines

  35. Risk/Opportunity Indexes • Steps for Cost or Schedule related Index at CA/Summary level • Determine Range of estimate (can defined as 20% to 80% on cumulative probability distribution (CPD) • Risk Range=CPD 80% -CPD 20% • Range is in dollars for Cost and time (days) for Schedule • The wider the range, the riskier (likely more uncertain) the program • Determine where CBB falls in the range (distribution) • Assuming CBB (PMB+MR) equal or less than 80% confidence level • PM decides how much opportunity to take in baseline and sets MR • Determine Risk Liability given the selected PMB and CPD • Risk Liability Baseline (RLB)= CPD 80%- CBB, should decrease over time

  36. Determine Risk Exposure Index (REI)= ratio of 1- (RLB/CBB) - REI=.75 indicates CBB only covers 75% of estimate program value at 80% confidence values -Contract Budget Base for Schedule is contractual end of contract • Determine Risk Susceptibility Index (RSI)= ratio of 1-(RLB/CBB) - RSI=.75 indicated MR can cover 75% of contract liability. Index goes to 1 at 80% confidence in Cost and/or Schedule • Similar Index can be calculated for Opportunity (other side of range) - Opportunity Range=CBB-CPD 20%

  37. Stove-Piped Knowledge • The Issue • To the extent EVMS knowledge exists in a Federal Agency at all, it is invariably confined to a handful of individuals (at best) • How it Differs from Industry • Clearly the state of EVM knowledge across industry also has room for improvement, but the fact that there have been long-standing EVMS requirements levied upon industry tends to ensure a broader knowledge base within industry • Industry usually has EVM Focal Points within organizations to “enforce” compliance • Defense Contract Management Agency “certifies” system compliance • The Challenge • Generate a broad sense of “ownership” of EVM that cuts across disciplines and wiring diagrams

  38. The Common Thread – Change Management • We Have Discussed Where Implementing EVM within Federal Agencies is not Necessarily the Same as Doing so in the Private Sector • We have Identified Examples of Likely Challenges • Mindset • Limited EVMS legacy • “Permanent” non-compliance characteristics • “Stove-piped” knowledge • No Matter what the Example, Implementing EVM within Federal Enterprises Requires Confronting the Prospect of Change • Change is hard no matter where it takes place • Organizational barriers to change • Individual barriers to change

  39. Leadership “Buy-in” • Leadership is About “Coping with Change” • Involves deciding what needs to be done • Creating networks of people and relationships that can accomplish an agenda • Ensure that those people actually do the job • Leadership Plays a Significant Role in Planning, Initiating and Sustaining Organizational Change • Planned change is extremely difficult, if not impossible, without “buy-in” from leadership at the top • In a Federal Enterprise, PM “buy-in” is critical for success • As we will see, we use a self-assessment concept to help accelerate process of CAM buy-in

  40. Leadership Challenges After “Buy-In” • Near Term Challenges • Evangelize the vision • Empower managers and staff to become missionaries • Far Term Challenges • Keep your eyes on the prize • Tear down barriers to change • Reshape the organizational culture

  41. Leading Change • Establish Sense of Urgency • Form Coalition to Empower Change • Create New Vision • Communicate the Vision Throughout the Organization • Empower Agents of Change • Consolidate Improvements, Reassess and Adjust • Reinforce Changes by Showing Relationship Between New Behaviors and Organizational Success Plan for, create, and reward short-term “wins” that move the organization toward the new vision

  42. Rolling Waves: Scoping the Short-Term • Use Rolling Waves to: • Set the Pace • Rolling Waves are an artificial construct to systematically explain detail • They can also be used to address the issue of complexity • Flow-down Leadership “Buy In” • PM then IPT Leads then CAMs • Further Refine and Define the Program • Slowly expand WBS tiers • Slowly expand OBS tiers • Thus, slowly refining RAM • Drive Knowledge and Understanding Down Through the Organization • Organization then Planning then Scheduling, etc.

  43. Use an Artifact-Based Focus • Avoid “the 32 guidelines” in Communications • A majority of EVM is just sound program management; why not just use program management language? • Focus First on Artifacts that: • Allow for easier implementation and evolution of a management system • 14 common artifacts (see next slide) lend themselves to being developed and maintained in automated tools • Reflect management needs rather than policy or compliance • “Performance” vs. “checking boxes” • Will act as catalysts for team development • Schedulers and analysts become CAM “buddies” • Communication across the team breeds success • Can be easily grouped into a “digestible” Rolling Wave • Don’t bite off more than you can chew for one rolling wave • Easier to hit the dart-board when you are closer to it

  44. Source: National Defense Industrial Association (NDIA) Program Management Systems Committee (PMSC) ANSI/EIA-748-A Standard for Earned Value Management Systems Intent Guide, January 2005 Edition EVM “Common” Artifacts by Guideline • Federal Agency Programs can work successfully from 14 Common Artifacts most of which can be developed and maintained in automated tools • Allows for easier implementation and evolution of a system

  45. Expand the WBS/OBS • Expand the WBS • The rolling waves further refine and define the work to be completed by expanding the WBS as necessary • As the Program defines capabilities to meet the mission need the WBS needs to be further defined • Expand the OBS • The rolling waves further refine and define the organization as CAMs are defined • As the Program matures and drives knowledge down through the organization the OBS needs to be further defined • This Expansion also Further Refines the RAM

  46. Accounting Considerations • Federal Agency Accounting Systems do not Meet GAAP Requirements • Actuals • Indirect budgets • Forward Pricing • Limited “Time and Attendance” System for Actual Cost • Usually collect “project” time, vacation time, and sick time • Federal staff use the OMB e300 cost rate for “direct” resources • Actual costs and hours are collected from Contract Managers • See next slide when we discuss “RBS” • No Indirect Budgeting for Federal Staff • “Indirect” Federal staff provide “support” but do not report “Actual hours” • “Work” on the Program but have no cost and thus no performance

  47. Accounting Considerations (cont’d) • No Forward Pricing Rates • “Forward Pricing” rates are derived from prior year actual rates • Requires the Program to perform reconciliation on a regular basis

  48. “Rolling Wave” IBR’s • Evolve the Understanding of the Baseline and IBR’s • IBR’s by EVMS area • Organize, Plan, Schedule … • CAM-level “IBR’s” • Supplier IBR’s • The “traditional” IBR • IPT level IBR’s • Program IBR • An IBR is Risk Reduction Methodology, not a Meeting…a Communication Forum • You can Hold an IBR on a $1B Baseline in an Afternoon…if you “Prepare” for it

  49. Combining IBR Prep and Surveillance • Why Perform an IBR? • “IBRs are intended to provide a mutual understanding of risks inherent in contractors' performance plans and underlying management control systems. Properly executed, IBRs are an essential element of a PM's risk management approach.”SOURCE: The Program Managers’ Guide to the Integrated Baseline Review Process, April 2003 • Why Perform Surveillance? • “(F)ocus on using EVMS effectively to manage cost, schedule, and technical performance. An effective surveillance process ensures that the key elements of the process are maintained over time and on subsequent applications.” SOURCE: NDIA PMSC EVMS Surveillance Guide, Oct 2004 • Why Combine Them? • Federal Enterprises are typically building baseline and system at same time • Helps measure as you go • “Self” assessment based surveillance accelerates CAM ownership and “buy-in” • Benefit: “Self” assessment allows you “identify” strong and weak CAMs and allows staff to “rate” themselves instead of external elements

More Related