1 / 45

Strengthening Performance Management Systems: Management & Audit Perspectives AGA Audio Conference, Oct. 3, 2012

Strengthening Performance Management Systems: Management & Audit Perspectives AGA Audio Conference, Oct. 3, 2012. Auditor Roles in Performance Measurement Project : Paul Epstein, Stuart Grifel, Principals King County, Washington :

gizi
Download Presentation

Strengthening Performance Management Systems: Management & Audit Perspectives AGA Audio Conference, Oct. 3, 2012

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Strengthening Performance Management Systems: Management & Audit PerspectivesAGA Audio Conference, Oct. 3, 2012 Auditor Roles in Performance Measurement Project: Paul Epstein, Stuart Grifel, Principals King County, Washington: Michael Jacobson Acting Deputy Director, Performance & Strategy, Office of Performance, Strategy, & Budget Ron Perry, Deputy Auditor www.AuditorRoles.org 1

  2. 2

  3. Two Audit Roles A New Service Model: Five Roles Auditors Play in Government Performance Measurement Role 2 Role 1 Role 3 Role 4 Three Assistance Roles 3 www.AuditorRoles.org

  4. Performance Management Systems Roles Management Runs the System Auditor Assures Accountability • Design & build the system • Determine how to measure performance • Develop strategic & operating plans & budgets with performance targets • Implement plans, deliver services, manage operations • Collect, report, & analyze performance data • Make decisions to improve • Can advise & cooperate on system development & improvement • Audit the relevance & reliability of performance information • Audit performance • Audit performance management systems www.AuditorRoles.org

  5. Most Relevant Onsite Courses Available thru AGA For Management For Auditors • Assessing and Strengthening Performance Management • Assessing Performance Measures and Data • Auditing Performance Management Systems • Auditing the Relevance & Reliability of Performance Information Contact the AGA Education staff at 800.AGA.7211, ext 307, or onsitecourses@agacgfm.org Course Descriptions: www.agacgfm.org/Events---CPE/Onsite-Courses.aspx www.AuditorRoles.org

  6. Performance Management & Performance Management Systems Performance Management: Using performance information to help make better decisions. Performance Management System: Use of performance information on a regular basis as part of a continually repeated cycle of performance monitoring, analysis, and improvement, in which measured results are fed back into decision making to improve future performance. Comprehensive PM Systems: Usually involve planning (strategic & operational), budgeting, service & project implementation, and monitoring & reporting. 6 www.AuditorRoles.org www.AuditorRoles.org

  7. Models to Aid Developing & Improving Performance Management SystemsMany “Managing for Results” Models Are Available to Choose from to Design or Audit a PM System Austin City Auditors used the Government Performance Accountability Systemas criteria for citywide audits See example at www.AuditorRoles.org Plan Strategic & Annual Planning Act Performance- based Decision Making Do Performance Budgeting & Service Delivery Check Performance Monitoring & Reporting 7 www.AuditorRoles.org www.AuditorRoles.org

  8. Resource Allocation King County’s Performance Management System Strategic Planning Business Planning Implementation Performance Evaluation

  9. Three phases of development • Starting out: What do you want to know? • Creating the building blocks • Developing the system: refinement & evolution

  10. Building Blocks • Started by creating Executive review sessions (“Stat” model, West Coast version) • Spent more than a year in design • Departmental “dashboards” • Executive Initiatives • Countywide statistics (HR, risk, etc.)

  11. Building Blocks • Had to agree on what specifically to measure  developed logic models for each department • Benefit: Executive attention & moved the focus from how much work we do (outputs) to what we are trying to achieve (outcomes)

  12. More Building Blocks • Created a PERFORMANCE REPORT - AIMs High: Annual Indicators and Measures • Worked to better integrate budget requests with performance through annual business plan instructions and review • TRAINING in Performance Measurement & Performance Management • Passed LEGISLATION requiring countywide strategic plan, ongoing performance management, and public reporting • Benefit: Increased visibility of measurement activities, internally and externally

  13. Management-Auditor Cooperation in Developing County Performance Management • Council created a Performance Management Work Group, headed by Auditor • Part of county code • Non-traditional role for auditor: An advisory role that maintains auditor independence • Work Group promotes performance management (PM) by • Advising council on PM • Providing forum for discussing best practices and implementation issues

  14. Audit Role – Performance Context • Performance Management • Performance Measurement • Performance Auditing

  15. The Performance Planning Sub-system: Strategic and Annual or Biennial Planning Plan Strategic & Annual Planning Act Performance- based Decision Making Do Performance Budgeting & Service Delivery Check Performance Monitoring & Reporting 15 www.AuditorRoles.org

  16. Value of the Performance Planning Sub-system • Drives entire performance management system • Produces strategic & performance plans • Creates initial expectations for accountability of agencies and programs • Precedes but aligns with performance budgeting • Strategic plan provides the opportunity for cross-agency goal setting 16 www.AuditorRoles.org

  17. King County Strategic Plan • Countywide (cross branch) in scope • Required fundamentally new processes and new information: • Employee survey • Resident survey w/ benchmarks • Public outreach • City engagement • Leadership engagement 17

  18. The Performance Budgeting Sub-system Plan Strategic & Annual Planning Act Performance- based Decision Making Do Performance Budgeting & Service Delivery Check Performance Monitoring & Reporting 19 www.AuditorRoles.org

  19. Value of the Performance Budgeting Sub-system • Funds accomplishment of performance expectations including goals, objectives, and targets • Establishes a foundation for accountability to stakeholders for cost & performance • Encourages the use of performance accomplishments as an important factor in funding decisions 20 www.AuditorRoles.org

  20. Target for Increased Funding Strategic Success in Achieving Community or Program Outcomes Target for Funding Cuts Useful Contributor to Government Success Strategic Performance Budget Decision Model HIGH Strategic Importance LOW POOR Performance Results GOOD Use of Strategic Goals and Performance Results in Prince William County Budget Decisions www.AuditorRoles.org

  21. Target for Increased Funding Strategic Success in Achieving Community or Program Outcomes Target for Funding Cuts Useful Contributor to Government Success Strategic Performance Budget Decision Model:In Reality: Varies with Fiscal Environment HIGH Less funding available Strategic Importance More funding available LOW POOR Performance Results GOOD Use of Strategic Goals and Performance Results in Prince William County Budget Decisions www.AuditorRoles.org

  22. Performance Monitoring, Analysis, & Reporting Sub-system Plan Strategic & Annual Planning Act Performance- based Decision Making Do Performance Budgeting & Service Delivery Check Performance Monitoring & Reporting 23 www.AuditorRoles.org

  23. Value of the Performance Monitoring, Analysis, & Reporting Sub-system • Determines whether performance expectations were achieved • Provides the basis for analyzing why performance expectations were or were not achieved • Supports performance based decision making and accountability to stakeholders 24 www.AuditorRoles.org

  24. Performance Reporting • King County Performance Report - AIMs High: Annual Indicators and Measures • Hard copy with budget, then online for the public • Reported on both community indicators and agency performance measures • Scorecard with 20 measures www.kingcounty.gov/aimshigh 25

  25. Assessing the Relevance of Performance Measures: Sample Criteria Measures should be: Aligned Linked to mission, goals, objectives Complete Includes essential aspects of performance UsefulTimely Understandable Comparable Responsive to change Meets broad needs of users 26 www.AuditorRoles.org

  26. Assessing the Reliability of Performance Measures & Data: Sample Criteria Each Measure and its data should be: AccurateComputed correctly Neither overstated nor understated Appropriately precise ValidCorresponds to the phenomena reported Correctly defined Data & calculation comply with definition Unbiased ConsistentConsistent with previous periods Controlled by adequate systems 27 www.AuditorRoles.org

  27. Assessing Performance Data Reliability Self-Assessment by Mgt Certification Audit 15 step process including steps specific to any audit Results placed in one of four assessment categories Performed by auditors Results reported externally Download “Certification Audit Program” Take “Auditing the Relevance & Reliability of Performance Information” offered by AGA • 10 step process conducted by management • Conducted by department management • Departments can replicate process on an ongoing basis • Download “Department Measures Self-Assessment Steps” • Take the course “Assessing Performance Measures & Data” offered by AGA Find these steps & many other tools for assessing reliability at: AuditorRoles.orgclick on “Auditor Toolkit/Tools for Role 2” www.AuditorRoles.org

  28. Performance Auditing Auditing efficiency, effectiveness, and compliance • What are the goals and measures? • Are the measures consistent with goals? • How to track, report, and use the results? • What does the measurement indicate? • What if measures lack?

  29. Incorporating PM into Audit Process • Review existing plan(s), performance management structure, and performance data • Evaluate for alignment, validity, usefulness, and results • Develop measures and benchmarks as audit criteria if none exist.

  30. Imagine the Possibilities! • No measures exist; no PM structure • Measures exist but not used to manage performance • Measures exist but not aligned with strategic plan • Measures exist but don’t measure what they should • Measures exist but aren’t structured to improve performance • Performance measurement system works effectively

  31. Example: Transit Procurement • Industry experts recommend procurement performance management, but, the transit industry as a whole has not yet embraced it. • King County Transit’s measurement of the success of the bus procurement program is limited. • Recommendation: Transit should implement a bus procurement annual retrospective process that includes: • Calculating performance measures • Developing action plans to improve procurement outcomes • Updating performance measures and targets to increase effectiveness

  32. Transit Procurement Recommended Bus Procurement Performance Measures

  33. Performance Decision Making Subsystem Plan Strategic & Annual Planning Act Performance- based Decision Making Do Performance Budgeting & Service Delivery Check Performance Monitoring & Reporting 35 www.AuditorRoles.org

  34. Value of the performance based decision making sub-system • Achieve high performance through the use of performance data and analyses • Develop and implement improvement strategies if performance reports indicate expectations are not being met • Revise current or future performance plans and budgets based on performance data and analyses 36 www.AuditorRoles.org

  35. GPRAModernization Act of 2010 (HR 2142) & Related Guidance (e.g., from OMB, OPM) • Expanded GPRA from strategic planning & reporting to requirements for actual performance management, e.g.: • Quarterly priority progress reviews and use of performance information • Agency COOs, Performance Improvement Officers (PIOs), Performance Improvement Council • Performance management skills & competencies to be specified by OPM • Review as “models” for potential criteria: • GPRA Modernization Act provisions • Current related guidance (e.g., from OMB, OPM) www.AuditorRoles.org 37

  36. Three phases of development • Starting out: What do you want to know? • Creating the building blocks • Developing the system: refinement & evolution

  37. Refinement & Evolution • New administration & new ideas • Building on what had come before • Added private sector orientation/language • Products • Customers • Lean/Continuous Improvement • Value Streams

  38. Refinement & Evolution • Created a “FrontRunners” training series • Large events for managers around Lean, Ken Miller’s work on customers and “pipes,” Kaizen

  39. What Next in King County? • New, integrated systems for human capital management, payroll, finance and budget • Performance management component not yet developed and will take more time • Connection between performance and budget is “TBD” as the executive develops new models • In lieu of integration, development of lines of business plans, goal level planning, “traditional” budgeting.

  40. Observations & Learning • Leadership drives direction (or not) • The broader, the better • Movement is possible without top leadership but the extent and degree will be impacted • Design, but only up to a point; then go! • A consistent framework is helpful, but not essential to start

  41. Observations & Learning • Need to be nimble/adaptable and make changes based on your experience • Use should inform design/approach (not just theory) • Plan-Do-Check-Act is different than constant redesign • It is a marathon, not a sprint

  42. Performance Management as Triple Loop Learning Revise Strategic Plan, Budget, Performance Plan (goals, objectives, standards, measures, targets) Plan Larger-scale Improvements Build Performance Management Capability Check: Measure & Assess Results Redesign or Change Policies, Practices, Programs Act to Improve Process Plan Steps to Use Process Do Steps to Use Process Multiple Cycles for: Organizational & State or Community Learning vs. goals, objectives, standards, & targets, where applicable 44

  43. Performance Management Systems Resource Allocation Strategic Planning Business Planning Plan Strategic & Annual Planning Implement-ation Act Performance- based Decision Making Do Performance Budgeting & Service Delivery Performance Evaluation Check Performance Monitoring & Reporting 45 www.AuditorRoles.org

  44. Questions? Related Onsite Courses Available thru AGA For Management For Auditors • Assessing and Strengthening Performance Management • Assessing Performance Measures and Data • Auditing Performance Management Systems • Auditing the Relevance & Reliability of Performance Information • A New Service Model: Auditor Roles in Government Performance Measurement • Auditing or Assessing the Quality of Performance Reports Contact the AGA Education staff at 800.AGA.7211, ext 307, or onsitecourses@agacgfm.org Course Descriptions: www.agacgfm.org/Events---CPE/Onsite-Courses.aspx www.AuditorRoles.org

More Related