Monitoring, reporting and evaluation – usage and benefits of performance indicators Egle Rimkute

1. 4th International Public Management Summer InstituteAugust 25- 29, 2008 Tukums, Latvia“Measuring Performance in Public Administration” Monitoring, reporting and evaluation – usage and benefits of performance indicators Egle Rimkute

2. Public (service users) Government (service provider) Legislator (parliamentary scrutiny, audit) Lines of accountability Lines of scrutiny Service delivery and accountability

3. What is monitoring, evaluation and reporting? • Monitoring is the regular observation, systematic collection and storage of data. It enables activities, projects, programs, plans, and strategies to be evaluated and reported upon. • Evaluation is a systematic research conducted to assess the linkages between programme inputs, activities, outputs and outcomes or factors in the programme environment. • Reporting is the documentation of results of monitoring and evaluation and the presentation of them to appropriate audiences at specified times Purpose - support and improve resource allocation, policy decisions and accountability

4. Principles of monitoring, reporting and evaluation Key principles are that monitoring, evaluation and reporting should: • be useful for all partners • focus on few critical aspects of performance • be simple, cost-effective, affordable and practical by: - avoiding duplication of effort, - using data for multiple purposes, - ensuring that users can obtain the data, and, - ensuring that users can easily find out whether suitable data already exist. • provide comparative information • historical trend information • comparative information • look foreword as well as back • specify assumptions within strategies and decision making processes • integrate financial and non-financial information.

5. Fly exercise

6. A spectrum of performance approaches

7. Performance measurement in monitoring, evaluation and reporting • Performance measurement is an elegant way of shaping accountability and provides basis for monitoring and reporting on performance • It is important that: • information is relevant • information about performance is measured systematically and quantified • information can easily be communicated • information can be supplied at the same time each year • information is fair and understandable • PIs describe whether the service has achieved the goals

8. Usage of PIs in monitoring, evaluation and reporting The use of PIs has been justified: • Clear statement of what has been achieved • Focus on results delivery • Basis for assessing what does and does not work • Better accountability

9. What makes a good PI? To achieve effective monitoring, reporting and evaluation, PIs should be: • balanced • Volume • Quality of services • Efficiency • Financial performance • Comparable • Robust • Amendable • Reliable • Cost-effective

10. KPIs are of wide interest Users of information Level of information Public, Parliament, Media, Management Goals (outcomes) External KPIs Interested organisations Programmes (outcomes, outputs) Indicators Users Processes (outputs) Actions, resources (inputs) Supervisors, implementers, staff Operational Internal Operational indicators are of interest internally Performance indicators and users

11. Benefits of PIs in monitoring, evaluation and reporting • Easy to explain to the public, Parliament, other stakeholders • Graphs: UK – progress at a glance • Dashboards: USA – scorecard, traffic-lights • Focus management attention on major issues • Concentrate on outcomes and outputs • Clear way of assessing if services are improving • Transparency – an organisation can make clear what products/services/impacts it has delivered • Learning – PIs, when monitored and reported for, encourage to ask questions

12. Benefits of PIs in monitoring, evaluation and reporting (cont.) • Appraising: • rewarding – PIs allow to reward good performance • sanctioning – PIs enable to sanction for insufficient performance • Self-assessment – PIs enable comparison - benchmark - between similar programmes, organisations or different jurisdictions • Devolved management – accountability for results • Intrinsic motivation compared to top-down management control • Improvement – PIs pull through radical improvement

13. Problems Using Performance Indicators • Measuring outputs and outcomes • unclear goals and objectives • hard to measure concepts like well being • summing multiple objectives • distortions if only partial measures • gaming – any measure, once selected as a performance indicators will change its behaviour (“Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes”, Goodhart’s Law)

14. Problems Using Performance Indicators (cont.) • Quasi inputs/creaming • Attributing causality • exogenous factors • need to look for value added • The best are made better, the worst - worse

15. How can PIs improve the situation? • Internal and external scrutiny • Political accountability • Hierarchial accountability • Quasi-market • Self-improvement Things to remember: only performance indicators will never be enough for monitoring, reporting and evaluation