Distinguishing Inputs, Outputs, Outcomes and Impacts
1 / 89

Important points to start with - PowerPoint PPT Presentation

  • Updated On :

Distinguishing Inputs, Outputs, Outcomes and Impacts. www.summerinstitute.eu. Important points to start with. Decision making. Policy analysis. Limited rationality. Incentives. Process and learning. Contents. Outputs & inputs. Outcomes. Indicators. Logic model. Budgeting & PI.

Related searches for Important points to start with

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Important points to start with' - Michelle

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Important points to start with l.jpg
Important points to start with

  • Decision making

  • Policy analysis

  • Limited rationality

  • Incentives

  • Process and learning

Contents l.jpg

  • Outputs & inputs

  • Outcomes

  • Indicators

  • Logic model

  • Budgeting & PI

  • Future

Contents4 l.jpg

  • Outputs & inputs

  • Outcomes

  • Indicators

  • Logic model

  • Budgeting & PI

  • Future

Outputs definition l.jpg
Outputs - definition

  • Services or goods which are provided by ministries and other state organizations to external stakeholders / beneficiaries (directly or through other organizations)

  • External stakeholders include: general public, citizens, businesses, NGOs, media and other state bodies including the minister, other ministries and state organizations, and the parliament

Examples l.jpg

  • Provision of prosthetic devices to disabled

  • Provision of schooling

  • Health centre services

  • Analysis and policy advice

  • Laws

  • Inspections

  • Training

Outputs should not be l.jpg
Outputs should not be

  • Activities

  • Processes

  • Inter-mediate output

  • Capacity building initiatives

Example output schooling l.jpg
Example: output – schooling

  • Number of graduates from school

  • Activity - teaching

  • Process - recruitment of teachers

  • IMO – Number of enrolled chidlren

  • Capacity building – increasing professional qualification of teachers

Slide9 l.jpg

Consumption or provision?

  • Economic theory view: the delivery of service takes place between producer and consumer, and therefore transaction is complete, when service is used (output consumed)

  • Theatre plays

  • Patients checked

  • Public administration view: for many outputs it does not mater whether they are consumed or not – the service should be there in case it is needed

  • Emergency services

  • Prison services

Output characteristics l.jpg
Output characteristics

  • Volume or quantity of provision

  • Timeliness in provision

  • Quality of provision

  • Satisfaction - surveys and evaluation

  • Comparison – to others or standard

  • Coverage, meting of demand and accessibility

  • Equity

Example output provision of prosthetic devices l.jpg
Example: output – provision of prosthetic devices

  • Volume – number of disabled people served

  • Timeliness – service provided in specific time – for example, one week after request

  • Quality – satisfaction of users of devices

  • Coverage – % of people in need of devices served

  • Equity – people with access difficulties served at home or transported to the service point

Selectivity l.jpg

  • Not all output characteristics can/should be used for all outputs – be selective

  • Focus on key outputs for external users

  • For all other outputs – have at least the volume

Aspects to consider l.jpg
Aspects to consider

  • Control of provision

  • Other government interventions

  • Measurability

  • Attribution

  • Users

  • Incentives & gaming

Control of provision l.jpg
Control of provision

  • Provision of a single output is controlled (directly or indirectly) by government organization

  • But not its effectiveness (achieving desired policy objective). Many government outputs make sense only when used together with other outputs

  • Road safety and reduction of fatal accidents depends ona group of outputs – quality of road, speed control, technical state of cars, use of seat belts.

Control of provision contd l.jpg
Control of provision (contd.)

Many organizations




One or more organizations

One organization

Other government interventions l.jpg
Other government interventions

  • To implement policies governments provide not only goods and services, but also:

  • regulation

  • capital investments

  • social and various other benefits & compensations

  • subsidies

  • If these are not outputs, how do they relate to them?

Other government interventions contd l.jpg
Other government interventions(contd.)

  • It can be helpful to think of three broad categories

  • goods and services = outputs

  • administered items

  • capital investment

  • mandatory benefits & compensations

  • subsidies and other payments

  • general regulation

  • But remember – most of administred items and general regulation require outputs

Slide18 l.jpg

Example – road safety

  • In budgets, some countries separate outputs / investment / financing in separate programs while others do not (but separate using economic classification)

Measurability l.jpg

  • Some outputs are harder to measure than others due to the nature of organizations providing them

Measurement difficulty



Residential care



Road police

Job Counselling

Training institutions

Unemployment agencies

Policy making ministries



Attribution l.jpg

  • Outputs mean something if they can be attributed to some objective (outcome) – they contribute to achieving of objective, e.g. Change in outcome

  • Generalized attribution: re-training of unemployed contributing to reduction of unemployment

  • Specific: re-training of unemployed contributing to employability of the trained person during the next year after training

Slide21 l.jpg

Difficulties in establishing attribution

Clear attribution:

  • Fixed arm

  • Medical service

  • Broken arm

Less clear attribution:

  • Survival

  • Services

  • Cancer

  • Earlier diagnosis?

  • Better treatment?

  • Healthier life style?

  • Beneficial effects of affluence?

Users l.jpg

  • Different PI users and decision making stages might have different information needs (and therefore require different formats of presentation of PI)



Ministry of Finance


Permanent Secretary

Unit Manager


Output focus

  • Using of PI means analysis and learning rather than money and / or promotion

Principal agent problem l.jpg
Principal – Agent Problem

  • Principals are “owners” and “bosses”, but they lack time and expertise to do everything

  • Therefore principals hire agents, who have time and expertise, but are self interested












Delegated authority

Tax collection

Profit making

Service delivery

Principal agent problem contd l.jpg
Principal – Agent Problem(contd.)

  • Agents have one strong advantage over principals - information

  • This creates problem called “gaming”, which is based on two factors:

  • There is information asymmetry between the principal and agent; and

  • There are wrong incentives for disclosing or hiding information

Gaming incentives l.jpg
Gaming & incentives

  • Bevan and Hood: "reactive subversion such as 'hitting the target and missing the point' and / or reducing performance where targets do not apply"

  • Gaming = manipulation with information that might affect service delivery

  • Incentives = factors (financial or non-financial) that provide motives for a particular course of action, or count as a reason for preferring one choice to the alternatives

  • Targets are the most common causes for perverse incentives

Slide26 l.jpg

Gaming & incentives



Gaming potential

Information asymmetry




Relationship with decision-making

Impact of gaming l.jpg
Impact of gaming

Strategy A

Strategy B

Strategy C

Strategy D

Arrival on time at final destination

“Saboteur” manipulation of service and data

“Smart Liar” or manipulation of data

“False Champion” or service manipulation

“Honest Fool” or no manipulation


Impact of gaming contd l.jpg
Impact of gaming (contd.)

  • Gaming can result in:

  • Cherry picking – handling of easy cases, patients etc. at the expense of difficult ones

  • Ratchet effect – slowing down today to be on time tomorrow

  • Distortions – improving areas where measurement is dome while neglecting other areas

  • Threshold effect – focusing on 2nd best and forgetting about the best and worst, as best will be achieved anyway while worst require too much of an effort

Dealing with gaming l.jpg
Dealing with gaming

  • Learning and data improvements

  • Avoiding strong negative incentives

  • Clarify use of PI in advance

  • “Loose” rather than “tight” connection to decisio making

  • Provide feedback on data relevance and quality

  • Reward those with good PI

  • Do not overload with data – use the key measures

Dealing with gaming contd l.jpg
Dealing with gaming(contd.)

  • Dealing with principal – agent problem at all levels

  • Use of “auditors”

  • National Audit Office

  • Internal audit

  • Encourage ownership at production level

Summarizing about outputs l.jpg
Summarizing about outputs

  • Goods and services

  • Basic element in performance information system

  • Can be measured in different ways, but there are difficulties

  • Thee are three most common difficulties:

  • Measurability

  • Attribution to objectives (outcomes)

  • Scope for manipulation

  • Loose link between output data and decision making generally is a better incentive to avoid gaming

Inputs definition l.jpg
Inputs - definition

  • Inputs are resources used in the delivery of goods and services

  • Labour and other means of service provision (facilities, computers, teachers, books etc.)

  • Money is not an input. It s a cost of input

Inputs output combination l.jpg
Inputs – output combination

More outputs



Less inputs

More inputs



Less outputs

Improving performance l.jpg
Improving performance

More outputs

Production possibility frontier

Less inputs

More inputs

Less outputs

Contents35 l.jpg

  • Outputs & inputs

  • Outcomes

  • Indicators

  • Logic model

  • Budgeting & PI

  • Future

Outcomes definition l.jpg
Outcomes - definition

  • The effects which government is trying to achieve for the public. They are often expressed as improvement in the living conditions of people or favourable changes that contribute to these changes

  • Outcomes are changes in the economic, physical, social and cultural environments which the state agency(ies) is trying to influence through provision of goods and services, general regulation and financing

  • Outcomes are those events, occurrences, or conditions that are the intended or unintended results of government actions (OECD)

Examples37 l.jpg

  • reduction in maternal / child mortality

  • increase in export

  • decrease of unemployment

  • reduced crime levels

  • improved literacy rate of the population

  • reduction in fatalities from road accidents

  • land productivity improvement

Examples contd l.jpg
Examples (contd.)

Examples contd39 l.jpg
Examples (contd.)

Examples contd40 l.jpg
Examples (contd.)

State measures and change l.jpg
State Measures and Change




change %

Number of road accidents




Number of injured




Number of fatal injuries




Number of registered cars




State measures and change contd l.jpg
State Measures and Change(contd.)

  • the same data can be called state measure or performance measure

  • it depends whether you attribute policy (outputs, regulation and finances) to it or not

  • if you do attribute – you are interested in change influenced by your policy (the question – whether different policy generates different result, e.g. marginal changes in result depending on inputs)

  • + - 0

Slide43 l.jpg

State Measures and Change(contd.)




change %

Number of road accidents




Number of injured




Number of fatal injuries




Number of registered cars




  • outputs

attribution of policy to change

  • financing

  • regulation

Different outcomes results impacts l.jpg
Different outcomes – results & impacts

Impact or longer term effects (FO)

Global objectives

Results or direct & immediate effects (SO)

Specific objectives

Operational objectives




Results impacts defined l.jpg
Results & impacts defined

  • Results relate to the direct and immediate effect brought about by a policy or program. They provide information on changes to, for example, the behaviour, capacity or performance of direct beneficiaries. Such indicators can be of a physical (reduction in journey times, number of successful trainees, number of roads accidents, etc.) or financial (leverage of private sector resources, decrease in transportation cost) nature

  • Impactsrefer to the consequences of theprogram beyond the immediateeffects on its direct beneficiaries. Two concepts of impact can be defined. Specificimpacts are those effects occurring after a certain lapse of time but which are, nonetheless, directly linked to the action taken. Global impacts are longer-term effectsaffecting a wider population

Slide46 l.jpg


Tax collection







Tax audits

Regulatory burden studies

Prisoner rehabilitation


Reduced tax evasion

Better regulation

Re-integration into community

results or SO

Reduced shadow economy

Enhanced competitiveness

Reduced repeat crimes

impacts or O

Results impacts defined contd l.jpg
Results & impacts defined(contd.)

  • Results = short to medium term outcomes = specific outcomes

  • Impacts = medium to long term or final outcomes = overall outcomes

  • Sometimes immediate, intermediate and final outcomes

Single outcomes and indexes l.jpg
Single outcomes and indexes

  • Sometimes indexes are used to describe and compare performance

  • Outcomes from several areas and sometimes even sub-areas are merged into one index

  • These are of particular use for policy planners

  • For example: the Enabling Trade Index

Example the enabling trade index 2007 8 l.jpg
Example: The Enabling Trade Index 2007/8





  • Regulatory environment

  • Availability & quality of transport infrastructure

  • Efficiency of customs administration

  • Tariff & non-tariff barriers

  • Proclivity or openness to trade

  • Physical security

  • Availability & use of ICTs

  • Efficiency of import – export procedures

  • Transparency of boarder administration

Example tourism competitiveness index 2008 l.jpg
Example: Tourism Competitiveness Index 2008




  • Policy & regulations

  • Air transport infrastructure

  • Human resources

  • National tourism perception

  • Environmental regulations

  • Ground transport infrastructure

  • Natural & cultural resources

  • Safety & security

  • Tourism infrastructure

  • Health and hygiene

  • Price competitiveness

  • Prioritization of T&T

Problems with outcome measurement l.jpg
Problems with outcome measurement

  • Timing

  • Data availability for measurement and transaction costs

  • Outcome (state) measures remain important even in the absence of policies and programs directed at them

  • Impact of other influences

  • Attribution

Attribution to outputs l.jpg
Attribution to outputs

Other influences

Influence of other factors

Degree of outcome attribution to outputs


Immediate – intermediate - final

Attribution modelling l.jpg
Attribution modelling






Knowledge & understanding

Changes in environment

Behavioural changes

Attribution modelling cont l.jpg
Attribution modelling(cont.)




Target group


Medium term

Short term

Long term

Improved knowledge about impact of farming on environment

Educational events

Less polluting farming

Systematic pollution monitoring in farms






Improved understanding about the use of pollutants

Control system

Polluting farmers

Changes to farming practices

Improved understanding about the ways to control pollution

Site visits / inspections

Improved environment

Summarizing about outcomes l.jpg
Summarizing about outcomes

  • Outcomes are changes in the economic, physical, social and cultural environments which the state agency(ies) is trying to influence

  • Focus on change (+/-/0) in outcomes is what makes attribution between outputs and outcomes)

  • Outcomes can be of immediate, medium term and long term nature. More immediate they are – stronger the attribution to outputs

  • Outcomes can range from single to more compplex ones based on some kind of a model

  • Outcomes are particularly useful for policy planning and evaluation

Contents58 l.jpg

  • Outputs & inputs

  • Outcomes

  • Indicators

  • Logic model

  • Budgeting & PI

  • Future

Indicators definition l.jpg
Indicators - definition

  • An indicator can be defined as the measurement of an objective to be met, a resourcemobilized, an effect obtained, a gauge of quality or a context variable

  • Indicators demonstrate how to measure elements of program that you are interested in or program in general

Important aspects of use of indicators l.jpg
Important aspects of use of indicators

  • Output indicators

  • State indicators and baselines

  • Targets

  • Single and ratio indicators

Output indicators l.jpg
Output indicators

  • Volume or quantity of provision is the most often used and most relevant to budgeting. However, other output dimensions can have indicators, too:

  • Timeliness in provision

  • Quality of provision: satisfaction and/or comparison

  • Coverage, meting of demand and accessibility

  • Equity

Example technical certification of cars service l.jpg
Example: technical certification of cars service

  • Volume: X number of cars per years

  • Coverage & demand: 100% every day

  • Timeliness in provision: less than ½ hour waiting, service within 1 hour

  • Quality: satisfaction of car users; quality in weakest areas improved; online registration etc.

  • Equity: special service for disabled

State indicators and baselines l.jpg
State indicators and baselines

  • State indicators describe the state of affairs in a given area of policy concern – unemployment, GDP, mortality, lack of labour mobility, demand for a service exceeding supply etc.

  • Baseline indicators refer to the initial value against which inputs, outputs or outcomes are subsequently measured.

  • State indicator = photograph of situation; Baseline indicator = starting point of measuring impact of policy

Targets l.jpg

  • Show desired level of performance

  • Make sense only if time bound and have baseline

  • Customer satisfaction from a service increased from 25 to 75% by June 2009

  • Every input, outputs and outcome can have indicators, but should it?

  • UK PSAs: from 300+ targets in 1998 to 30 targets in 2007

  • Often, if you want to have loose connectionto accountability and control, simple expected performance projections can work better than targets

Static and dynamic targets l.jpg
Static and dynamic targets












Single and ratio indicators cont l.jpg
Single and ratio indicators (cont.)











Summarizing about indicators l.jpg
Summarizing about indicators

  • Indicators are measures of change

  • Baseline and target data is important for good use of indicators

  • But be careful – not too many targets. When targets are set, make sure gaming potential is minimized

Contents71 l.jpg

  • Outputs & inputs

  • Outcomes

  • Indicators

  • Logic model

  • Budgeting & PI

  • Future

Model for structuring performance information l.jpg
Model for structuring performance information

Other influences

Problems and needs

Policy level



Structure, institutional and managerial arrangements

Program level

Activities and processes


Short to medium term outcomes

Final outcomes




Challenges l.jpg

Value to the nation

Measurement challenges

Measurement costs

Time delays

Relative magnitude

Linkages to our actions







Slide74 l.jpg

Bottom-up vs. top-down approach in modelling and planning

  • Outputs to outcomes (bottom-up) approach is most common to budgeting







  • While outcomes to outputs (top-down) – in policy

Policy objective



  • However, several OECD countries use the later in budgeting, focusing on outcomes and leaving connection to outputs loose

Contents75 l.jpg

  • Outputs & inputs

  • Outcomes

  • Indicators

  • Logic model

  • Budgeting & PI

  • Future

Slide76 l.jpg

Budgeting and performance information

  • Performance budgeting has been THE THEME for several decades now

  • It has not succeeded and no one does it right

  • There are two most discussed topics in this theme:

  • Outputs or outcomes focus?

  • Can everyone do it?

Slide77 l.jpg


  • Informational challenges, e.g. inserting output and outcome information into budget documentation

  • Decision making challenges, e.g. how much difference PI makes

Slide78 l.jpg

Informational challenges

  • Information overload

  • Difficulties to delegate

  • Cost of data

  • Information and knowledge gaps

  • Principal agent problems

Slide79 l.jpg

Decisional challenges

  • Better information = better decisions – only assumption

  • Hard to attribute action to outcome change. Requires high quality policy analysis

  • Fixed budgets are not performance budgets

  • But non-fixed budgets can lead to gaming

  • Information asymmetry and principal agent problems

  • Time delays

Slide80 l.jpg

Outputs or outcomes?




Program budgeting

Strategic planning

Evaluation and spending reviews


Need for strong policy analysis

Slide81 l.jpg

Outputs or outcomes? (cont.)

  • Performance budgeting is about using the budget to promote performance rather than a specific technique such as program or output budgeting. What all attempts at performance budgeting have in common is the attempt to increase the influence of policy analysis, strategic decision support, performance information in the budget system (Graham Scott)

  • Appropriate use of performance information by key actors at each stage of the budget cycle to inform their decisions concerning resource allocation and to improve efficiency of resources (World Bank)

Slide82 l.jpg

Can everyone do it?

  • Get the basic parts of the budget to perform first

  • Willingness and ability to execute voted money, to control expenditure

  • Sound financial accountability, respect for financial management procedures, and effective internal controls

  • Functioning government

  • Implanting of performance information in the budget process requires conductive environment – values, minimal degree of informality, less corruption, leadership etc.

Slide83 l.jpg

Where to start?

How much money you can spend or should save for the whole of Government?

Macroeconomic policy as in Budget Memorandum

1. Aggregate Fiscal Discipline

2. Allocative Efficiency

In what areas we want / need to spend the money on – health, education, transport?

Strategic policy documents of the Government

2.1 Inter-sectoral

In each area, what are the needs and priorities we want to spend money on – preventive health care, treatment etc.?

Sector level policy documents

2.2 Intra-sectoral

How well we spend the money we got? Could we spend it better?

Monitoring and analysis

3. Operational Efficiency

Slide84 l.jpg

Where to start? (cont.)

Aggregate fiscal discipline

However, any failure to successfully achieve macro fiscal goals will seriously undermine allocative and operational efficiency goals

Allocative efficiency

Operational efficiency

Importance of PI

Slide85 l.jpg

What preconditions?(cont.)

  • Baseline projections

  • Trend analysis

  • Attribution to change

Slide86 l.jpg

What preconditions?

  • Get the basics right (execution, control, accounting, reporting)

  • Sort out budget process (actors, information flows, incentives and capacity) – little sense to attempt performance budgeting within environment where short term political priorities and deal making prevail over rational analysis of spending options

  • Focus on performance, and not performance information

  • Focus on process, not only targets. Process is as important as targets

  • Role and capacity of Ministries of Finance and centres of Government

  • Holistic rather than patchy approach

Contents87 l.jpg

  • Outputs & inputs

  • Outcomes

  • Indicators

  • Logic model

  • Budgeting & PI

  • Future

Slide88 l.jpg

Future - open questions

  • How does your PI dimension affect your budget process in terms of actors, information flows between them, capacity and incentives?

  • How to improve usage of PI in budgetary decision making?

  • How to improve measurement?

  • How to improve quality / user friendliness / usability of information?

  • How to get politicians to use PI in decision making?