evaluation at the world bank
Download
Skip this Video
Download Presentation
Evaluation at the World Bank

Loading in 2 Seconds...

play fullscreen
1 / 28

Evaluation at the World Bank - PowerPoint PPT Presentation


  • 99 Views
  • Uploaded on

Evaluation at the World Bank. “...we will not measure our performance by dollars lent or projects approved, but by our development impact -- results on the ground. I cannot overstate the importance of this change.” James D. Wolfensohn Annual Meeting Speech October 1996. Oslo 10/14/99.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Evaluation at the World Bank' - zaza


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
evaluation at the world bank

Evaluation at the World Bank

“...we will not measure our performance by dollars lent or projects approved, but by our development impact -- results on the ground. I cannot overstate the importance of this change.”

James D. WolfensohnAnnual Meeting SpeechOctober 1996

Oslo 10/14/99

the world bank and development
The World Bank and Development
  • A global development finance institution
    • 179 country members
    • $25-30 billion annual borrowings
  • The largest single provider of development assistance
    • over 1800 projects under implementation
    • (worth more than $140 billion)
    • 250-300 new loans/credits a year
    • (worth $20-25 billion)*

* Excluding financial crisis loans (about $8–9 billion)

the world bank and development1
The World Bank and Development
  • A far-flung development network
    • 83 field offices
    • 72 aid groups
    • 800 trust funds
  • A Bank of ideas and skills
    • A significant research and knowledge management budget ($99 million)
    • $126 million worth of advisory and aid coordination services
    • $34 million of development training (EDI)
evaluation in the world bank
Evaluation in The World Bank
  • OED is independent: it reports to the Board of Executive Directors
  • 5,000 projects have been evaluated since OED’s creation in 1970 (325 operations, worth $26 billion, in FY96 -FY97 alone)
  • The function spans a variety of sectors (agriculture, energy, finance, transport, water supply, urban development)
  • It evaluates projects, programs, processes, and policies
the development environment has evolved
The Development Environment Has Evolved

Before

Now

Ascent of GlobalCapital Market

Money Bank

Full Service Bank

Planned Development

ParticipatoryDevelopment

Greater Pluralism

Knowledge Managementand Partnerships

End of Ideological Divide

Conditionality

Advent of Civil Society

DevelopmentEffectiveness

Resource Transfer

the very concept of development has changed
Fifties

Sixties

Seventies

Eighties

Nineties

Reconstruction

Growth

Basic Needs

Adjustment

Capacity Building

The Very Concept of Development Has Changed

Physical

Social

Human

Natural

Development

Decades

Major Objective

Main

Instruments

Dominant

Discipline

Technical Assistance

Projects

Sector Investment

Adjustment Loans

Country Assistance Strategy

Engineering

Finance

Planning

Neoclassical

Multi-disciplinary

the meaning of development effectiveness has changed too
The Meaning of Development Effectiveness Has Changed Too

Development Effectiveness connotes a demonstrable and efficient contribution to economically sound, socially responsible and environmentally sustainable growth

development effectiveness means
DOING THE RIGHT THINGS

Relevance: Are the objectives we seek responsive to country needs and Bank priorities?

Institutional development: Are we helping to build the capacity of our partners?

DOING THINGS RIGHT

Efficacy: Are we achieving objectives shared by the Bank and its partners?

Efficiency: Are we using Bank and partners’ resources economically?

Development Effectiveness Means:

Sustainability: Will the benefits of Bank intervention be sustained?

world bank performance criteria reflect the new development agenda
World Bank Performance Criteria Reflect the New Development Agenda:

Environmentally

Socially

and

Sustainable

Growth

Governance: social development, economic management, public sector reform, etc.

Poverty/gender

population,

health,

nutrition

Private sector

development

Environment

Rural

development

Finance

Education

Energy/mining

Infrastructure

Social

protection

Human economy

Physical economy

Market economy

slide10

Performance characteristics

Development themes

  • ECONOMICMacro policySector policy
  • FINANCIALCorporate financePublic financeCommercial aspects
  • SOCIALIncome distributionSocial impactResettlement Indigenous groups
  • INSTITUTIONALOrganization and managementLegalGovernance aspects
  • ENVIRONMENTALTechnical optionsEngineering optionsChoice of technology
  • ECONOMIC MGMT. AdjustmentDebtTrade policyFiscal policyPublic expendituresManagement
  • POVERTY REDUCTION AND HUMAN RESOURCESEducation/trainingPopulationHealth and nutritionGender aspects
  • PSD/PSMFinancial intermediationEnabling policiesCivil service reform
  • ENVIRONMENTALNatural resource managementPollutionInfrastructureRegional planning

As vehicles for policy and institutional reform, projects are evaluated in terms of the special emphases of the development agenda.

the evaluation system is being adjusted
The Evaluation System is Being Adjusted

Characteristics of evaluation system

Dominant concepts

  • Project focus
  • Investment-driven growth
  • Import substitution
  • Central planning
  • Country focus
  • Structural Adjustment
  • Outward-oriented policies
  • Decentralized decision making
  • Project evaluation
  • Cost/benefit analysis
  • Shadow pricing
  • Self-evaluation
  • Portfolio evaluation
  • Policy evaluation
  • Risk analysis
  • Participatory evaluation

THEN

NOW

Evaluation methodologies reflect the dominant paradigms of development policy.

measuring performance matters
Measuring Performance Matters
  • For accountability: Increasingly, shareholders want to know whether they are getting “value for money.”
  • For Bank management and staff: To enhance professional excellence and get results on the ground through organizational learning.
project performance measurement system
Project Performance Measurement System

OED

    • consistent
    • objective
    • continuous
  • professional
  • external credibility
  • work in process--e.g., NLS

Quality at exit

Monitoring during

implementation

Quality at entry

QAG

QAG

real time evaluation means quality assurance
Real Time Evaluation Means Quality Assurance

Two Complementary Components

  • Portfolio Improvement Program
  • Quality Assessment Program
    • Quality at Entry
    • Supervision
    • Economic and Sector Work
it also means quality management
It also Means Quality Management
  • Target managerial attention on “high risk” clusters
  • Regular monitoring and feedback to staff and managers
  • Partnering to disseminate lessons of experience
  • Linking portfolio management indicators to staff/managerial evaluations
the other main challenge is the scaling up of evaluation
The Other Main Challenge Is the “Scaling-up” of Evaluation
  • Country, sector, and thematic evaluations
  • Adding up project-level results and linking them to network and strategic goals.
  • Non-lending services, partnerships, and strategic alliances.
  • Moving towards a results-oriented framework for performance measurement.
evaluation gaps must be filled
Evaluation Gaps Must Be Filled
  • Today’s evaluation system concentrates on lending
  • Activity completion reporting is also needed for research, grants, economic and sector work, trust funds, training, etc…
  • Evaluation must move to a “higher plane”
country as unit of account
Country as Unit of Account

Country

Assistance Agenda

Country

Assistance

Strategy

Performance

Factors

Development

Effectiveness

Results

  • Social
  • Poverty head count, change
  • Private consumption p.c. growth
  • Child mortality, change
  • School enrollment, by gender, change
  • Bank
  • Country Dialogue
  • ESW Quality
  • Quality at Entry
  • Supervision Quality
  • Portfolio Management
  • Cost Effectiveness
  • Partnerships
  • Economic
  • GDP p.c. growth
  • Export growth
  • Debt/GDP
  • Inflation

Thematic

Priorities

  • Country Assistance Strategy Achievements
  • Relevance
  • Efficacy
  • Efficiency
  • Sustainability
  • Institutional Development

Assessment of

Past Projects

Development

Priorities

Development

Objectives

Operational

Programs

  • Borrower
  • Fiscal
  • Monetary
  • Trade
  • Size of Government
  • Quality of PublicExpenditure
  • Commitment
  • Environmental
  • Country-specific measures(e.g. salinity, forest cover),change
  • Environmental Index (e.g.enforcement of environmentalregulations), change

Country’s

Policies/Capacities

Partners’

Plans/Programs

  • Private Sector
  • Development
  • Private Investment/GDP
  • Foreign Investment/GDP
  • Privatizations
  • Exogenous
  • Terms of Trade
  • Interest Rates
  • Droughts/Floods
  • War
  • Capacity Building
  • Public Sector Management
  • Governance
sectors themes as objects of evaluation
Sectors/Themes as Objects of Evaluation

Thematic

and country

programs

3 year

sector/thematic

evaluation

Environ. sustainable development

Economic/social governance

Human/

social capital

Private sector development

PSD

Finance

Industry

Telecom.

Urban dev.

Rural dev.

Environ-

ment

Economic

mgmt.

PSM

Education

PHN

Soc.

policy/

social security

Energy

Water

Transport

Annual

business plans & retrospectives

Operating

services

programs

the ultimate objective is transparent evaluation of development effectiveness at corporate level
The Ultimate Objective Is Transparent Evaluation of Development Effectiveness at Corporate Level…

ARDE

Regions

AFR

EAP

SAS

ECA

MNA

LAC

Networks

HDD

ESSD

FPSI

PREM

Country Assistance

Strategy and Compacts

Sector Assistance Strategies

and Thematic Progress Reports

Nonlending Services

Nonlending Services

Lending Services

Lending Services

Aid

Coordination

and

Cofinancing

Reviews

Activity

Completion

Reports on

Nonlending

Services

Performance

Audit Reports

and Impact

Evaluations

Portfolio

Improvement

Program

Reviews

Research

Activity

Completion

Reports

Imple.

Completion

Reports

Sector

Impact

Eval.

Trust Funds

Completion

Memos

Supervision

Reports

Mid-term

Reviews

Regions/

OPR

Regions/

OPR

Networks/

OED

Regions/

QAG/OED

Regions/

OED

DEC/

Networks

Regions/

OPR

Regions/Networks

OED

QAG

OED

Proposed

combined with regular monitoring of bank performance scorecard
…Combined with Regular Monitoring of Bank Performance (Scorecard)

Borrowing

spreads

Reserves/

loan ratio

Investment

return

Admin.

expenses as

% of assets

Interest

coverage

ratio

  • Quality at exit:
  • Outcomes
  • Sustainability
  • Institutional development
  • Bank performance

TRE, RM, FPI

TRE, FPI

FPI

TRE, FPI

PBD

Client feedback

on staff

performance

OED

Regions

Financial

strength

  • Quality at entry:
  • 10 ECON commandments

Processing

times

OED/QAG

  • Portfolio management quality:
  • Projects at risk
  • Realism index
  • Proactivity index

PBD

Score-card on

Bank

performance

Client

responsiveness

Development

effectiveness

Disbursement

rates

QAG

PBD

  • Quality of non lending serv.:
  • Relevance
  • Efficacy
  • Efficiency

Values and

skills

Front-line services as % of total admin. cost

OED/QAG

PBD

  • Cost effec. indicators:
  • Completion coefficients
  • Cost/SY

Effectiveness of

staff training

Staff feedback

from surveys

Skills

assessment

PBD

Networks

LLC

ICD

this vision implies partnerships in evaluation
This Vision Implies Partnerships in Evaluation

Development effectiveness requires participatory evaluation

  • Locally: to support management conditions
  • Globally: to support strategic alliances

Thus ensuring proactivity of the overall development system

  • Global networking among evaluators
  • Evaluation capacity development
  • beneficiary involvement in evaluation
slide23

NGOs

Multilateral

Strategic alliances

Bilateral

Global

and

Regional

PERFORMANCE PARTNERSHIPS

Civil society

Country

Management coalitions

Government

Donors

slide24

Operations Evaluation Department

DGO (171/99)

Director OED (OEDDR 175/05)

Resource Management

Operations Advisor

Manager

Country Evaluations

& Regional Relations

(OEDCR - 175/20)

Manager

Sector & Thematic

Evaluations

(OEDST - 175/10)

Manager

Partnership & Knowledge

Programs

(OEDPK - 175/40)

Poverty, Gender, Social Development & Social Protection

Rural Development

Environment and Water

Finance and PSD

PSM

Energy,Mining & Telecommunication

Education

Knowledge

Management

HNP

Country Economics

Urban & Infrastructure

ECDP, Partnerships &

Training

Manager

Corporate Evaluations

and Methods

(OEDCM - 175/50)

Transport

Evaluation Process and Methods

Clusters are Interactive and Flexible

ACS Staff assigned to managerial group and cluster

= Thematic & Regional Coordinators

ad