Evaluation at the world bank
Download
1 / 28

Evaluation at the World Bank - PowerPoint PPT Presentation


  • 98 Views
  • Uploaded on

Evaluation at the World Bank. “...we will not measure our performance by dollars lent or projects approved, but by our development impact -- results on the ground. I cannot overstate the importance of this change.” James D. Wolfensohn Annual Meeting Speech October 1996. Oslo 10/14/99.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Evaluation at the World Bank' - zaza


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Evaluation at the world bank

Evaluation at the World Bank

“...we will not measure our performance by dollars lent or projects approved, but by our development impact -- results on the ground. I cannot overstate the importance of this change.”

James D. WolfensohnAnnual Meeting SpeechOctober 1996

Oslo 10/14/99


The world bank and development
The World Bank and Development

  • A global development finance institution

    • 179 country members

    • $25-30 billion annual borrowings

  • The largest single provider of development assistance

    • over 1800 projects under implementation

    • (worth more than $140 billion)

    • 250-300 new loans/credits a year

    • (worth $20-25 billion)*

      * Excluding financial crisis loans (about $8–9 billion)


The world bank and development1
The World Bank and Development

  • A far-flung development network

    • 83 field offices

    • 72 aid groups

    • 800 trust funds

  • A Bank of ideas and skills

    • A significant research and knowledge management budget ($99 million)

    • $126 million worth of advisory and aid coordination services

    • $34 million of development training (EDI)


Evaluation in the world bank
Evaluation in The World Bank

  • OED is independent: it reports to the Board of Executive Directors

  • 5,000 projects have been evaluated since OED’s creation in 1970 (325 operations, worth $26 billion, in FY96 -FY97 alone)

  • The function spans a variety of sectors (agriculture, energy, finance, transport, water supply, urban development)

  • It evaluates projects, programs, processes, and policies


The development environment has evolved
The Development Environment Has Evolved

Before

Now

Ascent of GlobalCapital Market

Money Bank

Full Service Bank

Planned Development

ParticipatoryDevelopment

Greater Pluralism

Knowledge Managementand Partnerships

End of Ideological Divide

Conditionality

Advent of Civil Society

DevelopmentEffectiveness

Resource Transfer


The very concept of development has changed

Fifties

Sixties

Seventies

Eighties

Nineties

Reconstruction

Growth

Basic Needs

Adjustment

Capacity Building

The Very Concept of Development Has Changed

Physical

Social

Human

Natural

Development

Decades

Major Objective

Main

Instruments

Dominant

Discipline

Technical Assistance

Projects

Sector Investment

Adjustment Loans

Country Assistance Strategy

Engineering

Finance

Planning

Neoclassical

Multi-disciplinary


The meaning of development effectiveness has changed too
The Meaning of Development Effectiveness Has Changed Too

Development Effectiveness connotes a demonstrable and efficient contribution to economically sound, socially responsible and environmentally sustainable growth


Development effectiveness means

DOING THE RIGHT THINGS

Relevance: Are the objectives we seek responsive to country needs and Bank priorities?

Institutional development: Are we helping to build the capacity of our partners?

DOING THINGS RIGHT

Efficacy: Are we achieving objectives shared by the Bank and its partners?

Efficiency: Are we using Bank and partners’ resources economically?

Development Effectiveness Means:

Sustainability: Will the benefits of Bank intervention be sustained?


World bank performance criteria reflect the new development agenda
World Bank Performance Criteria Reflect the New Development Agenda:

Environmentally

Socially

and

Sustainable

Growth

Governance: social development, economic management, public sector reform, etc.

Poverty/gender

population,

health,

nutrition

Private sector

development

Environment

Rural

development

Finance

Education

Energy/mining

Infrastructure

Social

protection

Human economy

Physical economy

Market economy


Performance characteristics Agenda:

Development themes

  • ECONOMICMacro policySector policy

  • FINANCIALCorporate financePublic financeCommercial aspects

  • SOCIALIncome distributionSocial impactResettlement Indigenous groups

  • INSTITUTIONALOrganization and managementLegalGovernance aspects

  • ENVIRONMENTALTechnical optionsEngineering optionsChoice of technology

  • ECONOMIC MGMT. AdjustmentDebtTrade policyFiscal policyPublic expendituresManagement

  • POVERTY REDUCTION AND HUMAN RESOURCESEducation/trainingPopulationHealth and nutritionGender aspects

  • PSD/PSMFinancial intermediationEnabling policiesCivil service reform

  • ENVIRONMENTALNatural resource managementPollutionInfrastructureRegional planning

As vehicles for policy and institutional reform, projects are evaluated in terms of the special emphases of the development agenda.


The evaluation system is being adjusted
The Evaluation System is Being Adjusted Agenda:

Characteristics of evaluation system

Dominant concepts

  • Project focus

  • Investment-driven growth

  • Import substitution

  • Central planning

  • Country focus

  • Structural Adjustment

  • Outward-oriented policies

  • Decentralized decision making

  • Project evaluation

  • Cost/benefit analysis

  • Shadow pricing

  • Self-evaluation

  • Portfolio evaluation

  • Policy evaluation

  • Risk analysis

  • Participatory evaluation

THEN

NOW

Evaluation methodologies reflect the dominant paradigms of development policy.


Measuring performance matters
Measuring Performance Matters Agenda:

  • For accountability: Increasingly, shareholders want to know whether they are getting “value for money.”

  • For Bank management and staff: To enhance professional excellence and get results on the ground through organizational learning.


Project performance measurement system
Project Performance Agenda:Measurement System

OED

  • consistent

  • objective

  • continuous

  • professional

  • external credibility

  • work in process--e.g., NLS

  • Quality at exit

    Monitoring during

    implementation

    Quality at entry

    QAG

    QAG


    Real time evaluation means quality assurance
    Real Time Evaluation Means Quality Assurance Agenda:

    Two Complementary Components

    • Portfolio Improvement Program

    • Quality Assessment Program

      • Quality at Entry

      • Supervision

      • Economic and Sector Work


    It also means quality management
    It also Means Quality Management Agenda:

    • Target managerial attention on “high risk” clusters

    • Regular monitoring and feedback to staff and managers

    • Partnering to disseminate lessons of experience

    • Linking portfolio management indicators to staff/managerial evaluations


    The other main challenge is the scaling up of evaluation
    The Other Main Challenge Is the “Scaling-up” of Evaluation

    • Country, sector, and thematic evaluations

    • Adding up project-level results and linking them to network and strategic goals.

    • Non-lending services, partnerships, and strategic alliances.

    • Moving towards a results-oriented framework for performance measurement.


    Evaluation gaps must be filled
    Evaluation Gaps Must Be Filled Evaluation

    • Today’s evaluation system concentrates on lending

    • Activity completion reporting is also needed for research, grants, economic and sector work, trust funds, training, etc…

    • Evaluation must move to a “higher plane”


    Country as unit of account
    Country as Unit of Account Evaluation

    Country

    Assistance Agenda

    Country

    Assistance

    Strategy

    Performance

    Factors

    Development

    Effectiveness

    Results

    • Social

    • Poverty head count, change

    • Private consumption p.c. growth

    • Child mortality, change

    • School enrollment, by gender, change

    • Bank

    • Country Dialogue

    • ESW Quality

    • Quality at Entry

    • Supervision Quality

    • Portfolio Management

    • Cost Effectiveness

    • Partnerships

    • Economic

    • GDP p.c. growth

    • Export growth

    • Debt/GDP

    • Inflation

    Thematic

    Priorities

    • Country Assistance Strategy Achievements

    • Relevance

    • Efficacy

    • Efficiency

    • Sustainability

    • Institutional Development

    Assessment of

    Past Projects

    Development

    Priorities

    Development

    Objectives

    Operational

    Programs

    • Borrower

    • Fiscal

    • Monetary

    • Trade

    • Size of Government

    • Quality of PublicExpenditure

    • Commitment

    • Environmental

    • Country-specific measures(e.g. salinity, forest cover),change

    • Environmental Index (e.g.enforcement of environmentalregulations), change

    Country’s

    Policies/Capacities

    Partners’

    Plans/Programs

    • Private Sector

    • Development

    • Private Investment/GDP

    • Foreign Investment/GDP

    • Privatizations

    • Exogenous

    • Terms of Trade

    • Interest Rates

    • Droughts/Floods

    • War

    • Capacity Building

    • Public Sector Management

    • Governance


    Sectors themes as objects of evaluation
    Sectors/Themes as Objects Evaluationof Evaluation

    Thematic

    and country

    programs

    3 year

    sector/thematic

    evaluation

    Environ. sustainable development

    Economic/social governance

    Human/

    social capital

    Private sector development

    PSD

    Finance

    Industry

    Telecom.

    Urban dev.

    Rural dev.

    Environ-

    ment

    Economic

    mgmt.

    PSM

    Education

    PHN

    Soc.

    policy/

    social security

    Energy

    Water

    Transport

    Annual

    business plans & retrospectives

    Operating

    services

    programs


    The ultimate objective is transparent evaluation of development effectiveness at corporate level
    The Ultimate Objective Is Transparent Evaluation of Development Effectiveness at Corporate Level…

    ARDE

    Regions

    AFR

    EAP

    SAS

    ECA

    MNA

    LAC

    Networks

    HDD

    ESSD

    FPSI

    PREM

    Country Assistance

    Strategy and Compacts

    Sector Assistance Strategies

    and Thematic Progress Reports

    Nonlending Services

    Nonlending Services

    Lending Services

    Lending Services

    Aid

    Coordination

    and

    Cofinancing

    Reviews

    Activity

    Completion

    Reports on

    Nonlending

    Services

    Performance

    Audit Reports

    and Impact

    Evaluations

    Portfolio

    Improvement

    Program

    Reviews

    Research

    Activity

    Completion

    Reports

    Imple.

    Completion

    Reports

    Sector

    Impact

    Eval.

    Trust Funds

    Completion

    Memos

    Supervision

    Reports

    Mid-term

    Reviews

    Regions/

    OPR

    Regions/

    OPR

    Networks/

    OED

    Regions/

    QAG/OED

    Regions/

    OED

    DEC/

    Networks

    Regions/

    OPR

    Regions/Networks

    OED

    QAG

    OED

    Proposed


    Combined with regular monitoring of bank performance scorecard
    …Combined with Regular Monitoring of Bank Performance (Scorecard)

    Borrowing

    spreads

    Reserves/

    loan ratio

    Investment

    return

    Admin.

    expenses as

    % of assets

    Interest

    coverage

    ratio

    • Quality at exit:

    • Outcomes

    • Sustainability

    • Institutional development

    • Bank performance

    TRE, RM, FPI

    TRE, FPI

    FPI

    TRE, FPI

    PBD

    Client feedback

    on staff

    performance

    OED

    Regions

    Financial

    strength

    • Quality at entry:

    • 10 ECON commandments

    Processing

    times

    OED/QAG

    • Portfolio management quality:

    • Projects at risk

    • Realism index

    • Proactivity index

    PBD

    Score-card on

    Bank

    performance

    Client

    responsiveness

    Development

    effectiveness

    Disbursement

    rates

    QAG

    PBD

    • Quality of non lending serv.:

    • Relevance

    • Efficacy

    • Efficiency

    Values and

    skills

    Front-line services as % of total admin. cost

    OED/QAG

    PBD

    • Cost effec. indicators:

    • Completion coefficients

    • Cost/SY

    Effectiveness of

    staff training

    Staff feedback

    from surveys

    Skills

    assessment

    PBD

    Networks

    LLC

    ICD


    This vision implies partnerships in evaluation
    This Vision Implies (Scorecard)Partnerships in Evaluation

    Development effectiveness requires participatory evaluation

    • Locally: to support management conditions

    • Globally: to support strategic alliances

      Thus ensuring proactivity of the overall development system

    • Global networking among evaluators

    • Evaluation capacity development

    • beneficiary involvement in evaluation


    NGOs (Scorecard)

    Multilateral

    Strategic alliances

    Bilateral

    Global

    and

    Regional

    PERFORMANCE PARTNERSHIPS

    Civil society

    Country

    Management coalitions

    Government

    Donors


    Operations Evaluation Department (Scorecard)

    DGO (171/99)

    Director OED (OEDDR 175/05)

    Resource Management

    Operations Advisor

    Manager

    Country Evaluations

    & Regional Relations

    (OEDCR - 175/20)

    Manager

    Sector & Thematic

    Evaluations

    (OEDST - 175/10)

    Manager

    Partnership & Knowledge

    Programs

    (OEDPK - 175/40)

    Poverty, Gender, Social Development & Social Protection

    Rural Development

    Environment and Water

    Finance and PSD

    PSM

    Energy,Mining & Telecommunication

    Education

    Knowledge

    Management

    HNP

    Country Economics

    Urban & Infrastructure

    ECDP, Partnerships &

    Training

    Manager

    Corporate Evaluations

    and Methods

    (OEDCM - 175/50)

    Transport

    Evaluation Process and Methods

    Clusters are Interactive and Flexible

    ACS Staff assigned to managerial group and cluster

    = Thematic & Regional Coordinators


    Trends in oed work program fy96 fy99 fy00 02
    Trends in OED Work Program (Scorecard)FY96, FY99 - FY00–02


    Trends in oed work program fy96 fy99 fy00 021
    Trends in OED Work Program (Scorecard)FY96, FY99 - FY00–02


    Trends in oed work program fy96 fy99 fy00 022
    Trends in OED Work Program (Scorecard)FY96, FY99 - FY00–02


    Trends in oed work program fy96 fy99 fy00 023
    Trends in OED Work Program (Scorecard)FY96, FY99 - FY00–02


    ad