Impact Evaluation Clinics for:
Download
1 / 25

Impact Evaluation Clinics for: The Institutional Support to Rural Sector Public Service Administration (AMOASR) Program in Cameroon - PowerPoint PPT Presentation


  • 317 Views
  • Updated On :
  • Presentation posted in: Travel / Places

Impact Evaluation Clinics for: The Institutional Support to Rural Sector Public Service Administration (AMOASR) Program in Cameroon. Case study 2 NONIE meeting in Lisbon October 1, 2008. Location of Cameroon in Central Africa. Administrative Organisation of the country. 10 provinces

Related searches for Impact Evaluation Clinics for: The Institutional Support to Rural Sector Public Service Administration (AMOASR) Program in Cameroon

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha

Download Presentation

Impact Evaluation Clinics for: The Institutional Support to Rural Sector Public Service Administration (AMOASR) Program in Cameroon

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Slide1 l.jpg

Impact Evaluation Clinics for: The Institutional Support to Rural Sector Public Service Administration (AMOASR) Program in Cameroon

Case study 2

NONIE meeting in Lisbon

October 1, 2008

Network of Networks on Impact Evaluation


Location of cameroon in central africa l.jpg

Location of Cameroon in Central Africa

Network of Networks on Impact Evaluation


Administrative organisation of the country l.jpg

AdministrativeOrganisationof the country

  • 10 provinces

  • 58 Divisions - départements

  • Subdivisions – sous-préfectures

  • Districts

Network of Networks on Impact Evaluation


Section a presentation of project l.jpg

Section A: Presentation of project

Network of Networks on Impact Evaluation


Outline of presentation l.jpg

Outline of presentation

  • Description of the problem the program deals with

  • Objectives of the program & results chain

  • Program theory/logic model

  • Program activities & budget allocations

  • Description of the target groups & selection criteria

  • A Summary of Administrative Data on the Program (financiers, implementation arrangements, budget, evaluation budget share)

  • Challenges to Evaluation of the Impacts of AMOASR Program

Network of Networks on Impact Evaluation


What is the problem l.jpg

What is the problem?

  • 1980’s: Huge budget cuts due to economic & public fiscal hardship

  • 1987 to Date: Attempt to focus on regulatory functions rather than on productive and redistributive functions – need for redeployment

  • Agriculture contributes 22 to 30% share of GDP and 60-75% of employment, and was therefore targeted to be the main drive of poverty alleviation efforts of the Government under the HIPC initiative

  • 2002: Organizational assessment of the Ministry of Agriculture & Rural Development (MINADER) & Ministry of Livestock, Fisheries and Animal Industries (MINEPIA)

Network of Networks on Impact Evaluation


What is the problem7 l.jpg

What is the problem?

  • Policy and program design is not informed enough by evidence and hard data

  • Weak coordination of policy implementation

  • Lack of appropriation of national policies and directives

  • Ineffective programming in terms of changes achieved at farmer level (work plans are not results-oriented)

  • Few activities and related budget are implemented as planned

  • Decision makers lack information to adjust implementation accordingly to changes in the context and observed performance

Network of Networks on Impact Evaluation


Objectives of the program l.jpg

Objectives of the program

  • Development Objective:

    • Strengthen the institutional capacity of MINADER and MINEPIA to implement the Agriculture, Livestock and Fisheries component of the rural sector development strategy (SDSR)

  • Specific objectives:

    • Strengthen the audit, monitoring & evaluation technical capacity of the two ministries;

    • Support the renovation of the agricultural, pastoral & fisheries statistical system;

    • Strengthen the organizational capacity of the services at national, provincial and divisional levels in the two ministries;

    • Promote dialogue among the different groups of players in the rural sector.

Network of Networks on Impact Evaluation


Results chain l.jpg

Results Chain

  • Impacts:

    • Better governance at national, provincial and divisional levels in MINADER and MINEPIA administrations

    • Increased impacts of public development interventions in the rural sector (Farmers and their organizations, private agric. firms and civil society organizations)

  • Outcomes:

    • Increased consumption rate of domestic budget and absorption rate of aid funds;

    • Results-oriented (annual & medium term) plans and budgets implemented at various levels of the public services of the target ministries;

    • Institutionalized dialogue between public decision makers and farmers organizations

    • Increased public service output delivery rate.

  • Outputs:

    • Training services

    • Technical assistance (design of procedures and templates; development of tools and software; communication packages)

    • Budget support

Network of Networks on Impact Evaluation


Program theory logic model l.jpg

Inputs

Activities

Outputs

Outcomes

Impacts

Staff

Technical assistance (statistics, M&E units, planning units, procurement and budget units)

Systems, procedures and tools in place, including software

Strengthened planning, programming, budgeting and monitoring capacity of MINADER & MINEPIA services

C2D Funds

Better governance of MINADER & MINEPIA services

Matching Funds

Procurement of goods and services for MINADER & MINEPIA services

Training services (sessions)

Results-oriented (annual & medium term) plans and budgets implemented at various levels of the ministries

Budget available in time to services

Goods available in time to services

M&E of the AMOASR Program

Increased consumption rate of domestic budget and absorption rate of aid funds

Increased farmer production & revenues

Overall Management of the AMOASR Program

Other contributing outputs (incentives & motivation; norms & values; leadership style; etc.)

Increased public service output delivery rate

Improved rural standard of living

Program Theory/Logic Model

Network of Networks on Impact Evaluation


Project activities and yearly budget allocation in usd 1 usd 453 cfaf or xaf l.jpg

Project activities and yearly budget allocation (in USD)1 USD = 453 CFAF or XAF

Network of Networks on Impact Evaluation


Target groups beneficiaries l.jpg

Target groups/beneficiaries

  • Services involved in the PPBS chain in the

    • Ministry of Agriculture & Rural Development (MINADER)

    • Ministry of Livestock, Fisheries & Animal Industries (MINEPIA)

  • 235 services in the 2 ministries

Network of Networks on Impact Evaluation


Target groups beneficiaries13 l.jpg

Target groups/beneficiaries

  • In the Ministry of Agriculture & Rural Development (MINADER):

    • The general inspectorate at the national level

    • 05 directorates at the national level

    • 30 services at provincial level (3 in each of the 10 provinces)

    • 116 services at division level (2 in each of the 58 divisions)

Network of Networks on Impact Evaluation


Target groups beneficiaries14 l.jpg

Target groups/beneficiaries

  • In the Ministry of Livestock, Fisheries & Animal Industries (MINEPIA):

    • The general inspectorate at the national level

    • 04 directorates at the national level

    • 20 services at provincial level (2 in each of the 10 provinces)

    • 58 services at division level (1 in each of the 58 divisions)

Network of Networks on Impact Evaluation


Selection of participants l.jpg

Selection of participants

  • All Services involved in the PPBS chain

  • Personnel in place (no recruitment)

Network of Networks on Impact Evaluation


Summary administrative data on amoasr l.jpg

Summary Administrative Data on AMOASR

  • Financiers:

    • Government of Cameroon (?%)

    • France’s Ministry of Foreign Affairs/C2D (?%)

  • Date of Approval: Feb. 2008

  • Start of Operation: August 2008

  • Total Budget: USD 15 748 189

  • Evaluation Budget: USD 397 350 (2,52%)

  • Administrative Arrangements:

    • Steering Committee

    • A Central Implementation Unit (7 Staff)

    • In-Service Focal Points (at beneficiaries level)

Network of Networks on Impact Evaluation


Data availability l.jpg

Data Availability

  • Very few data exist on the operations of the services in MINADER & MINEPIA including the routine administrative data:

    • Number and profile of personnel

    • Number and profile of the clients served – farmers, investors, other administrations, etc.

    • Description of services delivered

Network of Networks on Impact Evaluation


Section b impact evaluation of project l.jpg

Section B: Impact evaluation of project

Network of Networks on Impact Evaluation


Evaluation framework and context l.jpg

Evaluation framework and context

  • The development outcome that the project is seeking to address seems to be improved farmer production and revenues

    • Would the government and donor be satisfied if the governance of administration was improved but farmer welfare remained the same?

    • If not, complementary activities need to be considered to attain this ultimate objective

  • Who are the target group to be evaluated?

    • Currently, the project targets the capacity building for back office government personnel, not the «ultimate client«, the farmer.

Network of Networks on Impact Evaluation


Project outcomes and indicators l.jpg

Project outcomes and indicators

  • Outcomes and indicators for the project as designed

    • OUTCOME: Back office delivers effective and timely services to front line staff

    • Through improved capacity of back-office (indicator) measured by front office staff perceptions

    • Through compliance of plans with result-based standards (indicator, desk review of standards, based on expert assessments of quality of services)

Network of Networks on Impact Evaluation


Impact evaluation question l.jpg

Impact Evaluation question

  • Has the project’s inputs (training, goods and services etc.) improved back office delivery of services to front line staff?

    • The implied theory behind this evaluation question is that improved back office service delivery -> improved front line services to farmers -> improved farmer productivity and welfare


Program theory logic model revised l.jpg

Inputs

Activities

Outputs

Outcomes

Impacts

Staff

Technical assistance (statistics, M&E units, planning units, procurement and budget units)

Systems, procedures and tools in place, including software

Strengthened planning, programming, budgeting and monitoring capacity of MINADER & MINEPIA services

Better governance of MINADER & MINEPIA back office services

C2D Funds

Matching Funds

Procurement of goods and services for MINADER & MINEPIA services

Training services (sessions)

Results-oriented (annual & medium term) plans and budgets implemented at various levels of the ministries

Improved public service delivery by frontline services

Budget available in time to services

Goods available in time to services

M&E of the AMOASR Program

Other external factors (weather, price of the inputs, marketing infrastructure)

Increased farmer production & revenues

Overall Management of the AMOASR Program

Other contributing outputs (incentives & motivation; norms & values; leadership style)

Improved rural standard of living

Program Theory/Logic Model (revised)

Network of Networks on Impact Evaluation


Impact evaluation design l.jpg

Impact Evaluation Design

  • Who will be surveyed and how?

    • Unit of analysis is the service unit (both central level, and samples from provincial and division level)

    • Information collected will have to be for the entire MINADER and MINEPIA services including front line services

    • At provincial and division level, stratified random sample of units

  • Mixed methods approach (for each selected unit):

    • Focus group discussions to look at perceived barriers to set hypotheses, and use findings to design questionnaire for quantitative and qualitative surveys

    • Survey of front line service looking at perceived quality of back-office services

    • Desk review (expert assessment) of compliance with service standards and implementation

  • Baseline survey to assess current performance level

    • Follow-up surveys after 40% coverage of services, at mid-term and at completion

    • Follow ups use the rollout of service delivery to establish the counterfactual of impact on back-office services when project services have not been delivered

Network of Networks on Impact Evaluation


Impact evaluation challenges l.jpg

Impact Evaluation Challenges

  • Program Spillovers:

    • All projects implemented by MINADER/MINEPIA: the more effective is AMOASR, the more are impacts of RDI of those ministries

    • High mobility of personnel (trained ones may leave to others ministries; retirement)

  • Sources of Contamination:

    • Public Service Wide Modernization Project – Limited Influence

    • No similar initiatives within the target ministries

Network of Networks on Impact Evaluation


Estimation methods and dealing with challenges l.jpg

Estimation methods, and dealing with challenges

  • Quantitative survey has questions both measuring perceptions, but also including demographic and unit-specific characteristics (including geographic location, level of education/background of front-line etc.)

  • Identify sources of variation among surveyed units (using country knowledge and focus groups)

    • A robust quantitative questionnaire will be critical to evaluation success

    • The non-quantitative technical evaluation of services will also yield sources of variation

  • Difference-In-Difference approach to take care of some selection bias, and to take advantage of rollout feature.

    • Follow up focus group to tease out the ``why´´

  • Contagion issue a problem – do units know that others have received program services and thus have greater expectations about program services?

    • Include question about how much they know of program

Network of Networks on Impact Evaluation


ad
  • Login