evaluation of accelerated child survival programs in africa l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Evaluation of Accelerated Child Survival Programs in Africa PowerPoint Presentation
Download Presentation
Evaluation of Accelerated Child Survival Programs in Africa

Loading in 2 Seconds...

play fullscreen
1 / 37

Evaluation of Accelerated Child Survival Programs in Africa - PowerPoint PPT Presentation


  • 184 Views
  • Uploaded on

Institute for International Programs. ASADI V Accra, November 2009 . Evaluation of Accelerated Child Survival Programs in Africa. Jennifer Bryce Institute for International Programs The Johns Hopkins University. Outline – What have we learned?.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Evaluation of Accelerated Child Survival Programs in Africa' - jacob


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
evaluation of accelerated child survival programs in africa

Institute for International Programs

ASADI V

Accra, November 2009

Evaluation of Accelerated Child Survival Programs in Africa

Jennifer Bryce

Institute for International Programs

The Johns Hopkins University

outline what have we learned
Outline – What have we learned?
  • From the evaluation of the Accelerating Child Survival and Development (ACSD) Program?
  • From prospective evaluations of the Catalytic Initiative to Save One Miiilion Lives (CI) to date?
  • The way forward
acsd 2002 2005
ACSD, 2002-2005

11 countries in Africa

Support from CIDA and other partners

Implemented through UNICEF

Aim: To reduce mortality among children less than 5 years of age

Strategy: Accelerate coverage with three packages of high-impact interventions, with a special focus on community-based delivery

acsd program intervention packages
ACSD Program:Intervention packages

EPI+

Vaccinations

Vitamin A supplementation

ITNs for U5s & pregnant women

De-worming

  • IMCI+
    • Facility IMCI
    • Community case management (CCM) of childhood illnesses
      • Diarrhea: oral rehydration therapy (ORT)
      • Malaria: based on current policy
      • Pneumonia: referral to facility
    • Promotion of timely initiation of breastfeeding, exclusive breastfeeding to 6 months, timely complementary feeding
    • Promotion of household consumption of iodized salt
  • ANC+
  • Malaria prevention in pregnant women (IPTp)
  • Tetanus Toxoid
  • Iron/folic acid supplementation
  • Vitamin A post-partum
  • PMTCT
the retrospective independent evaluation of acsd
The retrospective independent evaluation of ACSD
  • High-impact districts in Benin, Ghana, Mali
  • Standard indicators
  • Existing DHS/MICS with oversampling
  • National comparison areas
  • Documentation of program implementation & contextual factors
  • No cost component
  • Stepwise design
slide6

ACSD Implementation: GHANA

Key: Bars represent districts in the following order: Builsa, Bawku East, Kasena-Nankana, Bolgatanga, Bawku West, Bongo

coverage for epi interventions before and after acsd in hids

Key

Coverage for EPI+ interventionsbefore and after ACSD, in HIDs

Before ACSD

After ACSD

Mali

Benin

Ghana

*

*

3

*

*

*

*

*

*

*

*

Increases in coverage across the board in Ghana and Mali; Benin achieved increases for vitamin A and ITNs.

*

Change was significant at p < 0.05.

coverage for imci interventions before and after acsd in hids

Key

Coverage for IMCI+ interventionsbefore and after ACSD, in HIDs

Before ACSD

After ACSD

Mali

Benin

Ghana

*

*

*

*

*

*

*

No coverage gains, and some significant losses, in sick child care. Exclusive breastfeeding increased in Ghana, declined in Mali.

*

Change was significant at p < 0.05.

coverage for anc interventions before and after acsd in hids

Key

Coverage for ANC+ interventionsbefore and after ACSD, in HIDs

Before ACSD

After ACSD

Mali

Benin

Ghana

*

*

**

*

*

*

*

*

*

*

*

*

Ghana and Mali improved care for childbearing women; delivery of TT and postnatal vit A benefited from EPI system in Mali.

*Change was significant at p ≤ 0.05.

** Measured level was 28%, but country team reported this was incorrect as IPTp had not been implemented in 2001.

under five mortality in the acsd hids and national comparison areas
Under-five mortality in the ACSD HIDs and national comparison areas

Declines in U5M in ACSD focus districts,

but not greater than national comparison areas.

No changes in child

nutritional status attributable

to ACSD.

did acsd implementation contribute to reducing inequities
Did ACSD implementation contribute to reducing inequities?

Socioeconomic inequalities, showing breakdown by wealth quintiles of ANC 3+ coverage in ACSD “high-impact” zones and the comparison area, Mali, 2006-7.

  • Yes, in Mali, where socioeconomic and urban/rural inequities decreased more in the ACSD HIDs than in the comparison area.
  • Baseline sample sizes too small to support analysis of equity trends in Benin or Ghana.
conclusions implications
Conclusions & implications
  • Intervention coverage CAN be accelerated if there is adequate funding & human resources.
  • Acceleration of mortality declines require:
    • Focus on interventions that have a large and rapid impact on major causes of child death
    • Sufficient time to fully implement approach and for coverage to translate into declines in mortality
    • Reasonable expectations, given level of resources

► Work for closer match between program resources & cause of death

►Be realistic about what can be accomplished

► Level of funding matters

conclusions implications14
Conclusions & implications
  • Policy barriers prevented key ACSD interventions directed at pneumonia and malaria from being fully implemented.
  • Breakdowns in commodities and gaps in funding stall progress toward impact.
  • More attention and operations research needed on incentives and supports for community-based workers

► Work for policy reform as first step, where needed

► Pay attention to health systems supports such as commodities, supervision, & incentives

contributors acknowledgements
Contributors & acknowledgements

Contributors

Acknowledgements

Ministries of Health, National Statistics Offices, UNICEF country staff, Collaborators in documentation

UNICEF regional and global staff Genevieve Begkoyian, Mark Young, Sam Bickel

Technical consultants Trevor Croft, Macro International

UNICEF leadership For their commitment to learning and change

Jennifer Bryce

Kate Gilroy

Elizabeth Hazel

Gareth Jones

Robert Black

Cesar Victora

two linked evaluations
Two Linked Evaluations

The Catalytic Initiative

Independent Evaluation of the MNCH Rapid Scale-Up

“Real-time”

Mortality Monitoring (RMM)

Overall objective: To monitor changes in under-five mortality in real-time.

  • Overall objective: Provide “proof of concept” that proven interventions can be scaled up rapidly to reduce newborn and child mortality.
  • Supported by: BMGF
  • Implementing partners: Governments and UNICEF, WHO, UNFPA
  • Supported by: CIDA
  • Implementing partner: Governments and UNICEF
  • Countries: Burkina Faso, Malawi, Mozambique
  • Countries: Ghana, Malawi, Mali, Mozambique
progress malawi in country partners centre for social research and national statistics office
Progress: MalawiIn-country partners: Centre for Social Research and National Statistics Office

Implementation

Evaluation (full)

Mortality monitored by:

Having CHWs report vital events

Calibrating facility deaths against community deaths

Two rapid survey approaches

Full documentation of program & contextual factors

Quality of care assessments at 1st-level facilities and for CHWs

Costs & equity tracked

12 districts: 6 “accelerated” and 6 routine

National platform approach under discussion

  • Features of accelerated approach:
    • Government-paid CHWs trained to deliver CCM for pneumonia, malaria, diarrhea (including zinc)
    • Strengthening district health management
  • Implementation status:
    • In 10 intervention districts, 5-15% of CHWs trained by June 2009
progress mozambique in country partner eduardo mondlane university
Progress: MozambiqueIn-country partner: Eduardo Mondlane University

Implementation

Evaluation (full)

  • Mortality monitoring by calibrating facility to community deaths.
  • Stepped-wedge design based on scheduled cohorts for Rapid Scale Up
  • Documentation of baseline health & nutrition, inputs & contextual factors and coverage for each cohort using national evaluation platform approach
  • Will support dose (program intensity) – response (coverage & modeled impact) analyses
  • Features of accelerated approach:
        • Increased access to quality child health care in facilities
        • Quality of immunization services improved
        • Long-lasting insecticide-treated nets (ITNs) distributed and used
        • Vitamin A supplementation
        • Breastfeeding promotion
        • BMGF funds used to fill gaps in maternal health
  • Implementation status:
        • CI planned to be implemented in 33 districts each year from 2008 to 2012
progress burkina faso in country partner institut sup rieur des sciences de la population issp
Progress: Burkina FasoIn-Country Partner: Institut Supérieur des Sciences de la Population (ISSP)

Implementation

Evaluation (no RMM)

  • National platform approach in 9 intervention and 2-3 comparison districts
  • New “LiST” survey to collect district-level estimates of coverage for proven MNCH interventions
  • Modeled mortality using LiST
  • Analysis using pre-/post-intervention with comparison and/or dose response
  • Features of accelerated approach:
    • Volunteer community health workers providing:
      • CCM for diarrhea (ORT + zinc) and malaria (ACT) in 9 districts
      • CCM for pneumonia in 2 districts
    • Strengthened district planning and supervision
  • Implementation status:
      • Materials ready; cascade training of all CHWs in 9 districts to be completed before end 2009
slide22

Lesson 1:Existing plans include high-impact interventions…

*If targets fully achieved at adequate service quality.

slide23
…but if feasibility and speed are issues,just 4 or 5 interventions can achieve ≥ 20% reduction in U5M by 2015

Pre-publication results; not for citation or distribution

lesson 2 implementation takes time
Lesson 2:Implementation takes time

Implementation status of functional village health clinics with CHW trained in CCM, Malawi CI districts, June 2009

(18 months after project start-up)

*1 trained CHW per village health clinic

especially when policy reform is needed
…especially when policy reform is needed.

In Mali, the MoH scheduled a “forum” to decide

on CCM for childhood pneumonia & malaria.

Discussions about how to implement are still under way

Forum held; agreed “YES” on CCM

Original date

(cancelled)

Planned (cancelled)

Planned (cancelled)

July 2008

March

2009

November

2008

February

2009

November

2009

Months

4

+

3

+

1

=

7 months

in a 3-year CI project

lesson 3 virgin comparison areas do not exist
Lesson 3:“Virgin” comparison areas do not exist
  • Simultaneous implementation of multiple programs
  • Separate, uncoordinated, inefficient evaluations, if any

Mozambique

lesson 4 there are no shortcuts for mortality measurement at least not yet
Lesson 4: There are no shortcuts for mortality measurement (at least not yet)
  • Capturing a 25% difference-in-difference for rates of child mortality in a two-year period requires a survey of ≈ 12,300 households in each group*
  • Promises of measuring declines in 1 year using survival analysis or other techniques still require these prohibitively large sample sizes, plus detailed info on age of death
  • CI work on “real-time” mortality monitoring will assess the validity of alternative methods, but in first trials require validation against a gold standard

*based on Malawi; sample sizes will increase as mortality rates decrease, e.g. in Ghana

contributors
Contributors

In-country partners

IIP-JHU

Burkina Faso: ISSP, INSP

Ghana: Noguchi Institute, University of Ghana

Malawi: NSO, CSR, Department of Economics, University of Malawi

Mali: CREDOS

Mozambique: Eduardo Mondlane University

Agbessi Amouzou, Abdullah Baqui, Robert Black, Jennifer Bryce, Kate Gilroy, Elizabeth Hazel, Gareth Jones, Marjorie Opuni, Jeremy Schiefen, Cesar Victora, Damian Walker

what is a national evaluation platform nep
What is a national evaluation platform (NEP)?
  • District-level databases covering the entire country
  • Containing standard information on:
    • Inputs (partners, programs, budget allocations, infrastructure)
    • Processes/outputs (DHMT plans, ongoing training, supervision, campaigns, community participation, financing schemes such as conditional cash transfers)
    • Outcomes (availability of commodities, quality of care measures, human resources, coverage)
    • Impact (mortality, nutritional status)
    • Contextual factors (demographics, poverty, migration)
  • Permits national-level evaluations of multiple simultaneous programs
neps a common evaluation framework
NEPs: A common evaluation framework
  • Common principles (with IHP+, Countdown, etc.)
  • Standard indicators
  • Broad acceptance
neps sound evaluation principles
NEPs: Sound evaluation principles
  • In-country evaluation counterparts
    • Local expertise, able to provide continuing evaluation research support to the MOH
    • Continuity of inputs from evaluation team; cross-country network of investigators
  • Linked “independence”
    • Investigators not involved in implementation of MNCH activities
    • Regular exchange with in-country implementation team
    • Ongoing activity; not one-off approach
  • Attribution by approach
    • Documentation of all contributions
    • Comparison of accelerated approach with “routine” approach
what types of questions can an nep answer
What types of questions can an NEP answer?
  • Are programs being deployed where need is greatest?
  • Is implementation strong enough to have an impact?
  • Did programs increase coverage?
  • Was coverage associated with impact?
  • How equitable are the programs?
  • How much did programs cost?
how can the moh and partners use the platform
How can the MOH and partnersuse the platform?
  • Which approaches or combinations are contributing to rapid scale-up?
  • Are some districts more efficient than others? Why?
  • Are changes in epidemiology (e.g., due to IRS) reflected in reallocation of resources in district plans?

To learn from well-performing districts and guide those doing less well

why should you consider a national platform approach or not
Why should you consider a national platform approach (or not) ?

Advantages

Limitations

Observational design (but no other alternative may be possible)

Cost, particularly due to large size of surveys (!But cheaper than many standalone surveys!)

Requires transparency and collaboration by multiple programs and agencies

  • Adapted to current reality of multiple simultaneous programs/interventions and partners
  • Flexible design allows for changes in implementation
  • Can be used to evaluate multiple programs (child survival, HIV, malaria, maternal health, etc.)
  • Supports country ownership and capacity building
slide37

Thank you

Further details at www.jhsph.edu/iip

and

www.cherg.org