Outline of Presentation

Zimbabwe Results Based Financing ProjectEvidence and LessonsBrigadier General Dr. Gwinji, MOHCCDr.Goverwa -Sibanda, MOHCC

Outline of Presentation • Key messages, strategic context and overview of RBF program • Brigadier General Dr. Gwinji, MOHCC • Preliminary results from impact evaluation and process evaluation • Dr. Sibanda, MOHCC

Main Takeaway Messages I • RBF is a health systems management tool intended to improve the efficiency of utilization of system inputs • RBF in Zimbabwe piloted in 2 front runner districts in July 2011, then scaled up to 16 additional districts in March 2012 • Contextual background: Dramatic improvements in MCH indicators witnessed throughout the country (MICS 2014) • Yet faster rates of improvement in RBF districts for key indicators • 13 percentage point improvement in the in-facility delivery rate • 12 percentage point improvement in post-natal care coverage • Significant improvement in the quality of ANC services • Not all indicators show relative improvement under RBF • No differential gain in coverage of ANC services • Small gain in use of modern contraceptives

Main Takeaway Messages II • Improvements in part due to • Team based incentives facilitating teamwork • Regular and structured supervision visits yielding feedback to improve performance • Enhanced community participation • Challenges to be addressed include • Facilities in remote areas or with small catchment populations • Capacity to fully operationalize the quality of care components • Demand side barriers related to religious and socio-economic factors

Country & Project Context During Design Phase [2010-2011] • Population: 13 million (2002 Census estimates) • Decline in public sector financing • Effect on management • supportive supervision • Increase in household out-of-pocket expenditures due to various forms of user-fees • Decline in outcomes & slow progress on some key health MDGs [MDGs 4 & 5]

RBF in the Zimbabwe Context • RBF aligned with and supports National Health Strategy and policy–equity in access to health services; • User fee removal (package of high impact services) • Rebuild the quality of care standards • Increase access to priority maternal, family planning and child health services • Strengthen the referral system (promotes appropriate care seeking at appropriate levels) • Decentralized service delivery and revitalized primary health care • Prioritized package of services directly linked to burden of disease for mothers, newborns and children under 5 • RBF used to operationalize GoZ Results-Based Management Strategy and Results-Based Budgeting Pilot

Three Project Components

Package of RBF Services Tetanus TT2+ ARVs to HIV+ preg. Women (PMTCT) Family planning short and long term methods High risk perinatal referrals Vitamin A supplementation Children fully immunized Growth monitoring, children < 5yrs Cure discharged acute malnutrition children < 5yrs (October 2012) OPD new consultations First ANC visit during the first 16 weeks of pregnancy (October 2012) Ante natal care 4 visits completed Post natal care 2 or more Normal deliveries HIV VCT in ANC Syphilis RPR test IPT (x2 doses) Normal deliveries in district hospital Deliveries with complications (caesareans excluded) and post partum complications Caesareans performed Family planning: Tuba Ligations Counter referral note arrives at RHC (October 2012) Rural Health Centers District Hospital

Data Quality and Verification • Verified data from health facility registers collected by Local Purchasing Units (LPUs) and entered in the RBF database is utilized for payments • Data flow integrated within national HMIS, no parallel system used • Quantity verification by the LPU undertaken every month • Quality verification by the DHE (for RHCs) and by PHE (for DH) undertaken every quarter • Client satisfaction performed by the CBOs every month and by the counter-verifier every quarter

Interaction between ProgrammeData and HMIS HMIS - DHIS 2 Routine Reporting - T-Series Entry and Submission into DHIS2 District Facility ≈ Updating HMIS with Verified Data Verification Data Accessible in Programme System: Same Platform Payment to Facility RBF Indicators Downloaded Programme Database HFOs Verified Data Entered Into System

RBF Financing & Coverage • RBF Funding • DFID and Government of Norway (US$35m) • Ministry of Finance: US$5 million per year from 2014 • US$ 28 million disbursed (including $5 million Government counterpart funding) • Population Coverage : 4,1 million • Geographic coverage : 18 rural and 2 low-income urban and peri-urban districts (Harare and Bulawayo) • July, 2011 to October, 2015

Governance & Institutional Arrangements MoHCW CORDAID Private Purchasing Agency (NPA) Contract National Steering Committee Policy and Supervision Provincial Health Executive Contract + Payment Policy and Supervision CORDAID Local Purchasing Unit External verification Payment District Health Executive Contract District Steering Committee Policy and Supervision Health Facilities and HCC (415) Payment Contract + Verification Tracing clients and client satisfaction Community Based Organisations Clients

Impact and Process Evaluations: Answering Policy Relevant Questions Process Monitoring and Evaluation (PME) (November 2013) Program Inception 2nd PME Round Planned for October 2014 Endline IE (TBD) Midline Impact Evaluation (Mar-Sept 2014) Baseline (2011) Technical Review (June 2012) Mid-Term Review (January 2013) Routine Performance Review (Quarterly) – Operational Data Technical Adjustments: Prices and Services Technical Modifications –clinical quality, streamlining verification, equity monitoring

Evaluation Design I The IE seeks to determine the causal impact of RBF on priority service utilization and related health indicators • Treatment:Facilities and patients residing in districts that introduce the RBF program • Comparison:Facilities and households in matched “business as usual” districts • Districts matched on various characteristics including: • Average catchment size of facility • Proportion of staff positions filled • Population rates over 2008 – 2010 of ANC coverage, in-facility delivery rates, immunization coverage

Participating Districts

Evaluation Design II Given the purposive selection of study districts, the evaluation must be quasi-experimental. Specifically, • A difference-in-difference (“diff-n-diff”) estimator between matched districts in treatment and control (16 in each arm) estimates program impact • To estimate actual impact: two years of program exposure (2012-2014) is contrasted with a two year period immediately (2008-2010) before the program

Data Sources I Household information – • Population representative surveys of health behavior, • including utilization, recall of procedures • health outcomes, • including anthropometry, satisfaction with care, knowledge • and mediating variables • socio-demographics • Baseline data for community and household utilized the 2011 DHS • Follow up data at community, household structured to replicate and supplement the earlier DHS • Yields a sample of ~2800 recent pregnancies/births

Data Sources II Facility survey – a comprehensive review of the structure, provision, and quality of care at clinic level • Instruments • Facility checklist • Health worker tool • Exit interview tool (ANC, child illness) • Direct observation (ANC, labor and delivery and child illness) • Chart audit* of routine and complicated delivery • Only in the follow up 180 facilities in 2011 baseline (the NIHFA) and 231 in follow up Technical support from USAIDand UNICEF was critical in this undertaking

Preliminary Results I • Data has just been collected and still being processed • However the preliminary results from both population and facility data are now available… • Overall it is a story of strong gains in select health indicators for the entire nation (consistent with the MICS results), with yet more rapid improvement in RBF districts • RBF gains in both the quantity and quality of care, but not for all prioritized indicators

Preliminary Results II • RBF led to gains in both the quantity and quality of care • A 13 percentage point increase in the in-facility delivery rate • A 12 percentage point increase in post-natal care coverage • More women receiving full package of ANC services including urine tests, blood tests, tetanus shots • But not for all prioritized indicators • Little change in ANC coverage and contraceptive use– baseline rates already high

Dif-n-dif in Practice: Receipt of Urine Test during ANC Care Start of RBF

Summary Regression Measures of Impact Most impacts measured in proportional terms. Statistically significant results with in red.

Complementary Results from Facility Level Data Patient exit interviews • ANC interviews (n=1105) indicate significant improvements in care processes such as measured abdomen and urine sample • Child health interviews (n=1612) indicate significant improvement in measurement and growth monitoring • Client satisfaction is also higher (although at marginal significance, p=.128) Facility measures • Improvements in select facility conditions after RBF: • 8 percentage point increase in availability of bio-med waste disposal • Increases in supervision and community involvement after RBF: • Increase in number of HCC meetings (2.7 more per year) • Increase in external assessments of staff (2.4 more per year) • 34 percentage point increase in presence of facility work plan

Moving Beyond the ‘Black Box’ Program RBF Investments Outcomes

Process Monitoring & EvaluationRationale • Need to go beyond numbers • Capture rich experiences and lessons from frontlines of PBF implementation & context • Community engagement and support (HCC) • Geographic influences (supervision aspect, cross-catchment area patient movement) • Health facility management skills and dynamism • Extent of mentorship and clinical supervision by district/province • Explain contextual factors that matter the most & account for variation in provider performance • Supporting impact evaluation and not substituting it –better understanding of the our intervention

Select Findings from Qualitative Study Providers at high performing health facilities (HPF) were reported to be highly motivated. • Regular and well structured supportive supervision improved the capacity to earn more subsidies e.g one (HPF) had an increase in subsidies of 53% to 94% between the first and second quarters. • Team work improved communication ...we are now working as a team to ensure that the T5 (data compilation form) is correct and accurate and we are able to send it to the district in time. We are motivated to work as we get paid for our effort. --( HF Staff Member) • Higher client satisfaction attracted more patients (patient choice)

Challenges Affecting HF Performance When low performance is found, facilities attribute reasons to: • Smaller catchment populations & at times geographic remoteness • Irregular supportive supervision • Socio-economic and religious factors in the population

Policy Actions Considered & Next Steps • Revisit the remoteness criteria & facility catchment issue • RBF performance payment calculation formula being reviewed (higher remoteness bonus) • Content & structure of quality supervision checklists • Written feedback by district managers mandatory • Modified supervision checklist – more process measures of clinical care introduced • District Health Executive supervision contract revisited – ensure incentive to regularly supervise remote facilities • Integrate RBF into HR management

Thank You!

Extra slides

Policy Relevant Questions - Zimbabwe • What is the impact of RBF on maternal and child health services utilization and outcomes? • What is the impact of RBF (counter verification) on health management information system (HMIS) and on supportive supervision? • What is the effect of RBF on community participation and social determinants of health? • What is the effect of RBF on health workers’ attitude, job satisfaction, retention and attrition, etc? • What is the effect of RBF on patient/client satisfaction and in health seeking behavior?

Evaluation Design II Given the purposive selection of study districts, the evaluation must be quasi-experimental. Specifically, • A difference-in-difference (“diff-n-diff”) estimator between matched districts in treatment and control (16 in each arm) estimates program impact • In a diff-n-diff estimator, the observed trends in outcomes for the comparison districts stand for what would have happened in the RBF districts if not for the RBF program • The validity of this approach can be assessed by comparing pre-RBF trends of same outcomes in RBF and comparison districts before RBF program • To estimate actual impact: two years of program exposure (2012-2014) will be contrasted with a two year period immediately (2008-2010) before the program

Dif-n-dif in Practice: In-facility Delivery Rate Start of RBF

Dif-n-dif in practice: Rate of ANC coverage (any ANC) After introduction of RBF

Moving beyond the ‘Black Box’ II • Key areas PME assessed • Factors affecting health provider performance • Factors influencing changes on the demand (community) side • Compliance of the RBF stakeholders to the project’s implementation guidelines.

PME Study sites Binga, Chipinge, Kariba, Mazowe and Zvishavane.

Sampling & Approach • Selected three facilities in each of five districts • Sequential mixed method deployed • Participants selected included mothers of reproductive age, male community members, influential leaders, health center committee members, district management team, as well as clinic health staff

Summary of Qualitative Evidence • RBF: • Strengthens relationships between health care providers & communities and improves access to services at community level (e.g. community ambulance) • Fosters innovations and entrepreneurship among health workers • Improves district managers supportive supervision • Improves health facility infrastructure (service delivery environment) • Improves staff morale and promotes team work

Challenges (cont.) • Staff shortage “When one of us attends a workshop only one person is left to deal with registers and the workload is huge and most of the mistakes we make are due to fatigue. We are now losing patients to other institutions…”HF staff “There are a number of clinics surrounding therefore there is competition for example X clinic is GVT owned and well equipped, has drugs and well-staffed so patients prefer going there than spending more time in long queues here where there is staff shortage”. -- HF Staff Member • Shortage of drugs e.g. vaccines • Inadequate infrastructure

Outline of Presentation