Pay for performance initiatives a growing trend, but are they effective?

1. Pay for performance initiatives a growing trend, but are they effective? Josephine Borghi London School of Hygiene & Tropical Medicine Ifakara Health Institute

2. Objective • Provide an overview of the challenges in measuring the effectiveness of PBF • PBF refers to a diverse set of interventions • Implementation may not match the ideal ‘vision’ • Need a better understanding of the health system impact of PBF • What kinds of evaluation should we be doing? • What is feasible and politically acceptable?

3. Performance based financing – A range of applications • Based on who is the recipient of the bonus payment • Donor to government (aid modalities) • Within the public sector (facilities or districts / regions) • Government/donor to non-state provider • Health workers • Process or outcome indicator/target • Payment method and size • When we say PBF is effective what do we mean?

4. Implementation – potential constraints in low income settings • Design constraints • Complexity versus simplicity – designing a simple system which is effective? • Roll out constraints • Logistics • Timely availability of funds • Appropriate management of information • Training of managers and health workers • Monitoring constraints • Need for external validation - costly • Lack of reliable HMIS data

5. Tanzanian experience – vision versus reality

6. PBF: Impact on the Health System? Potential Risks Understanding the processes of change as well as the impact on health outcomes

7. Potential risks • Impact on health worker behaviour • Supplier induced demand – unnecessary service delivery • Quantity over quality • Opportunity cost – focus on targeted conditions • Equity effects • Patients: Focus on easy to reach groups • Geographic: areas where health system stronger may have advantage • Health worker migration

8. Need for improved evaluation design

9. Key Components of an Evaluation What is the status of implementation? How does it vary by area? Process Evaluation Need for qualitative assessment to determine causal pathway What facilitates and impedes implementation? What changes take place that result in the observed impacts? What are the ‘positive’ effects of PBF on coverage of health services? What are the health benefits of PBF? Impact Evaluation Are there any unintended consequences of PBF? Economic Evaluation Is PBF a good use of resources?

10. An evaluation framework based on Tanzania

11. Positive Effects Negative Effects Re-allocate resources Increased motivation and trust Damage intrinsic motivators Improve quality of care / increased patient satisfaction Coercive strategies to increase utilisation Mis-reporting performance Increased utilisation of targeted health services Crowding out of non-targeted health services / reduced quality of care Reduced maternal and newborn mortality Recommendations for improved performance

12. Impact Evaluation Design Options Strongest evidence of impact Randomised Control Probability design Political acceptability Non-randomised Control Need to control for inherent differences Plausibility design Interrupted time series Need good historical data Adequacy design Historical or no control Weakest evidence of impact

13. Practical challenges and need for innovative thinking • Political resistance to controls • Randomisation may not be possible • Randomisation of performance contracts (Rwanda) • In absence of control or historical data importance of documenting the process

14. Summary • Need for better understanding of which aspects of PBF are effective and pre-conditions for their effectiveness • What is implemented may not match what was intended, need for simple and saleable models • Some unanswered questions on health system impact • Need for process evaluations and more impact evaluations within realm of political feasibility