Secretaría de Programas Sanitarios

1. Secretaría de Programas Sanitarios Provincial Maternal -Child Health Investment Project Joint Health Results Based Financing (HRBF) and Spanish Impact Evaluation Fund (SIEF) workshop

2. Plan Nacer in the Argentinean Health System Final outcomes targeted

3. Plan Nacer Argentina Plan Nacer Argentina 1st Phase since 2005 2nd Phase since late 2007

4. Argentina's health systemorganization * Estimates based on Permanent Household Survey (PHS) - INDEC

5. Plan Nacer’s Share Target Population: Children under six and pregnant women, until the 45th day after delivery, who don’t have health insurance (other than the public sector budget)

6. Final outcomes being targeted Contribution to the reduction of inequality in health outcomes Contribution to the The Millennium Development Goals Evolution of infant mortality rates in Argentina Expressed as a rate per 1,000 live births Evolution of Infant Mortality Gini Coefficient Argentina 1990-2006

7. The National Government have embarked on a new kind of relationship with the Provinces to coordinate efforts in health provision To strengthen the Public Health Subsystem in particular the Primary Healthcare Network To increase accessibility and improve quality of health care (effective coverage) To Contribute to the reduction of maternal and infant morbimortality rates Plan Nacer´s Main Goals

8. Coverage Rate Evolution by Region NE and NW Regions

9. Beneficiaries by region / Investment : NOA 274.253Sept ´08 359.189Target NEA 249.655Sept ´08 307.455Target CUYO 73.974Sept ´08 188.660Target CENTRO 269.657 Sept ´08 1.053.288Target Total Investment 1st PhaseUSD135.8 MM (70% used) since 2005 2nd PhaseUSD300 MM (5% used) since late 2007 Total Beneficiaries897.219 - Sept-09 1.4 MM have been enrolled in the Plan Nacer since de begining of de Program PATAGONIA 29.680 Sept ´08 76.035 Target

10. Health Providers by Region– August 2008 NOA 1.054 NEA 813 CUYO 490 CENTRO 1.351 Total Providers with Agreement 4.061 PATAGONIA 353

11. Principal – Agent Framework Main Actors of the strategy

12. Plan Nacer’s Payment Mechanism 60% Enrollment National Level Capitation split in 2 Result Based-Financing (RBF) 40% Health Outcomes (Tracers) Provincial Level Fee for Service Providers Use of nomenclature (Primary Health care Practices) Target Population Change in health outcomes

13. Theoretical Framework Principal-Agent Relationships with asymmetric information predicts opportunistic behavior NATION (PRINCIPAL) - PROVINCE (AGENT) PROVINCIAL LEVEL OPPORTUNISM • Funding crowding out of spending • Pre contract opportunism (hiding information when defining targets) • Post contact opportunism (shirking effort & hiding information to avoid higher targets in future periods) • PROVINCE (PRINCIPAL) - HEALTH PROVIDER (AGENT) • HEALTH PROVIDER LEVEL OPPORTUNISM • Misuse of practices • Overbilling of high prices practices (delivery) • Potential fraudulent overbilling

14. Theoretical Framework Principal-Agent Relationships with asymmetric information predicts opportunistic behavior • PROJECT RESPONSES • Change in the results’ payment scheme • Stakeholder approach to monitor and put credible monitoring threats (coordinate different principals guide in the provider’s effort), it includes beneficiaries empowerment: Social Control • Monetary sanctions firmly applied • Strengthening and directing Auditing Activities (External Concurrent Auditing and Internal Auditing and Supervision, National and provincial level auditing organisms)

15. % Trans. Mensual Base 5% 4% 3% 2% 1% 0% LS ij Change in the incentives’payment scheme Incentives payment function From: All-or-nothing To: Flexible or continuous (3 thresholds scheme) Thresholds rule has better incentive properties in terms of avoiding opportunistic behavior

16. The need to define a model for institutional change

17. New Paradigm in Public Management New Model in Public Management Bureaucratic Model Rule fulfilment Incentives Central Intervention Decentralized responsibility Focus on inputs Focus on the Performance Focuson implementation Focus on impact Focus on formal fulfilment Focus on user performance and satisfaction Transparency and Social Oversight Closed informationsystem

18. Specific Health Goals: Plan Nacer Tracers

19. Funds Transfers – August 2008 Total Transfers 1st Phase since 2005 USD 60,261,148 NOA USD 31,421,935 Total Transfers 2nd Phase since late 2007 USD 11,994,026 NEA CENTRO USD 28,839,214 USD 8,417,679 CUYO PATAGONIA USD 2,714,473 USD 811,873

20. Nomenclature’s Practices

21. Nomenclature’s Practices Group and Subgroup practices: Percentage Structure August- 2008

22. Inscription Monitoring by Province DepartmentsProvince: Misiones Inscription Coverage Rate ranged by Infant Mortality Rate August 2008

23. Tracer I Evolution: Early Detection of Pregnant Women Evolution Tracer I: Early Detection of Pregnant womenNE - NW - 1 st Phase

24. Evolution Tracer II: Effectiveness of Childbirth and Neo - Natal Care NE - NW – 1st Phase 98,9% 100% 97,2% 97,2% 90,8% 89,0% 87,7% 87,8% 87,9% 80% 74,3% 72,8% 72,5% 71,9% 69,8% 70,5% 69,2% 66,5% 68,8% 60% 52,8% 50,9% 48,8% 40% 39,4% 36,6% 36,5% 33,7% 30,5% 25,3% 24,0% 20,3% 20% 15,0% 12,2% 8,5% 4,6% 0% II-2005 III-2005 I-2006 II-2006 III-2006 I-2007 II-2007 III-2007 I-2008 II-2008 III-2008* Average NE Average 1 st Phase Average NW Projection III - 2008*: Tracer II Evolution: Effectivenss of Childbirth and Neo Natal Care

25. Tracers I four month period-2008 Department I By Dep II By Dep III By Dep IV By Dep Calamuchita 8 1,6% 58 1,5% 58 1,6% 29 1,0% Capital 80 15,8% 1530 40,9% 1389 39,3% 1341 46,9% Colón 36 7,1% 187 5,0% 184 5,2% 118 4,1% Cruz del Eje 43 8,5% 198 5,3% 191 5,4% 183 6,4% General Roca 3 0,6% 19 0,5% 21 0,6% 18 0,6% General San Martín 32 6,3% 185 4,9% 168 4,8% 86 3,0% Ischilín 6 1,2% 44 1,2% 43 1,2% 5 0,2% Juárez Celman 8 1,6% 37 1,0% 41 1,2% 36 1,3% Marcos Juárez 7 1,4% 26 0,7% 24 0,7% 13 0,5% Minas 1 0,2% 2 0,1% 2 0,1% 1 0,0% Pocho 0 0,0% 0 0,0% 0 0,0% 0 0,0% Pte. Roque Sáenz Peña 4 0,8% 41 1,1% 41 1,2% 4 0,1% Punilla 71 14,0% 269 7,2% 254 7,2% 162 5,7% Río Cuarto 38 7,5% 512 13,7% 474 13,4% 479 16,7% Río Primero 13 2,6% 5 0,1% 10 0,3% 2 0,1% Río Seco 0 0,0% 0 0,0% 0 0,0% 0 0,0% Río Segundo 14 2,8% 33 0,9% 32 0,9% 27 0,9% San Alberto 6 1,2% 15 0,4% 14 0,4% 13 0,5% San Javier 9 1,8% 114 3,0% 112 3,2% 95 3,3% San Justo 42 8,3% 158 4,2% 172 4,9% 139 4,9% Santa María 10 2,0% 136 3,6% 128 3,6% 4 0,1% Sobremonte 7 1,4% 0 0,0% 1 0,0% 1 0,0% Tercero Arriba 4 0,8% 60 1,6% 63 1,8% 0 0,0% Totoral 6 1,2% 0 0,0% 0 0,0% 0 0,0% Tulumba 2 0,4% 1 0,0% 1 0,0% 1 0,0% Unión 57 11,2% 115 3,1% 111 3,1% 103 3,6% Sin Especificar 0 0,0% 0 0,0% 0 0,0% 0 0,0% Total Córdoba 507 100,0% 3.745 100,0% 3.534 100,0% 2.860 100,0% Tracers Monitoring by Province DepartmentsProvince: Córdoba Tracers Analysis by Department(Every four months) Tracers I, II, III, IV are related with pregnancy and delivery

26. Use of Funds Monitoring by Province Province: Corrientes

27. A change in payment mechanism is mainly a change in the way the different actors relate with each other in a public policy This may bring important changes in how to achieve health outcomes In this change data accuracy play an important role Uncertainty and asymmetric information is the natural context we must deal with. Main Lessons

28. To encourage health teams to provide integral health care coverage Strengthening decentralized monitoring capabilities and steering role of the provinces Generalize the payment mechanism in terms of expenditures in the public health system The national goverment decided to include a high complexity practice in the Plan Nacer Nomenclature: Congenital Heart Disease Surgery Challenges

29. To measure the change in the Health Care Facilities’ behavior that provide Primary Health Care to eligible population when facing the new payment strategy introduced by the Plan Nacer. To measure the impact in coverage and quality of the health care services provided to pregnant women and children under 6 yeas of age To measure the change in Health outputs and outcomes in pregnant women and children under 6 yeas of age as a result of the introduction of the program Plan Nacer Impact Evaluation. Main Objectives

30. Plan Nacer and itsImpact Evaluation Strategy in time New Strategy to measure causality: Instrumental variables – Pilot Survey 3 Party TEAM … P.NACER+WB+CEOP:1st STRATEGY 1st Method: a Household Survey Baseline 2nd Phase Baseline 1st Phase I.E. Jun 05 Feb 06 Oct 06 Feb 07 Oct 07 Provinces start to Enroll All Provinces are included - Provinces start to Enroll Payment mechanisms start in 2nd Phase Provinces start Payment Mechanism Plan Nacer Jun 05 Jun 07 2005 2006 2007 2008 Dec 2008

31. We plan 3 moments in the evaluation process: Moment 0 Corresponds to the baseline measurement Moment 1 To measure the intermediate impacts of the program (scheduled for 2009/2010) Moment 2 To measure the final impacts of the program (scheduled for 2011/2012) Time line of the impact evaluation

32. Sampling design and selection of providers and eligible population in their catchment area (in the provinces adherent to the Plan Nacer). Sampling design and selection of a control group Design of questionnaires to gather data Data gathering at households, health facilities, medical records and specific measures of anemia, height and weight Data base organization of the collected data to permit the follow up of treatment and control groups Specific objectives

33. Sampling technique • Representative at provincial level Both requirements were fulfilled • Sample power to capture change • A multi-stage stratified sampling procedure was implemented to select localities, then providers and finally the eligible population in their catchment area

34. A first attempt to capture causality in 1st Phase • Propensity score matching between localities in provinces of the 1st Pahse and localities in sorrounding provincies : • Dependent variable: Y=1, treatment; Y = 0, not treatmet • We used available data at locality and facility levels as indipendent variables (sources: Population Census 2001 and Remediar Plan providers` list. Both constitute the sampling frame). • A probability (p) was estimated to pair localities. Locallities were paired and we had a balanced treatment and control groups Statistical tests were applied to evaluate the sample power

35. A first attempt to capture causality in 1st Phase • Multi-stage sampling and Propensity Score Matching allowed to select the localities’ sample • Within the localities sample a providers’ sample was selected (60 providers in each province) • For each provider a sample of 10 households was selected (eligible population with the latest child born in the previous 13 months

36. We planned a panel survey in moments 1 and 2 to obtain a difference in difference estimates of the change in main output and outcome indicators using a regression equation like the following one: y = 0 + 1d + 2t + 3(dt) + X + v Where: Y: is an output or outcome variable d: is 1 if the person is a beneficiary of the program and 0 if the person belongs to the control group t: is 1 in moments 1 or 2 and 0 in the baseline dt: is 1 if the person is beneficiary in moments 1 or 2 and 0 in other cases X: Vector of variables that affect and v: random variable. Difference in differences

37. Sample Sizes Planned Sample Sizes in Plan Nacer´s Impact Evaluation

38. Questionnaires contents

39. The national coverage of the program in 2007 required a new strategy to capture causality We chose a randomized promotion at locality level The randomized promotion design focuses on the effect of the program on beneficiaries. For outcomes two and three (impact in providers behavior), a number of alternatives are being considered New Strategy: Instrumental Variables

40. The promotion should satisfy the following sufficient and necessary conditions to qualify as a valid instrumental variable: A pilot survey was implemented to verify the fullfiment of the conditions and to compute sample power analysis (oct-2007) Randomized Promotion Design • Being correlated with program enrollment • Not being correlatedthe the error term (other features of the population) • Only affecting health results through Plan Nacer interventions

41. We want to learn more about the design of the providers incentives mechanism and its impact in the use of Primary Health Care practices that hits in health outcomes and prevent billing opportunism Promoted localities follow up to enhance the quality of the Instrumental Variable Adequate design of moments 1 and 2 to better capture the Plan Nacer´s impact Diffusion of survey results and knowledge Methodological challenges

42. Plan Nacer and itsImpact Evaluation prospective time-line Midterm Ealuation causality to learn about providers behaviour 1st Phase : Randomizad promotion for pregnant women 2nd Phase : Baseline Final Impact Evaluation 1stPhase Final Impact Evaluation 2nd Phase I.E. • 1st Phase: • Provinces fund 30% • of the USD 5 per • beneficiary • 2nd Phase: • Improve use of • practices • Monitoring use of • funds DIVULGE RESULTS End of 1st Phase End of 2nd Phase Plan Nacer 2009 2010 2011 2012 Dec 2012

44. Relationships PROVINCE - PROVIDER NATION - PROVINCE Payment for enrollment Payment for PHC practices billed Payment for health outcomes Decision of final use of funds Principal – Agent Framework A two level incentives system

45. Coverage Rate EvolutionNE Region