Improving Education Management in Madagascar ( agemad )

1. Results of an Impact Evaluation Improving Education Management in Madagascar(agemad) Préparée par Jee Peng et Cornelia Présentée par Paul RANDRIANIRINA ACCRA, GHANA Mai 11, 2010

2. Primary Education in Madagascar: Much Progress, but stillMany Challenges • Signs of progress: • Primarycompletion rate doubledfrom 35% in1999 to 71% in 2008 • Evidence of weaksector performance: • ½ of eachcohort of 1st graders does not finish the primary cycle; • Repetition rate stillhighat 18% in 2005 (30% in 2000) • Lowquality: in 2004-5 PASEC, average test score of 50% in Maths and Malagasy and 32% in French; deterioratedsince 1997-98 • Multiple systemic causes : • Inconsistencies in teacher allocation acrossschools; • Ineffective management of pedagogicalprocessesatschool and classroomlevels

3. Les défis de l’AGEMAD • Améliorer l’allocation des ressources entre les écoles. • Assurer que les ressources allouées sont transformées en résultats au niveau des élèves. • Identifier des interventions qui permettent de renforcer la gestion du système. • Tester les interventions et évaluer leur impact afin d’éclairer les choix sur des généralisations possibles

4. Une démarche en 4 étapes : 1. Identifier acteurs du système éducatif= ceux qui ont des responsabilités à assurer 2. Déterminer quelles tâches ils ont à accomplir 3. Elaborer des outils de travail pour qu’ils accomplissent ces tâches: outils (procédures, tableaux de bord, statistiques) rationalisés, adaptés , conçus et testés en collaboration avec les utilisateurs 4.Clarifier les conditions incontournablespour la bonne exécution du système de gestion.

5. Tighter Management to Improve Accountabilty Conceptual Intervention Framework & IE Design: • Workflow tools to clarify tasks and internal accountabilities; • Facilitation of meetings between school and community; • Better information flows within school and between school and community; • Structured training for teachers and school heads Leading to: • improvement in actors’ behavior through bottom up and top down accountability • better managed school • increased school quality • higher student learning

6. Key Questions for Policymakers • What is the impact of tighter management of processes on school functioning and student performance? • At what administrative level are management interventions the most effective (school, district or inspection level)?

7. Impact Evaluation Design (1) Method: Randomized experimental design over 2 school years Interventions: • Specify actors’ responsibilities & their mutual accountability processes through: • Management Tools and Guides for key tasks (e.g. pedagogical, administrative) • Training • Focus attention on results by clarifying goals through: • Report cards:School, district and inspection report cards • School meetings:Facilitated school meetings & development of school improvement plans based on school report cards

8. School & District Report Cards for Better Information Flow • Report cards for school directors, sub-district and district levels officers: • Complement the tools and processes • Draw attention to schooling outcomes • Include comparative data, allowing a school to compare its outcomes with those of other schools • Serve as basis for dialogue and accountability

9. 15 CISCO AGEMAD 15 CISCO CONTROL 89 ZAP AGEMAD 84 ZAP NON-AGEMAD 80 ZAP CONTROL 303 Schools CISCO AGEMAD TREATMENT 1 303 Schools CONTROL Impact Evaluation Design (2) 303 Schools AGEMAD TREATMENT 3 303 Schools ZAP AGEMAD TREATMENT 2

10. Collecting Data Actors’ Behavior (direct effects): • Questionnaire from impromptu school visits in 1,200 schools, with information for 4,000 teachers • Questionnaires for District and Community admin. level • Collection and analysis of tools used in 40 schools (850 tools) Schooling outcomes (indirect effects): • Test scores from standardized tests in 3 subjects • National year-end school census data: flow rates, repetition, CEPE pass rate Timeline: • 2 school years, 2005-2007 • Baseline survey/test and post-intervention survey/test

11. What tasks are deemed essential? • Teacher: • Takes daily roll call • Prepares daily lesson plan • Prepared bi-monthly lesson plans • Monitors student learning • Has tested pupils during the past two months • Helps lagging students • Discusses student learning issues with the director • School director: • Keeps a register of enrollments • Signs off on daily roll call • Analyzes student absences on a monthly or bi-monthly basis • Reviews pupils’ test results • Takes stock of teacher absences • Informs sub-district or district officer about teacher absences • Follows up with teachers on lesson planning

12. Results: Effects on Actors’ Behavior

13. Results: Effects on Schooling Outcomes

14. Policy Implications • Prioritize school-level actors • “Cascade” training model alone, as currently defined, doesn’t work • Though results are encouraging, better management essentially entails changing peoples’ behaviors, which takes time and effort • Mainstream IE results into MoE activities • Need a champion from the start • Need early involvement of a national team, with good technical support • Necessary to sustain change in actors’ attitudes & behaviors • Use existing structures and mechanisms for scale up: • Tools, guides and training modules integrated into teacher training • Tool distribution, training and facilitated school meetings funded through the local catalytic funds based on regional, district and school performance plans and needs • Develop leaders to drive change in management practices • Discussion underway on collaboration in leadership training between Madagascar MoE and partner organization in another country

15. Stay tuned…Publications forthcoming • Africa Human Development Working Paper Series «Améliorer la gestion de l'enseignement primaire à Madagascar - Résultats d'une expérimentation randomisée » • Journal Article undergoing peer review «Managing for results in primary education in Madagascar: Evaluating the impact of selected workflow interventions »

16. It takes a village… • Government commitment: • Stable counterpart team (15 staff from MoE with coordinator) • Partner commitment: • Financial and technical assistance from AFD (via two staff) • WB team lead by Jee-Peng Tan and Cornelia Jesse, consisting of Gérard Lassibille and Trang van Nguyen (with in-country field coordinators) • Local NGO Aide et Action to assist with training • Financing: WB, AFD, MoE, EFA-FTI (EPDF), Irish Aid, Ministry of Foreign Affairs of Norway • Timeline: 2004 – 2007 • Total number of people involved: 50

17. The Perils of Data Collection… Thank You!