1 / 18

November 2013

The role of civil society to strengthen accountability and to build evaluation systems The experience of Fundación IDEA. November 2013. Content. About Fundación IDEA The Indigenous Education Gap Index PEC FIDE ( School based management program ): evaluation method and results

mateja
Download Presentation

November 2013

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The role of civil societytostrengthenaccountability and tobuildevaluationsystems Theexperience of Fundación IDEA November 2013

  2. Content About Fundación IDEA TheIndigenousEducation Gap Index PEC FIDE (Schoolbasedmanagementprogram): evaluationmethod and results Civil society’s role in M&E M&Eand policy-making

  3. Fundación IDEA is a non-partisanthinktankthatprovidesrigorousanalysis and soundsolutions. • Ourvision: informedpolicydecisions • Rigorousanalysis and soundsolutions • 8 years of experience • Projects in Mexico, Guatemala, Colombia, USA, Italy, Peru • Impartial and non-partisan • Workon a variety of topics: education, economicdevelopment… 3

  4. Fundación IDEA workson a variety of topics. Fundación IDEA has developedexpertiseoneducation. • “Evaluación Transversal: políticas y programas para el desarrollo de la juventud” • “Estrategia nacional para la vivienda sustentable” • “¿Qué hacer con la diabetes?” • “¿Quiénes son los emprendedores innovadores mexicanos?” • “Propuesta de programa de inclusión laboral para jóvenes” • “Estudio sobre calidad docente en zonas rurales e indígenas en México” • “Índice compuesto de eficacia de los sistemas escolares” • “Índice de equidad educativa indígena” • “Evaluación de impacto PEC-FIDE (Fortalecimiento en inversión directa a escuelas)”

  5. Fundación IDEA foundthatindigenouspeople in Mexicohaveachieved 66% of theeducationalachievementcomparedto non-indigenous. • Thereis a gap of 34%. • Implications of thestudy: • Showingthe gap topushfortargetedpolicy.

  6. Relevant SBM indicators Impactevaluation of the federal School-Based Management Program PEC FIDE (PROGRAMA ESCUELAS DE CALIDAD-FORTALECIMIENTO EN INVERSIÓN DIRECTA A ESCUELAS) • Pilotvariation of program • Schoolzoneapproach • Focusonverylow-incomeneighborhoods • Focuson at riskstudents • Focuson training (shiftfrominsfrastructure) • More money per schoolthanaverage PEC

  7. Fundación IDEA wastheexternalevaluator. PEC FIDE • Impartial, rigorousapproach, publishresults; • Usefulness of recommendations; • Implicationsof evaluation: designprogramsbasedonevidence, keepprogramsthatwork, improveprogramsthat do notwork. • Evaluationcontext: • 50% of schoolswerebeneficiaries (selectionbias). • Infoavailable: annualschoolcensus; ENLACE test; populationcensus. • Methodology(mixedmethods): • quantitative: diff-in-diff w/ propensity score matching. • qualitative: interviews, participantobservation and documentanalysis in 10 schools.

  8. Theevaluationshowed positive impactfor PEC FIDE (2008-09) in graduationrates. Resultados de Skoufias y Shapiro (2006a) Resultados de Murnane, Willett y Cárdenas (2006) Resultados de Abreu y Santibañez (2010) Tasa de aprobación Tasa de aprobación* Tasa de aprobación 1.6 pp 0.24 pp 0.11 pp Efectos de un ciclo escolar Efectos después de tres ciclos es escolares Efectos de un ciclo escolar vs PEC FIDE PEC Otherresults: Approx 20 more points in ENLACE test (readingcomprehension); approx 1/3 of standard deviation. Approx -4 percentagepoints of students at insufficientlevel in ENLACE test.

  9. Theevaluationshowedimpactonreadingcomprehension at theelementaryschoollevel (ENLACE test). 19.5 puntos de ENLACE/año 12.8 puntos de ENLACE/año Inicio de ciclo 2008-2009 Fin de ciclo 2008-2009

  10. Fundación IDEA presentedresultstoadministrators of theprogram. • Qualitativeanalysisshowedincreasedmotivation. • Investmentfocusedoninfraestructure, hardshiftto training. • Limitedparticipation: howtopromote more and betterparticipation? • Theimportance of targetingneighborhoods? • Importanceonschoolzoneprogramdesign? • Targetingstudents at risk?

  11. Content About Fundación IDEA TheIndigenousEducation Gap Index PEC FIDE (Schoolbasedmanagementprogram): evaluationmethod and results Civil society’s role in M&E M&E and policy-making

  12. Civil societyneedstopromoteanevidencebasedpolicy culture. Publicpolicybasedonevidence Programsmay be implementedbecause of: • Evidence • Experience • Ideology • Politics • Budget • Technicalcapacity

  13. Evaluationshould be used as a toolfordecisionmaking and efficientspending. We are experiencing a paradigmshift in M&E. • Evaluationis a toolforaccountability and transparency - programs use publicresources and so societyneedstoknowifobjectives are accomplished and ifprograms are effective • Evaluation as a learningprocess - toimprovedesign and operation of programs - togenerateknowledgeaboutcomplexproblems and aboutinterventionsthatwork - toknowif a programworksornot - “bangforthebuck”

  14. M&E, ifappropriatelyused, can improvehowprograms are managed and theoutcomesthey can have. The use of M&E byorganizations M&E isusedfordecisonmaking Good consumer and capacity to perform M&E activities Some M&E activities No M&E t

  15. There are somemisconceptionson M&E practice. Types of evaluation: • Needsassessment • Programtheory (theory of change) • Designevaluation • Implementationorprocessevaluation • Results (change) • Impact (causality) • Efficiency (CBA, costeffectiveness)

  16. Impactevaluation can changepolicyifimplementedbasedon a goodunderstanding of M&E practice. Key lessonsforcapacitybuilding: • ValidCounterfactual - understandwhentoevaluateimpact and whennot - manyorganizations decide toevaluatebefore-afteror compare beneficiarieswith non-beneficiaries: notalwaysgood use of resources • Theory of Change - importantforunderstaningwhattheorganizationisdoing and howthey are measuringit - sometimesthereis no clarity/consensus of thepurpose of a program amongstakeholders • Impact • Objectives • Inputs • Products • Intermediateresults

  17. Thankyou

More Related