Operationalizing ie case study example textbooks and teacher training in sierra leone
Sponsored Links
This presentation is the property of its rightful owner.
1 / 17

Operationalizing IE: Case Study example: Textbooks and Teacher Training in Sierra Leone PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

Operationalizing IE: Case Study example: Textbooks and Teacher Training in Sierra Leone. APEIE Workshop, May 10-14 2010. Operational Key Steps. IE Design – methodological issues. Available data and required data Identify control group and treatment group Randomization Transparency

Download Presentation

Operationalizing IE: Case Study example: Textbooks and Teacher Training in Sierra Leone

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Operationalizing IE:Case Study example: Textbooks and Teacher Training in Sierra Leone

APEIE Workshop, May 10-14 2010

Operational Key Steps

IE Design – methodological issues

  • Available data and required data

  • Identify control group and treatment group

    • Randomization

    • Transparency

  • Sampling and sample sizes

    • Power calculations

    • Budget and access constraints

  • Identify outcome indicators

Data Instruments

  • Utilize survey experts

  • Who do you survey?

    • Households, community, facilities, government…

  • Pilot instruments

    • Clear questions and coding

    • Includes all information needed for analysis and covariates

Intervention Roll Out

  • Possible contamination of evaluation results!

    • Intervention benefits rolled out to control and treatment schools  evaluation compromised

    • Intervention or treatment schools only receive additional benefit  evaluation compromised

  • Monitor roll out

    • Train implementers – they need to know why!

    • Hire supervisors

    • Continue supervision throughout roll out


  • In June 2007, the World Bank and a team from the Sierra Leone Ministry of Education and Sports (MOES) met to rigorously evaluate the impact of two pilot studies on student performances.

  • Two questions

    • What is the effect of textbooks on student performance?

    • What is the effect of teacher training on student performances?

How do we design the study to get rigorous results?

  • Random Selection

    • 5 districts in Sierra Leone

      • Kailahun, Kambia, Pujehun, Western Rural, Western Urban

    • Neediest schools in district (using EMIS data)

      • Student to teacher ratio, presence of latrine, school construction

      • 360 schools total

  • Random Assignment

    • Comparison schools: receive regular allotment of books

    • Books schools: receive additional books

    • Books and training schools: receive additional books + teacher training


Sept: WB and MOE agree on pilot

Oct: Met with local councilors to select districts


Nov: Baseline data collection and teacher training manual developed

Dec: Roll out of textbooks and teacher training

Nov: Mid-term data collection




June: Baseline data collection

June: Roll out of textbooks


NO teacher training up to this point

Dec: Mid-term data collection

What happened?

  • Met with several challenges on the ground that caused significant delays in the project:

    • Textbook distribution

    • Teacher Training

    • Data Collection

Challenge 1: Textbook Distribution

  • Target date: December 2007

  • Actual date: June 2008

  • Number of books:

    • MOES accidentally distributed original books, so had to use Sababu books

    • had fewer books than project initially called for

  • Communication – information not filtering down

    • Sababu not properly informed

    • Service Providers unwilling/unable to change distribution

    • Headteachers in comparison schools unaware of project

Challenge 1: Textbook Distribution continued

  • Textbook distribution: June 2008

  • Data collection: December 2008

  • Consequences:

    • Not enough books distributed in some districts to make a statistical comparison

    • Distribution happened very late and at a bad time (right before summer vacation)

      • Allowing for only a small window for students to utilize the extra textbooks

    • Some Comparison schools received books

Challenge 2: Teacher Training

  • Target date: December 2007

  • Actual date: has not yet occurred

  • Time and budget constraints

  • The ‘+’ part of the ‘intervention +’ so took a back seat to the textbook distribution

Challenge 3: Mid-term Data Collection

  • Target date: November 2008

  • Actual date: December 2009

  • Delayed funding disbursement

  • Non-existent schools (poor quality of initial data)

  • Headteachers unaware of project – not willing for enumerators to collect data


  • Back up list of schools on hand for enumerators

  • Better communication with all levels of organization involved

  • Constant project sensitization

  • Greater monitoring of distribution

  • Inclusion of Process Surveys

  • Allow for sufficient time for procurement and disbursement

Next Steps

  • Analyze mid-term data

    • Was there a significant effect due to textbooks?

    • Has education in Sierra Leone improved in the past 2 years?

    • Where in the textbook distribution are there major informational stop gaps?

  • Move forward on teacher training

  • Write up findings and circulate report


  • Impact evaluations a useful tool for policy considerations

  • But need to understand every process involved in the impact evaluation

    • Textbook distribution, not just textbooks

  • Study can be ineffective if you don’t have full buy-in and understanding at all levels of government, all service providers, and headteachers:





  • Login