The Ishraq Program in Egypt

1. The Ishraq Program in Egypt Ghada Barsoum Population Council PEP Meeting June 2008

2. Ishraq: Basic Facts • Targets out-of-school adolescents (12-15) • In rural Upper Egypt: Lowest human development indicators: 26% of girls receive no schooling (ELMPS06); unpaid family enterprise workers; early marriage and childbearing; FGM • Intervention lasts for 20 months • Girls meet 4 times weekly for 3 hours

3. Ishraq: Basic Facts • Four Int’l NGOs: Save the Children; CEDPA (the Center for Development and Population Activities); Caritas; Population Council • Government partner with an increasing role: National Council of Youth (NCY) to allocate specific hours in youth centers to be used as “safe spaces” for girls • Local NGOs implementing, adapting the model and mobilizing communities on the village level

4. Ishraq: Basic Facts Three Components: • Literacy (Learn to be Free, Caritas): Girls sit for the literacy exam and are given the opportunity to be mainstreamed into the formal educational system (grade 7) – Program provides help in issuing birth certificates (required for school enrollment) • Life-Skills (RH, Basic: livelihood, nutrition, girls rights .. (CEDPA) • Sports and Recreation (PC) • Monitoring and evaluation (PC)

5. Intervention so far .. • Piloted in 2001 in 4 villages in Minia (N=275) • Expanded in 2004 to 6 villages in Minia (2nd rounds in 2 villages from the pilot stage) (N=336) • Expanded in 2006 to 5 villages in Beni Suef (N=289) • A new phase is beginning September 2008 with a new scale-up and –out plan

6. A New Phase of Vertical and Horizontal Scale up • Transfer of Program ownership to National Council of Youth (NCY) – the longstanding government partner • 50 villages (2500 girls) – new components (graduation funds, financial literacy, livelihood) • The role of the resource team envisioned : transfer of the tested innovative design; training (NGOs and NCY); resource mobilization; research and evaluation; advocacy dissemination; • THEN: Within 5 years, NCY would incorporate Ishraq in its budget and independently operate it.

7. Hence, the importance of rigorous analysis and thorough documentation of this phase

8. Analysis & Documentation of Earlier Phase (mainly descriptive) • 92% of those who sat for the literacy exam passed • Over half achieved an “Excellent” score • 66% (re-)entered the formal education system • as compared to a national average of 6% for those who pass literacy exam nationally Monitoring and follow-up • 98 graduates enrolled in Sept’06 in grade 7 • 22 graduates are currently in technical secondary education

9. Current proposal

10. LITERATURE REVIEW • To estimate the impact of a program we need to know the difference between: (1) the outcomes for participants if they participated in the program and (2) the outcomes had they not participated • The latter is not observed • Primary problem in identifying treatment effects is the estimation of that counterfactual

11. Selection Problem • A common proxy for the counterfactual: average outcomes of untreated units (e.g., a comparison or control group) • Works out in random social experiments • Problem:violation of non-randomness of participation frequently arises when the (to-be) treated and the untreated differ in (often unobservable) characteristics  selection issue • Selection bias problem results in mixing-up the treatment effect with pre-program differences.

12. Non-experimental (econometric) methods • Use non-experimental (econometric) methods where a comparison group with “comparable” characteristics is constructed. • Whole body of literature has grown and incorporated econometric advancements such as non-parametric techniques. Many attempts to compare the performance of different econometric estimates (e.g., Lalonde 1986 and the response it triggered). • Different methods rely on different assumptions : they would produce the same results only if there is no selection issue.

13. Techniques Used • Selection on observables: • Linear regression • Matching methods • Selection on unobservables • Instrumental variables • Longitudinal data analysis

14. 1. Selection on Observables: • Assumption: participation is determined by characteristics that are fully observable to the evaluator (X).  by statistically conditioning on these characteristics, the selection issue is resolved. • Linked to linear regression • Matching methods • Outcomes of an individual are matched with the outcomes of an individual in the comparison group that is most similar using non-parametric techniques. • Not impose linearity

15. 2. Selection on Unobservables • Instrumental variables • Affects participation but not outcomes • Ex: natural experiments, RDD, randomized experiment design. • Longitudinal data analysis • Difference-in-differences (fixed-effects): difference out the unobserved differences between the treated and the untreated by comparing the pre-post change of treated units with that of the untreated units. • Assumption: unobserved differences are time-invariant

16. 3. Combining longitudinal methods with other methods • For example, control variables can be added to the difference-in-differences estimator. • Difference-in-differences can be combined with matching methods by differencing the data first and then applying the matching method

17. Dropout Problem Two ways are employed in the literature • Re-interpret as the impact of the offer of the treatment rather than its receipt. • Adopt an estimate adjustment • Introduced by Bloom (1984). Requires availability of follow-up data for dropouts, full-participants and control group members • Bloom (1984) focused on “no-show” dropouts and a “partial-show” case in the context of multiple-treatment design. • Heckman, Smith and Taber (1998) focus on alternative treatment impact parameters in the case of partial treatment. They use an instrumental variable approach. • Heckman, Hohmann, Smith and Khoo (2000) analyze dropout cases where participants dropout to pursue better alternatives to the program (they also looked at the substitution problem where control group members seek other programs). They provide an estimate adjustment to deal with dropout and substitution.

18. DATA COLLECTION PLAN • Start with about 6 clusters of pre-selected villages that fulfill certain basic criteria. • Each cluster will initially consist of about 12 villages • 5 will be selected as intervention villages and 2 as controls • Target group: 12-15 out-of-school girls. • Mapping & listing • Intervention villages: all households in the intervention villages to identify households containing out-school adolescent girls in the 12-15 age range. • Control villages: depending on size, either the entire village or randomly selected segments.

19. DATA COLLECTION PLAN- CONTD • Surveys: • Baseline • Endline • 5 years after completion (?). • To sum up: info on eligible out-of-school girls in: • intervention villages (participants and non-participants) • control villages • Info on their households and “gatekeepers”.

20. DATA COLLECTION PLAN- CONTD • Examples of information to be collected • Household characteristics • Parental education, work • Owned assets • Individual characteristics: • Age • Schooling, including grade attainment, and repetition. • Participation in literacy programs. • Variables we plan to use as instruments

21. Data Collection Quality • Based on experience from the pilot phase and other surveys in Egypt, we expect: • High response rates • High follow-up rates. • Low attrition rates in control villages • Double-check measure: compare population figures from our survey to corresponding age-group population figures from the 2006 Census.

22. Recruitment • Two classes of 25 girls in each intervention village • Participation is voluntary • First-come first-served basis. We foresee that participation randomization will not be not feasible • Based on experience from the pilot stage, applicants arrived at different points in time. • Different size of pool of applicants in different villages  Randomizing the selection of intervention and control villages. Not randomizing participation within intervention villages  Self-selection issue

23. Examined Outcomes • Literacy • level of success in passing government-sponsored literacy tests and rates at which girls are mainstreamed back into preparatory (middle) schools. • Attitudes about marriage and childbearing: • Ideal age at marriage • Views about decision-making regarding marriage partners and timing of marriage. • Ideal family size and fertility intentions.

24. Examined Outcomes- CONTD • Knowledge about nutrition, hygiene, and reproductive health. • Attitudes about harmful traditional practices (e.g., FGM) • Social isolation, peer networks, and participation in group or community activities • Gender norms index

25. Foreseen Challenges • Self-selection into participation • Non-randomness of drop-out • Effect on attitudes

26. Self-Selection Sources • Project recruitment process • Household visits by promoters (pilot phase) • Geographical proximity • Friends and relatives • Girls that self-select into the program are more likely to have parents who are more motivated and committed to girls’ education

27. Non-random Dropout • Dropout more likely in the following cases • Early marriage • Child work • Domestic chores • Lower commitment to female education

28. PROPOSED METHODOLOGY • Adopt strategies corresponding to the different stages of the project implementation • Recruitment / Pre-program activities • Baseline data collection • Estimation strategies

29. I - Recruitment Activities • Improved Ishraq recruitment strategy • Visual advertisements • More training for promoters • Ishraq village committees • Including a local NGO for implementation • Ishraq workshop/registration event • Broader outreach • More information  less dropout • Keep track of who registered and form. • Pre-program trial period (?)

30. II- Randomization Strategies • Randomized assignment from the pool of registered girls may not work • Alternative randomization procedures we thought of • Randomly choose girls from baseline data • “Randomized encouragement” procedure

31. III – Data Collection • Measures to ensure data quality • Collection of variables to serve as instruments

32. IV – Estimation Strategies • Preliminary steps • Compare the baseline characteristics of control village and intervention village members • Compare characteristics of participants, waiting list members and non-participants. • Probit models for participation into Ishraq

33. Estimation Strategies - CONTD • Plan to use several models: • intent-to-treat • instrumental variable • difference-in-differences • Other specification / models / Specification tests are also discussed.

34. Intent-to-treat • outcomes of control village members are compared to intervention village members whether they were treated or not • the treatment explanatory variable = 0  if the girl resides in a control village = 1  if she resides in an intervention village • Advantage: no selection issue. • Disadvantage: effect of the treatment will be diluted as the model will include untreated girls when estimating the average treatment effect

35. Instrumental variable • Instrument that affects participation but not directly affect outcomes. • Distance between home and youth center (proxy for costs of participation). + Distance to the village center as an explanatory variable in both the participation and outcome equations to capture access to other services and the effect of the centrality of the household location. • Collect information on distance (or time) based on respondents’ answers and on GPS coordinates.

36. Difference- in-differences • Using a group dummy, period dummy (pre- or post- program) and an interaction of the group and period dummies. • Combine the differences- in differences with other methods in additional specifications: a “regression-adjusted” specification by adding observable characteristics. • Considering combining difference-in-differences with the propensity score method

37. Other Models/Specifications • Estimate the difference between the outcomes of treated girls and that of waiting list girls. • Good comparison group: equivalent in terms of motivation and valuation of girls’ education • Heckman and Hotz (1989) pre-program specification test • Rationale: prior to treatment, treatment dummies should not affect outcomes for the control or the to-be treated. • Null hypothesis: treatment variable coefficient is equal to zero when using baseline data. • Limitation: assume that if a model successfully adjusts for pre-program differences, treated and untreated units will also do so in a post-program period • Not widely used. • RDD on age (?)

38. Human subjects concerns • IRB: an oversight committee, whose mandate is to protect the rights and safety of the subjects of the Council’s research. • Confidentiality, minimal risk to participants and informed consent.

39. Timeline • Data collection in August 2008 • Intervention Sept 08 – April 10 • Endline data collection (expedited to fit schedule, using MCAPI): February 2010 • Draft report: June 2010