1 / 20

Preparing for Life Programme

Measuring Investment in Human Capital Formation: An Experimental Analysis of Early Life Outcomes Orla Doyle, Colm Harmon, James Heckman, Caitriona Logue, Seong Moon UCL, 25 th -26 th June 2012. Preparing for Life Programme.

carrington
Download Presentation

Preparing for Life Programme

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Measuring Investment in Human Capital Formation: An Experimental Analysis of Early Life Outcomes Orla Doyle, Colm Harmon, James Heckman, Caitriona Logue, Seong Moon UCL, 25th-26th June 2012

  2. Preparing for Life Programme • One of the first experimental early childhood intervention in Ireland • Funded by Irish Government (DCYA) & The Atlantic Philanthropies • Community-led initiative operated by Northside Partnership in Dublin • ~ 6,400 inhabitants • 33% dependent on social welfare • 60% live in social housing (>3 times national average) • 47% lone mothers (29% national average) • 16% unemployed (3 times national average) • 66% early school leavers (38% national average) • 5% third level education (29% national average) Source: Census (2006)- CSO • Belief within the community that children were lagging behind their peers

  3. Design of PFL Aim:Improve levels of school readiness by assisting parents in developing skills to prepare their children for school Bottom-up approach: community initiative involving 28 community groups, service providers, & local representatives Theoretical Framework: Grounded in several psychological theories of development including the theory ofhuman attachment, socio-ecological theory of development and social-learning theory Evaluation: Randomised Control Trial design

  4. Design of Preparing for Life

  5. PFL Evaluation Impact Evaluation • Data collection: Pre-intervention (baseline), 3 mnts (WASI), 6mths, 12mths, 18mths, 24mths, 3yrs, 3.5yrs, 4yrs • Informant: Mother is the primary informant, but also fathers, child, other independent data sources (birth records) Implementation Evaluation • Aim: Delve into the blackbox of programme effectiveness & evaluate fidelity to the PFL model • Data collection: • Implementation data on the Database Management System • Focus groups with participants • Semi-structured interviews with mentors/IO

  6. Recruitment • Eligibility Criteria: • Cohort of pregnant women residing in PFL catchment area between Jan 2008-August 2010 (32 months) • Includes preparious and non-preparious women • Recruitment: • Maternity hospital at first booking visit (b/w 12-26 weeks) • Within the local community • Population-based recruitment rate, based on all live births during the recruitment phase, was 52%

  7. ComputerizedRandomisation Procedure Unconditional probability randomisation strategy After informed consent obtained, mother ask to press the key to allocate her assignment condition Evaluation & Implementation team automatically received an email with the assigned PFL ID number and treatment condition Preserves the integrity of the procedure as no opportunity for recruiter to intentionally influence assignment 115 allocated to High treatment group 118 allocated to Low treatment group

  8. Baseline Data Collection • Conducted post randomisation, pre intervention • Baseline data wave completed in August, 2010 • 123 measures: Demographics & SES, health behaviour & pregnancy, parenting, social support, psychological assessments (maternal well-bring, personality traits, self-esteem, self efficacy, attachment, time preferences) • Interviews conducted • PFLHigh treatment group: 104 • PFLLow treatment group: 101 • Used to determine the effectiveness of the randomisation procedure

  9. Methodology • Permutation based hypothesis testing (Heckman et al. 2010) • Classical hypothesis tests unreliable when the sample size is small and the data are not normally distributed • Permutation test are distribution free, thus suitable in small samples • Based on the assumption of exchangeability between treatment conditions under the null hypothesis • Stepdown procedure (Romano & Wolf, 2005) • Ignoring the multiplicity of tests may lead to the rejection of “too many” null hypotheses • Test multiple hypotheses simultaneously by controlling overall error rates for vectors of hypotheses using the family-wise error rate (FWER) as a criterion • Less conservative & more powerful than other methods as takes account of statistical dependencies between tests

  10. Summary of Permutation Tests Examining Differences at Baseline • Randomisation worked!

  11. Permutation Tests Comparing Baseline Differences in Selected Family Socio-Demographics

  12. Analysis of 6 Month Outcome Data • Test for treatment effects across 8 main domains including 160 measures • Child development • Child health • Parenting • Home environment • Maternal health • Social support • Childcare & service use • Household factors & SES • 25 Step-down categories defined • 258 interviews conducted • PFLHigh treatment group: 84 • PFLLow treatment group: 90

  13. Attrition & Disengagement up to 6 months Baseline Characteristics Associated with Attrition • Official dropout between baseline - 6 months: • High treatment – 13% • Low Treatment – 6% • Disengagement between baseline - 6 months : • High treatment – 9% • Low Treatment – 10%

  14. Results for high & low treatment groups: Child development Notes:1 one-tailed (right-sided) p value from an individual permutation test with 1000 replications. 2 one-tailed (right-sided) p value from a Step Down permutation test with 1000 replications. * indicates the variable was reverse coded for the testing procedure. ‘ns’ indicates the variable is not statistically significant.

  15. Summary of Six Month Results

  16. Summary of 6 month results • Majority of the results for High V’s Low are in hypothesized direction • A few significant differences identified (14%) • In line with other home visiting programmes

  17. Additional Analysis • Interaction & Subgroup analysis • Gender, lone parents v partnered parents, first time v non first time parents, high versus low IQ mothers, high v low family risk • Programme appears to benefit high IQ mothers, families with multiple children & families with multiple risks • Analysis of engagement (treatment intensity) • On average, high treatment group received 14 home visits of ~1 hr in duration= total treatment = 14 hours • Higher maternal IQ & vulnerable attachment style & joined programme earlier associated with higher engagement, while smoking during pregnancy associated with lower engagement

  18. Contamination in RCTs • Contamination occurs when the control groups either actively or passively receive the intervention intended for the treatment group (Cook & Campbell, 1979) • Potential for contamination is high in PFL • Members of the treatment and control groups may be friends, neighbours, colleagues, same family! • Community comparison group used to safeguard against contamination • Included a range of direct (‘blue-dye’) in each survey

  19. Contamination in PFL • If contamination DID NOT take place: • High & Low treatment group will differ in their responses • Low treatment group will not differ from comparison group • High treatment group will differ from the comparison group • Contamination Questions in 6 month PFL survey • Little evidence of contamination among treatment groups • Evidence of absorption of PFL knowledge among high treatment group

  20. Update on PFL • Oldest PFL child is almost 4 years and old & youngest is 13 months • 18, 24, & 36 month surveys are currently in the field • 12 month results available during Summer • Over 1,500 interviews conducted as part of impact evaluation • Conducted four school readiness surveys with junior infant cohorts in 2008, 2009, 2010, 2011 • Conducted focus groups with low & high treatment groups & semi-structured interviews with all PFL mentors • Evaluation will continue until all children start school Programme website:www.preparingforlife.com Evaluation website: http://geary.ucd.ie/preparingforlife/

More Related