Measuring
This presentation is the property of its rightful owner.
Sponsored Links
1 / 20

Preparing for Life Programme PowerPoint PPT Presentation


  • 109 Views
  • Uploaded on
  • Presentation posted in: General

Measuring Investment in Human Capital Formation: An Experimental Analysis of Early Life Outcomes Orla Doyle, Colm Harmon, James Heckman, Caitriona Logue, Seong Moon UCL, 25 th -26 th June 2012. Preparing for Life Programme.

Download Presentation

Preparing for Life Programme

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Preparing for life programme

Measuring Investment in Human Capital Formation: An Experimental Analysis of Early Life Outcomes

Orla Doyle, Colm Harmon, James Heckman,

Caitriona Logue, Seong Moon

UCL, 25th-26th June 2012


Preparing for life programme

Preparing for Life Programme

  • One of the first experimental early childhood intervention in Ireland

  • Funded by Irish Government (DCYA) & The Atlantic Philanthropies

  • Community-led initiative operated by Northside Partnership in Dublin

    • ~ 6,400 inhabitants

    • 33% dependent on social welfare

    • 60% live in social housing (>3 times national average)

    • 47% lone mothers (29% national average)

    • 16% unemployed (3 times national average)

    • 66% early school leavers (38% national average)

    • 5% third level education (29% national average)

      Source: Census (2006)- CSO

  • Belief within the community that children were lagging behind their peers


Design of pfl

Design of PFL

Aim:Improve levels of school readiness by assisting parents in developing skills to prepare their children for school

Bottom-up approach: community initiative involving 28 community groups, service providers, & local representatives

Theoretical Framework: Grounded in several psychological theories of development including the theory ofhuman attachment, socio-ecological theory of development and social-learning theory

Evaluation: Randomised Control Trial design


Design of preparing for life

Design of Preparing for Life


Pfl evaluation

PFL Evaluation

Impact Evaluation

  • Data collection: Pre-intervention (baseline), 3 mnts (WASI), 6mths, 12mths, 18mths, 24mths, 3yrs, 3.5yrs, 4yrs

  • Informant: Mother is the primary informant, but also fathers, child, other independent data sources (birth records)

    Implementation Evaluation

  • Aim: Delve into the blackbox of programme effectiveness & evaluate fidelity to the PFL model

  • Data collection:

    • Implementation data on the Database Management System

    • Focus groups with participants

    • Semi-structured interviews with mentors/IO


Recruitment

Recruitment

  • Eligibility Criteria:

    • Cohort of pregnant women residing in PFL catchment area between Jan 2008-August 2010 (32 months)

    • Includes preparious and non-preparious women

  • Recruitment:

    • Maternity hospital at first booking visit (b/w 12-26 weeks)

    • Within the local community

  • Population-based recruitment rate, based on all live births during the recruitment phase, was 52%


Computerized randomisation procedure

ComputerizedRandomisation Procedure

Unconditional probability randomisation strategy

After informed consent obtained, mother ask to press the key to allocate her assignment condition

Evaluation & Implementation team automatically received an email with the assigned PFL ID number and treatment condition

Preserves the integrity of the procedure as no opportunity for recruiter to intentionally influence assignment

115 allocated to High treatment group

118 allocated to Low treatment group


Baseline data collection

Baseline Data Collection

  • Conducted post randomisation, pre intervention

    • Baseline data wave completed in August, 2010

  • 123 measures: Demographics & SES, health behaviour & pregnancy, parenting, social support, psychological assessments (maternal well-bring, personality traits, self-esteem, self efficacy, attachment, time preferences)

  • Interviews conducted

    • PFLHigh treatment group: 104

    • PFLLow treatment group: 101

  • Used to determine the effectiveness of the randomisation procedure


Methodology

Methodology

  • Permutation based hypothesis testing (Heckman et al. 2010)

    • Classical hypothesis tests unreliable when the sample size is small and the data are not normally distributed

    • Permutation test are distribution free, thus suitable in small samples

    • Based on the assumption of exchangeability between treatment conditions under the null hypothesis

  • Stepdown procedure (Romano & Wolf, 2005)

    • Ignoring the multiplicity of tests may lead to the rejection of “too many” null hypotheses

    • Test multiple hypotheses simultaneously by controlling overall error rates for vectors of hypotheses using the family-wise error rate (FWER) as a criterion

    • Less conservative & more powerful than other methods as takes account of statistical dependencies between tests


Summary of permutation tests examining differences at baseline

Summary of Permutation Tests Examining Differences at Baseline

  • Randomisation worked!


Permutation tests comparing baseline differences in selected family socio demographics

Permutation Tests Comparing Baseline Differences in Selected Family Socio-Demographics


Analysis of 6 month outcome data

Analysis of 6 Month Outcome Data

  • Test for treatment effects across 8 main domains including 160 measures

    • Child development

    • Child health

    • Parenting

    • Home environment

    • Maternal health

    • Social support

    • Childcare & service use

    • Household factors & SES

  • 25 Step-down categories defined

  • 258 interviews conducted

    • PFLHigh treatment group: 84

    • PFLLow treatment group: 90


Attrition disengagement up to 6 months

Attrition & Disengagement up to 6 months

Baseline Characteristics Associated with Attrition

  • Official dropout between baseline - 6 months:

    • High treatment – 13%

    • Low Treatment – 6%

  • Disengagement between baseline - 6 months :

    • High treatment – 9%

    • Low Treatment – 10%


Preparing for life programme

Results for high & low treatment groups: Child development

Notes:1 one-tailed (right-sided) p value from an individual permutation test with 1000 replications. 2 one-tailed (right-sided) p value from a Step Down permutation test with 1000 replications. * indicates the variable was reverse coded for the testing procedure. ‘ns’ indicates the variable is not statistically significant.


Summary of six month results

Summary of Six Month Results


Summary of 6 month results

Summary of 6 month results

  • Majority of the results for High V’s Low are in hypothesized direction

  • A few significant differences identified (14%)

  • In line with other home visiting programmes


Additional analysis

Additional Analysis

  • Interaction & Subgroup analysis

    • Gender, lone parents v partnered parents, first time v non first time parents, high versus low IQ mothers, high v low family risk

    • Programme appears to benefit high IQ mothers, families with multiple children & families with multiple risks

  • Analysis of engagement (treatment intensity)

    • On average, high treatment group received 14 home visits of ~1 hr in duration= total treatment = 14 hours

    • Higher maternal IQ & vulnerable attachment style & joined programme earlier associated with higher engagement, while smoking during pregnancy associated with lower engagement


Contamination in rcts

Contamination in RCTs

  • Contamination occurs when the control groups either actively or passively receive the intervention intended for the treatment group (Cook & Campbell, 1979)

  • Potential for contamination is high in PFL

    • Members of the treatment and control groups may be friends, neighbours, colleagues, same family!

  • Community comparison group used to safeguard against contamination

  • Included a range of direct (‘blue-dye’) in each survey


Contamination in pfl

Contamination in PFL

  • If contamination DID NOT take place:

    • High & Low treatment group will differ in their responses

    • Low treatment group will not differ from comparison group

    • High treatment group will differ from the comparison group

  • Contamination Questions in 6 month PFL survey

  • Little evidence of contamination among treatment groups

  • Evidence of absorption of PFL knowledge among high treatment group


  • Update on pfl

    Update on PFL

    • Oldest PFL child is almost 4 years and old & youngest is 13 months

    • 18, 24, & 36 month surveys are currently in the field

    • 12 month results available during Summer

    • Over 1,500 interviews conducted as part of impact evaluation

    • Conducted four school readiness surveys with junior infant cohorts in 2008, 2009, 2010, 2011

    • Conducted focus groups with low & high treatment groups & semi-structured interviews with all PFL mentors

    • Evaluation will continue until all children start school

      Programme website:www.preparingforlife.com

      Evaluation website: http://geary.ucd.ie/preparingforlife/


  • Login