slide1 n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Education PPP: an Output-Oriented Approach PowerPoint Presentation
Download Presentation
Education PPP: an Output-Oriented Approach

Loading in 2 Seconds...

play fullscreen
1 / 34

Education PPP: an Output-Oriented Approach - PowerPoint PPT Presentation


  • 58 Views
  • Uploaded on

Education PPP: an Output-Oriented Approach. Exp = PPP ×Quantity ×Quality. Tertiary. Vocational. S econdary. Primary. No schooling. Outline: 1. Quantity 2. Quality 3. Conceptual model. Tertiary. Vocational. S econdary. Primary. No schooling.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Education PPP: an Output-Oriented Approach' - derick


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

Education PPP:

an Output-Oriented Approach

Exp = PPP ×Quantity ×Quality

Tertiary

Vocational

Secondary

Primary

No schooling

slide2

Outline: 1. Quantity

2. Quality

3. Conceptual model

Tertiary

Vocational

Secondary

Primary

No schooling

measuring the quantity of education
Measuring the Quantity of Education
  • Basic element is pupil counts: how many pupils consumed education services?
  • Enrollment figures generally available from national Education Management Information Systems (EMIS), but must be validated
  • Actual services consumed by pupils calculated using a time-on-task measure of the pupil-hours volume of instructional activity
validating enrollment figures method 1
Validating enrollment figures (method 1)
  • Adjust for school coverage (are all schools included in National EMIS systems?)
  • Sample survey of schools
  • Compare to time series trends
  • Adjust for pupil expenditures
  • Ensure all sub-sectors report and look for consistencies
validating enrollment figures method 2
Validating Enrollment Figures (method 2)
  • Compare Enrollments from EMIS and Household Surveys, such as DHS
  • Differentiating between Enrollment and Attendance (EMIS captures enrollment, household surveys – attendance)
  • Use Verification Protocols developed by AED, UIS, UNICEF and UNESCO
method 2 verification protocols
Method 2 verification protocols
  • Use UIS data when no survey MICS/DHS data are available
  • Use MICS/DHS when no UIS data available
  • Use UIS when data is < 5% difference,
  • Use expert when data is > 5 % difference
adjust quantity by estimating pupil hours
Adjust Quantity by Estimating Pupil Hours
  • Subtract Days Lost from Official School Year
  • Collect Teacher Absence Data and Roll-up
  • Collect Student Absence Data and Roll-up
  • Calculate Daily Time Loss
  • Survey Time on Task
  • Sample Schools and Estimate Absence Data across Stages/Levels/Organization
stratification
Stratification
  • Disaggregation by ISCED levels (some national education systems do not align neatly w ISCED)
  • Possibly, disaggregation by:
    • Public
    • Private
    • Complementary models (religious, community schools)
slide9

Measuring the Quality of Education

  • Moving towards a common learning metric
  • Composite Learning Scores in Education PPP
  • Data imputation will help address the problem of missing quality scores
why impute learning scores in education ppp
Why impute learning scores in education PPP?
  • A fair assessment of value is inevitably linked to outputs
  • In education, outputs are learning outcomes, measured by assessments of student achievement
  • Participation in assessments is growing, but far from global
  • Imputation of missing values is a way of accounting for quality in a broader group of countries than those currently available
  • International and regional assessments will be used.
towards a common quality adjustment
Towards a Common Quality Adjustment
  • Data imputation is necessary to include all countries in a quality adjustment process
  • All available information must be used
  • Reliability and validity will be tested
imputation methods some examples
Imputation Methods: Some Examples
  • Crouch (Crouch and Vinjevold, 2006)
    • Recursive regression imputation in several steps, based on correlation with TIMSS 1999 scores
    • Univariate regressions with target learning score (TIMSS 1999) as dependent variable, one assessment at a time as sole predictor
    • Result: several predicted scores for some countries, weighted average taken as imputed value
    • Weights based on correlation with TIMSS 1999
  • Hanushek and Woessmann (2009, 2010)
    • “Calibration” of major international achievement studies into a new scale.
    • Based on U.S. performance in international studies, and NAEP as reference point.
    • Mean and standard deviation of 13 OECD countries
multiple imputation mi
Multiple Imputation (MI)
  • Long-established methodology for treating missing data (for overview see Rubin, 1996; Allison, 2002)
  • Allows to use all available information
  • Accounts for uncertainty associated with imputation (multiple estimates generated)
  • Imputes more than one variable with missing data at a time
  • Drawback: demands on technology/ time to produce reliable estimates.
imputation of learning scores the process
Imputation of Learning Scores: the Process
  • Define dataset: three “core” assessments (state-of-the-art) and predictors of education quality
    • GDP per capita
    • Age structure (percent youth)
    • Per pupil expenditure
    • Teacher-student ratio
    • Other variables known or theorized to predict quality
  • Missing values on predictor variables will also be imputed
multiple imputation mi the mechanics
Multiple Imputation (MI): the Mechanics
  • Conditional prediction models are specified for each of the variables with missing data
  • MI runs iteration cycles “filling in the blanks” in all of the variables across 3-5 chains, based on the specified models
  • The goal is to achieve convergence of the distributions of imputed values across the chains, thereby increasing reliability
  • Estimates are then pooled across the chains
  • The result is 3-5 completed (imputed) datasets
the composite learning score stages
The Composite Learning Score: Stages
  • Because of the high proportion of missing data on learning assessments, the construction of the learning scores will be done in stages
  • Stage 1: Countries with international assessments: Impute all three “core” assessments, construct single score from weighted average (90 out of 183)
  • Stage 2: Countries with regional assessments: impute one composite learning score (19 countries)
  • Stage 3: Countries with no assessments, but complete data on predictors of quality: impute one composite learning score (TBD: total is 74)
  • Actual or imputed learning score will be transformed to a specified scale (currently Mean= 500; SD=100)
sources of error and threats to validity
Sources of Error and Threats to Validity
  • Unequal, nonrandom geographic representation in international assessments
  • Regional assessments and EGRA lacking in standardization, hence lower reliability
  • Large proportion of ICP countries (74) never participated in learning assessments, hence quality scores will be derived solely based on inputs and proxies.
  • MI will account for some of the uncertainty (averaging out across multiple estimates)
  • Crude methods (e.g. single imputation using regression or predictive mean matching) may be applied for difficult cases.
after the imputation adjustments
After the Imputation: Adjustments
  • Once in the PPP model, learning scores will be adjusted for student SES and length of schooling
  • SES Index: variables from available international achievement databases, as well as household surveys
  • Length of schooling: weighted average of years spent through graduation from secondary, based on the proportion attending preschool
after the imputation adjustments1
After the Imputation: Adjustments
  • Once in the PPP model, learning scores will be adjusted for student SES and length of schooling
  • SES Index: variables from available international achievement databases, as well as household surveys
  • Length of schooling: weighted average of years spent through graduation from secondary, based on the proportion attending preschool
slide23

Conceptual model

Quantity

Value of

Education

Services

Quality

slide24

EMIS counts

Verification,

Adjustments (Moses, Stukel)

# Pupils

Other sources

Volume

Value of

Education

Services

V = f(Pupils, y)

Pupils = EMIS counts x adjustment

OR

V = HH survey attendance

Quality

slide25

EMIS counts

Verification,

Adjustments (Moses, Stukel)

# Pupils

Other sources

Volume

Time on Task

Value of

Education

Services

Quality

slide26

EMIS counts

Verification,

Adjustments (Moses, Stukel)

# Pupils

Other sources

Volume

Time on Task

Value of

Education

Services

International assessments

iScore

(composite learning score)

Quality

Predictors of

quality

slide27

EMIS counts

Verification,

Adjustments (Moses, Stukel)

# Pupils

Other sources

Volume

Time on Task

Value of

Education

Services

International assessments

iScore

(Crouch, MI)

Quality

GDP, other

SES adjustment

HH surveys

slide28

EMIS counts

Verification,

Adjustments (Moses, Stukel)

# Pupils

Other sources

Volume

Time on Task

Value of

Education

Services

International assessments

iScore

(Crouch, MI)

Quality

GDP, other

SES adjustment

HH surveys

Preschool duration & coverage

Average years to 8th grade

Repetition

Dropout

slide29

EMIS counts

Verification,

Adjustments (Moses, Stukel)

# Pupils

Other sources

Volume

Time on Task

Value of

Education

Services

International assessments

iScore

(Crouch, MI)

Quality

GDP, other

SES adjustment

HH surveys

Preschool duration & coverage

Average years to 8th grade

Repetition

Tertiary adjustment

Dropout

Earnings ratio

Tertiary/ Lower secondary

further notes
Further notes

All calculations by ISCED level

Private, public, religious may be aggregated

Q applied to preschool – secondary

Q(t) separate for tertiary

Data on key inputs available for most countries

Pilot to include:

EMIS count verifications

Time on task estimates

Learning assessments (?)

the roadmap
The Roadmap
  • (ICP/Regional and External) Refine proposed conceptual model of output methodology into an implementable instrument
  • (External) First-level secondary data assessment and collection (for model-testing phase)
  • (External) iScore construction: imputation, adjustment.
  • (External) Identify insufficient data coverage on important predictors, develop plan to collect missing data.
the roadmap cont d
The Roadmap, cont’d
  • (Local/regional and external) Train local or regional technical teams to verify pupil counts, implement the sample survey for EMIS information verification.
  • (Local/regional and external) Train local or regional technical teams to assess actual instruction time, implement survey to collect missing information on instruction time.
  • (Local/regional) Assess whether EMIS counts or household survey estimates are the more reliable source of pupil count information; verify EMIS counts
  • (Local/regional) Implement assessment and surveys of actual instruction time
the roadmap cont d1
The Roadmap, cont’d
  • (External, but can train regional experts) Collect SES, repetition, dropout, and preschool attendance data for all ICP countries where such information is available.
  • (External, but can train regional experts) Calculate Quality and Quantity for all countries with complete dataset.
  • (External) Rerun the imputation models with newly collected information, in particular, time-on-task.
the roadmap cont d2
The Roadmap, cont’d
  • (External) Evaluate the value added of time of task, and the difference from validated pupil enrollment numbers.
  • Analyze and process the information into PPP adjustment values for the piloted countries, to evaluate the process, and to propose adjustments where necessary for a full-scale implementation of the output approach to education PPPs.