some perspectives on the importance of policy evaluation
Download
Skip this Video
Download Presentation
Some perspectives on the importance of policy evaluation

Loading in 2 Seconds...

play fullscreen
1 / 19

Some perspectives on the importance of policy evaluation - PowerPoint PPT Presentation


  • 107 Views
  • Uploaded on

Some perspectives on the importance of policy evaluation. Joost Bollens HIVA- K.U.Leuven. Evaluation of ALMP’s. Why evaluate ALMP’s ? How to measure effectiveness ? Some practical issues Unanswered questions. ALMP’s. Active Labour Market Policies Training for the unemployed ;

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Some perspectives on the importance of policy evaluation' - etta


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
some perspectives on the importance of policy evaluation

Some perspectives on the importance of policy evaluation

Joost BollensHIVA- K.U.Leuven

Joost Bollens

evaluation of almp s
Evaluation of ALMP’s
  • WhyevaluateALMP’s?
  • Howtomeasureeffectiveness?
  • Some practical issues
  • Unansweredquestions

Joost Bollens

almp s
ALMP’s
  • Active LabourMarketPolicies
    • Training for the unemployed;
    • Private sector incentiveschemes (wage subsidies, start-upgrants,…);
    • Direct employment programmes;
    • Counseling, monitoring, job search assistance, sanctions;

Joost Bollens

why evaluate almp s
WhyevaluateALMP’s?
  • Active policies : beneficialeffects
  • Strong beliefs
  • Is thisreally the case?  Impact evaluation
    • Different programmes in one country
    • Allequallyeffective?

Joost Bollens

why evaluate almp s1
WhyevaluateALMP’s?
  • Evidencebasedpolicy : givenevaluationresults, decideto:
    • Continue the programme
    • Expand the programme
    • Restructureorredesign the programme
    • Abolish the programme
  • In the end : a matter of accountability

Joost Bollens

evaluation
Evaluation
  • Processevaluation
    • How is the programmeimplemented?, Management quality?, Proper design?, Selectionprocesses?,…
  • Impact evaluation : effectiveness
  • Efficiency : costeffectiveness
    • Twoequallyeffective programmes may have a quite different cost per participant

Joost Bollens

impact evaluation
Impact evaluation
  • Effectiveness : a lot of possibleoutcomes
    • % of participantsthatfind a job, % thatleaveunemployment, % thatfind a stable job orstableemployment,…,
    • % thatfind a decent job, effectsonhealth, psychologicaleffects, effectsonwell-being

Joost Bollens

gross versus net effectiveness
Gross versus net effectiveness
  • Observedoutcome : effect of programmeparticipation + effect of factors outside the programme
      • Therefore, if we observethat 6 monthsafter finishing the programme e.g. 60% of the participants do have a job, thiscannotentirelybeattributedtoprogrammeparticipation : even without participating in the programme, someunemployedwould have found a job within 6 months

Joost Bollens

net effectiveness
Net effectiveness
  • In order tofind the proper impact of the programme (the “valueadded”, or the “net effectiveness”, or the “impact”) , we have to correct the observedgross effect :

Net effect = [Gross effect] - [the % of participantsthatwould have found a job even without participating]

  • Sinceparticipantscannot at the same time benon-participants, the red quantitycannotbeobserved (“counterfactual”) and must beestimated

Joost Bollens

estimating counterfactuals
Estimatingcounterfactuals
  • Non-experimentalapproaches(includingquasi-experiments)
    • Several, more orless sophisticated approaches
    • Basically : compose a comparisongroup of personswho are comparabletoparticipants, BUT whodidnotparticipate
    • Potentialweakness : comparabilitynot complete, e.g. dueto (self-) selectioneffects. Example : motivation

Joost Bollens

estimating counterfactuals1
Estimatingcounterfactuals
  • Experimentalapproaches
    • Basically : take the group of personswho are willingtoparticipate in a programme, and randomlyassign half of themto a experimentalgroup, and half of themto a controlgroup
    • Experimentalgroup is allowedtoparticipate, controlgroupnot
    • Results of controlgroup serve as counterfactual
    • Advantage : betterguaranteeforcomparability, factors like e.g. motivationwillon average be the same in bothgroups
    • Strong resistance in a lot of countriestothisapproach : “unequaltreatment”. However, given the cost of ALMPs and the intrinsicuncertainty as totheireffects, thisshouldbereconsidered

Joost Bollens

some practical issues
Some practical issues
  • Planning helps
    • Plan before the introduction of a newprogramme
    • However, avoid the evaluation of a brand newprogramme
    • Radicallychanging (orabolishing) a programmebefore the end of the evaluationmakes the resultssomewhat irrelevant

Joost Bollens

some potential conflicts
Somepotentialconflicts
  • Time is onourside?
    • Policy makers, evaluation sponsors, programme administrators want immediatelyevaluationresults↔evaluatorwillinsistthat a thoroughevaluationtakes time
    • Impact evaluationresultsnecessarilywillonlybeavailablesome time afterparticipation
    • The resulting “this is old stuff”-argument is not per sevalid

Joost Bollens

some other potential conflicts
Someotherpotentialconflicts
  • Different expectations: “usableinformation” (e.g. whatcanbeusedto fine tune the programme) ↔whereasevaluatorsoften are (somewhatmyopically?) in the first place interested in the validity of their impact estimates
  • Makeevaluation more usefulbyuncoveringrelationshipbetweeneffectiveness and design aspects

Joost Bollens

some other potential conflicts1
Someotherpotentialconflicts
  • Moreover, policy makers etc. onlyseemtobeinterested in impact estimateswhen these are positive, whilenegativeresultsoften are downplayedoroutrightneglected
  • (apparently?) contradictoryconclusions
  • Meta-analysiscan help

Joost Bollens

remaining questions 1
Remainingquestions 1

Is net effectivenessrelatedto …

  • …specificgroups? What does (doesn’t) workforwhom and why (not)?
  • …combination of severalpolicies? Order?
  • …timing of intervention ?
  • …labourmarketinstitutions?
  • … intensityor “dose” orduration?

Joost Bollens

remaining questions 2
Remainingquestions 2

Is net-effectiveness different between …

  • …public versus private provider?
  • …local versus nationalprogramme?
  • …favourable and unfavourable business cycleconditions?
  • …short run and long run ? (locking in?)
  • …sample in evaluationstudy, and futureparticipants? (externalvalidity)

Joost Bollens

macro effects of almp s
Macro-effects of ALMP’s
  • Thusfar: effect onparticipants
  • Butalso :
    • Effect onnon-participants? Substitution, displacement, deadweight loss,….;
    • General equilibriumeffects
    • Effectsonemployment, unemployment, productivity, matchingeffectiveness, ….
  • Very important, yet a lot of uncertainty

Joost Bollens

ad