Some perspectives on the importance of policy evaluation
Download
1 / 19

Some perspectives on the importance of policy evaluation - PowerPoint PPT Presentation


  • 107 Views
  • Uploaded on

Some perspectives on the importance of policy evaluation. Joost Bollens HIVA- K.U.Leuven. Evaluation of ALMP’s. Why evaluate ALMP’s ? How to measure effectiveness ? Some practical issues Unanswered questions. ALMP’s. Active Labour Market Policies Training for the unemployed ;

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Some perspectives on the importance of policy evaluation' - etta


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Some perspectives on the importance of policy evaluation

Some perspectives on the importance of policy evaluation

Joost BollensHIVA- K.U.Leuven

Joost Bollens


Evaluation of almp s
Evaluation of ALMP’s

  • WhyevaluateALMP’s?

  • Howtomeasureeffectiveness?

  • Some practical issues

  • Unansweredquestions

Joost Bollens


Almp s
ALMP’s

  • Active LabourMarketPolicies

    • Training for the unemployed;

    • Private sector incentiveschemes (wage subsidies, start-upgrants,…);

    • Direct employment programmes;

    • Counseling, monitoring, job search assistance, sanctions;

Joost Bollens


Some perspectives on the importance of policy evaluation

Public expenditureonALMP’s

Joost Bollens


Why evaluate almp s
WhyevaluateALMP’s?

  • Active policies : beneficialeffects

  • Strong beliefs

  • Is thisreally the case?  Impact evaluation

    • Different programmes in one country

    • Allequallyeffective?

Joost Bollens


Why evaluate almp s1
WhyevaluateALMP’s?

  • Evidencebasedpolicy : givenevaluationresults, decideto:

    • Continue the programme

    • Expand the programme

    • Restructureorredesign the programme

    • Abolish the programme

  • In the end : a matter of accountability

Joost Bollens


Evaluation
Evaluation

  • Processevaluation

    • How is the programmeimplemented?, Management quality?, Proper design?, Selectionprocesses?,…

  • Impact evaluation : effectiveness

  • Efficiency : costeffectiveness

    • Twoequallyeffective programmes may have a quite different cost per participant

Joost Bollens


Impact evaluation
Impact evaluation

  • Effectiveness : a lot of possibleoutcomes

    • % of participantsthatfind a job, % thatleaveunemployment, % thatfind a stable job orstableemployment,…,

    • % thatfind a decent job, effectsonhealth, psychologicaleffects, effectsonwell-being

Joost Bollens


Gross versus net effectiveness
Gross versus net effectiveness

  • Observedoutcome : effect of programmeparticipation + effect of factors outside the programme

    • Therefore, if we observethat 6 monthsafter finishing the programme e.g. 60% of the participants do have a job, thiscannotentirelybeattributedtoprogrammeparticipation : even without participating in the programme, someunemployedwould have found a job within 6 months

Joost Bollens


Net effectiveness
Net effectiveness

  • In order tofind the proper impact of the programme (the “valueadded”, or the “net effectiveness”, or the “impact”) , we have to correct the observedgross effect :

    Net effect = [Gross effect] - [the % of participantsthatwould have found a job even without participating]

  • Sinceparticipantscannot at the same time benon-participants, the red quantitycannotbeobserved (“counterfactual”) and must beestimated

Joost Bollens


Estimating counterfactuals
Estimatingcounterfactuals

  • Non-experimentalapproaches(includingquasi-experiments)

    • Several, more orless sophisticated approaches

    • Basically : compose a comparisongroup of personswho are comparabletoparticipants, BUT whodidnotparticipate

    • Potentialweakness : comparabilitynot complete, e.g. dueto (self-) selectioneffects. Example : motivation

Joost Bollens


Estimating counterfactuals1
Estimatingcounterfactuals

  • Experimentalapproaches

    • Basically : take the group of personswho are willingtoparticipate in a programme, and randomlyassign half of themto a experimentalgroup, and half of themto a controlgroup

    • Experimentalgroup is allowedtoparticipate, controlgroupnot

    • Results of controlgroup serve as counterfactual

    • Advantage : betterguaranteeforcomparability, factors like e.g. motivationwillon average be the same in bothgroups

    • Strong resistance in a lot of countriestothisapproach : “unequaltreatment”. However, given the cost of ALMPs and the intrinsicuncertainty as totheireffects, thisshouldbereconsidered

Joost Bollens


Some practical issues
Some practical issues

  • Planning helps

    • Plan before the introduction of a newprogramme

    • However, avoid the evaluation of a brand newprogramme

    • Radicallychanging (orabolishing) a programmebefore the end of the evaluationmakes the resultssomewhat irrelevant

Joost Bollens


Some potential conflicts
Somepotentialconflicts

  • Time is onourside?

    • Policy makers, evaluation sponsors, programme administrators want immediatelyevaluationresults↔evaluatorwillinsistthat a thoroughevaluationtakes time

    • Impact evaluationresultsnecessarilywillonlybeavailablesome time afterparticipation

    • The resulting “this is old stuff”-argument is not per sevalid

Joost Bollens


Some other potential conflicts
Someotherpotentialconflicts

  • Different expectations: “usableinformation” (e.g. whatcanbeusedto fine tune the programme) ↔whereasevaluatorsoften are (somewhatmyopically?) in the first place interested in the validity of their impact estimates

  • Makeevaluation more usefulbyuncoveringrelationshipbetweeneffectiveness and design aspects

Joost Bollens


Some other potential conflicts1
Someotherpotentialconflicts

  • Moreover, policy makers etc. onlyseemtobeinterested in impact estimateswhen these are positive, whilenegativeresultsoften are downplayedoroutrightneglected

  • (apparently?) contradictoryconclusions

  • Meta-analysiscan help

Joost Bollens


Remaining questions 1
Remainingquestions 1

Is net effectivenessrelatedto …

  • …specificgroups? What does (doesn’t) workforwhom and why (not)?

  • …combination of severalpolicies? Order?

  • …timing of intervention ?

  • …labourmarketinstitutions?

  • … intensityor “dose” orduration?

Joost Bollens


Remaining questions 2
Remainingquestions 2

Is net-effectiveness different between …

  • …public versus private provider?

  • …local versus nationalprogramme?

  • …favourable and unfavourable business cycleconditions?

  • …short run and long run ? (locking in?)

  • …sample in evaluationstudy, and futureparticipants? (externalvalidity)

Joost Bollens


Macro effects of almp s
Macro-effects of ALMP’s

  • Thusfar: effect onparticipants

  • Butalso :

    • Effect onnon-participants? Substitution, displacement, deadweight loss,….;

    • General equilibriumeffects

    • Effectsonemployment, unemployment, productivity, matchingeffectiveness, ….

  • Very important, yet a lot of uncertainty

Joost Bollens