1 / 53

Introduction Quantitative Evaluation Methods Application of Non-parametric Approaches

Development and application of advanced quantitative methods to ex-ante and ex-post evaluations of rural development programmes in the EU. „ Advanced methods of ex ante and ex post evaluation of RD-policies: New insights from the EU-project ADVANCED-EVAL - Christian Henning, University of Kiel

ingrid
Download Presentation

Introduction Quantitative Evaluation Methods Application of Non-parametric Approaches

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Development and application of advanced quantitative methods to ex-ante and ex-post evaluations of rural development programmes in the EU • „ • Advanced methods of ex ante and ex post evaluation of RD-policies: New insights from the EU-project ADVANCED-EVAL- • Christian Henning, University of Kiel • Presentation at the seminar “Evaluating rural development policies” • 15 January 2010 in Uppsala, Sweden

  2. Road Map • Introduction • Quantitative Evaluation Methods • Application of Non-parametric Approaches • Macroeconometric approaches of ex post evaluation of RD-policies: GPS-Matching • Nonparametric approaches for ex ante evaluation of RD-policies: Generalized Matching • Application of model based evaluation methods • Evaluation of RD-Policies based on a micro-macro linked ABM-CGE-model • Critical Discussion

  3. Project Evaluation: A General Framework Project State of the World (Rural Development) Input PI1 . . . PIn Output PO1 . . . POm Results O1 . . . Ok Impact I1 . . . Ir PO=g(PI) R=f(PO) I=h(O)

  4. Why Evaluation ? • Policy Learning of truerelation between project input and project impact via ex post evaluation. • Policy Planning of optimal policy design via ex ante evaluation. • Accountability: Control and effort of programme implementation

  5. How to do Policy Evaluation? Dimensions of Evaluation Methods Design Data Model Strategy Evaluation Measure

  6. Evaluation Models Descriptive Models Statistical Models Theoretical Models • Logical framework Matrix • Expert report • Beneficiary report • Regression analysis • Matching • IV approach • Selection models • Partial equilibrium • General equilibrium • Agent-Based Modelling

  7. Obstacles and challenges of policy evaluation • Practical and methodological challenges of rigorours evaluation • Methodological Know how and theoretical knowledge • Data availability • Evaluator skills and capacities • Financial and time resources • Political and administrative feasibilty

  8. Project Objectives • Quantitative methods of ex-post evaluation of RD-programmes. • Quantitative methods of ex-ante evaluation of RD-programmes. • Derivation and measurement of Rual development index (quality of life index in rural areas).

  9. Project Objectives • Theory of rural development. In particluar, role of tangible and intangible (social network relation) in rural development and impact of RD-programmes. • Improvement of existing evaluation methods of EU RD-programmes • Using and disseminating Knowledge • Raising Public Participation and Awareness

  10. Project Structure Advanced-Eval WP 2 Quantitative Methodolgy for ex-post evaluation (statistical methods) WP 3 Derivation and econometric estimation of Rural Development Index WP 4 Quantitative network analysis of rural networks WP 5 Quantitative Methodolgy for ex-ante evaluation (Agent-based-models) WP 6 Comparison of new quantitative methodolgy with existing qualitative methodology

  11. European Commission Coordination & Project Office University of Kiel: Prof. Dr. Dr. Christian H.C.A. Henning Project Steering Committee Partners WG 1 - Team Leader: Dr. Jurek Michalek, Uni Kiel Working Group Leaders WG2 WG3 WG4 WG5 WG6 WP2 WP3 WP6 Project Management

  12. Projekt Partner Europrojects_LBV Germany Center of International Rural Development Studies University of Utrecht The Netherlands Department of Sociology, ICS University of Bonn Germany Department of Agricultural Economics University of Mannheim Germany Mannheim Centre of European Social Reserach University of Kiel Germany Department of Agricultural Economics, Department of Econometrics and Statistics University of Sussex United Kingdom Science and Technology Policy Research Centre Polish Academy of Science Poland Institute of Rural and Agricultural Development Institute of Agricultural and Food Economics Poland Research Institute of Agricultural and Food Economics Slovakia

  13. Dissemination of Knowledge http://www.advanced-eval.eu/

  14. Quantitiative Evaluation Methods Ex post Ex ante Parametric Non-parametric Parametric Non-parametric Micro Macro Micro Macro Micro Macro Micro Macro Micro-models - FHM - SFA - LP Macro-models - SAM - CGE - ABM - Micro-Macro- Models • Matching • PSM • DID-PSM • Matching • GPSM Micro-models - FHM - SFA - LP Macro-models - SAM - CGE - ABM - Micro-Macro- Models • Matching • GM

  15. Macroeconometric approaches of ex post evaluation of RD-policies: GPS-Matching • Generalized Propensity Score Matching: Hirano/Ibens 2004 • Extension of PSM in a setting with continuous treatment • Application within ADVANCED-EVAL: Evaluation of RD-measures on regional development

  16. Outline of methodological procedure of GPS-Matching (see: Hirano/Imbens 2004) 1. Step: Estimation of conditional denisty of the treatment given covariates: The GPS has a balancing property similar to that of the standard propensity score, i.e. Within the strate of the same vaule of r(t,X) the probabaility that T=t does not depend on the value of X.

  17. Outline of methodological procedure of GPS-Matching (see: Hirano/Imbens 2004) 2. Step: Bias removal using GPS: Two steps: 2.1 Estimate conditional expectation of outcome as a function of two scalars, the treatment level T and the GPS R, (t,r): 2.2 Estimate average potential outcome at treatment level t as: Doing this for each treatment level of interest results the entire dose-response function!

  18. Empirical application within WP3 of ADVANCED-EVAL

  19. Poland (PSM): Positive impact of measure M3 (development of rural infrastructure) on the RDI Index (Yet, “0-1” if less than 66% of the mean ! => arbitrary) • Empirical application within WP3 of ADVANCED-EVAL: RDI 20022005 DID Unmatched (1) 0.0065 0.0025 neg Unmatched (0) 0.0498 0.0512 pos Unmatched (1- 0) -0.0433 -0.0487 -0.0054 (negative) Matched (1) 0.0123 0.0088 neg Matched (0) 0.0126 0.0078 neg ATT (1-0)(PSM)-0.00030.0010 0.0013 (positive) Source: J. Michalek (2009)

  20. Empirical application within WP3 of ADVANCED-EVAL: Poland (GPS): Positive (increasing) impact of measure M3 on RDI Index – bootstrapping => no arbitrary thresholds! Source: J. Michalek (2009)

  21. Application of GPSM Method - Poland • Empirical application within WP3 of ADVANCED-EVAL: Source: J. Michalek (2009)

  22. Slovakia: Impact of all SAPARD measures on unemployment => positive response ! • Empirical application within WP3 of ADVANCED-EVAL: Total SAPARD Measures (Generalized Propensity Score Method) => label var treat "Treatment values" label var diff "Derivative dose-response function: E[Y(t+1)- Y(t)]" label var es_diff "Standard Error of the derivative dose response function" label var mean_value "E[Y(t)]" label var mean_value_plus "E[Y(t+1)]" label var mean_diff "E[Y(t+1)] - E[Y(t)]" label var es_mean_diff "Standard error of E[Y(t+1)] - E[Y(t)]" label var es_diff "Standard Error of the derivative dose response function" Source: J. Michalek (2009)

  23. Empirical application within WP3 of ADVANCED-EVAL: Comparisons => Slovakia: all measures (with thresholds) Source: J. Michalek (2009)

  24. An ABM-CGE modelling framework for Ex Ante Evaluation ADVANCED-EVAL WP5 Development and application of advanced quantitative methods to ex-ante and ex-post evaluations of rural development programmes in the EU

  25. Intervention Logic and Model Structure

  26. Intervention Logic and Model Structure

  27. Final Project Workshop of ADVANCED-EVAL Brussels, 27. October 2009 Structure of the CGPE-ABM • Module I: Linked FHM_CGE-Model: • Micro: FHM with SFA and ideosyncratic technology shocks, • Macro: interregional CGE model including transaction cost and exogenous price shocks at macro level. • Module II: ABM-Model of belief formation of economic agents in communication networks • Module III: ABM-Model of land market. Modeling investment and exit/entry decisions of individual farm-households via sigmoid-functions based on beliefs on future prices and technological progress

  28. Final Project Workshop of ADVANCED-EVAL Brussels, 27. October 2009 Structure of the CGPE-ABM Module IV: Modeling the local political sector and incorporating endogenous political behavior of local government into the general equilibrium approach Module V: Modeling migration and network dynamics

  29. Applying a Micro-Macro linked ABM-CGPE Model for Ex–ante Evaluation • General equilibrium effects • Non-market interaction among agents • Bounded rationality of agents • Endogenous local government behavior

  30. Empirical Specification of Micro-Macro-Linked ABM-CGE Model for 8 selected rural region in Poland and Slovakia • Econometric estimation of FHM-model (individual data of 1000 individual farm households) • Econometric estimation of farm exit, investment and migration decisions • Econometric estimation of social network structure • Econometric estimation of transaction costs • Calibration of local CGE based on local village SAM‘s

  31. Simulation of Rural development policy effects Axis 1 policy measure: • Support Scheme for Farm Investments (50% subsidies of total investment cost) • Training and information of farmers • Farm advisory services Axis 3 policy measure: • Diversification of the rural economy • Improvement of quality of life

  32. Simulation Scenarios

  33. Average Treatment Effects • Farm investment • - Farm profit • Farm income

  34. Average Treatment Effects: Investment Average Treatment Effects

  35. Average Treatment Effects: Profit

  36. Average Treatment Effects: Income

  37. Technical Progress: Sector level

  38. Impact of Implementation Scheme on ATT: Micro level

  39. Treatmeant Effect at Micro Level • Farm investment • Labor and land productivity • Commercial Inputs • Farm profit • Off-farm labor • Farm income

  40. Final Project Workshop of ADVANCED-EVAL Brussels, 27. October 2009 Farm Investment

  41. Final Project Workshop of ADVANCED-EVAL Brussels, 27. October 2009 Total Farm Productivity

  42. Final Project Workshop of ADVANCED-EVAL Brussels, 27. October 2009 Farm Profit

  43. Final Project Workshop of ADVANCED-EVAL Brussels, 27. October 2009 Farm Income

  44. Off-Farm Labor supply

  45. Treatmeant Effect at Macro Level • Per Capita Income • Unemployment • Environment • Labor productivity

  46. Impact of RD-Policies on Per Capita Income

  47. Impact of RD-Policies on Unemployment

  48. Impact of RD-Policies on Enviornment

  49. Impact of RD-Policies on Labor Productivity

  50. Heterogenous impact of RD-Policy on Per Capita Income

More Related