1 / 26

GROWING EVALUATION CAPACITY THE MID TERM EVALUATION IN OBJECTIVE 1 AND 2 REGIONS 8 OCTOBER 2004

GROWING EVALUATION CAPACITY THE MID TERM EVALUATION IN OBJECTIVE 1 AND 2 REGIONS 8 OCTOBER 2004. Process of the Mid Term Evaluation. PROCESS. Partnership Working Evaluators Quality Cost. PARTNERSHIP.

reginat
Download Presentation

GROWING EVALUATION CAPACITY THE MID TERM EVALUATION IN OBJECTIVE 1 AND 2 REGIONS 8 OCTOBER 2004

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GROWING EVALUATION CAPACITYTHE MID TERM EVALUATION IN OBJECTIVE 1 AND 2 REGIONS8 OCTOBER 2004

  2. Process of the Mid Term Evaluation

  3. PROCESS • Partnership Working • Evaluators • Quality • Cost

  4. PARTNERSHIP • Partners: Commission, Member State, Managing Authority, Monitoring Committee, Evaluator, Beneficiary • Steering Groups: positive contribution • Organisation: generally a strong point • Timing: work started appropriately early

  5. EVALUATORS • 170 companies • Growing incidence of consortia • Limited transnational working • Some evaluators over-stretched • No significant increase in the size of the marketplace of evaluators

  6. QUALITY (1)(Assessment by Commission on basis of MEANS criteria)

  7. QUALITY (2) • Positive Result showing increased evaluation capacity, but • Reports not too severely judged given breadth • Too much analysis and editorial work compressed into short period (draft final reports often not finished)

  8. COST (1)

  9. COST (2)

  10. COST AND COST EFFECTIVENESS • Over €35 million in total for Objectives 1 and 2: • Resources appropriate, but cost effectiveness limited by: • timing, • breadth of evaluation, and • methodological weaknesses

  11. Methodologies

  12. METHODOLOGIES Mix of methodologies including desk and primary research to include evaluation of: • developments in programme environment • implementation mechanisms • performance of measures, priorities and programme

  13. DESK RESEARCH • Included in all evaluations • Difficulties with monitoring information: • late or slow start of programmes • monitoring systems not yet functioning

  14. PRIMARY RESEARCH • Surveys • Focus Groups/Workshops • Case Studies

  15. METHODOLOGIES - CONCERNS • Limited primary research • Weak monitoring data • Analysis • Over-emphasis on financial analysis • Conclusions and Recommendations • Presentation of the reports

  16. Findings of the Evaluations

  17. FINDINGS OF THE EVALUATIONS 1. Key Evaluation Questions • Appropriateness of Strategy • Effectiveness • Efficiency • Quantification of Objectives • Implementation Systems 2. Contribution to Lisbon Priorities • Transport • Knowledge Based Economy • Entrepreneurship and Financial Instruments • Social Inclusion • Environment and Sustainable Development

  18. KEY EVALUATION QUESTIONS • Limited Need for Strategy Shifts • Effectiveness – Much Activity Underway after a Slow Start • Macro-economic Impacts • Results and Outputs • Efficiency – A Start Made in the Analysis of Unit Costs • Quantification of Objectives – an Urgent Need for Improvements • Implementation Systems – Significant Improvements compared to the Past

  19. CONTRIBUTION TO LISBON PRIORITIES • Transport – Good progress in large Road and Rail projects; smaller scale projects behind schedule • Knowledge Based Economy • Research, Innovation, Information and Communication Technologies (ICT) – A growing priority with good results in Objective 2 regions; slower but definite progress being made in Objective 1 regions • Human Capital

  20. CONTRIBUTION TO LISBON PRIORITIES (1) • Entrepreneurship and Financial Instruments –Performing well but affected by the global economic downturn • Social Inclusion – Integrated actions are most effective but are resource intensive • Environment and Sustainable Development – more consideration needed of the practical implementation of horizontal priorities

  21. Use of the Evaluations

  22. USES OF THE EVALUATIONS • Mid Term Review • Adaptation of Implementation Systems • Disseminating information about the Structural Funds • Feeding debate on public policies

  23. USING THE EVALUATIONS • Who used the evaluation? • In what way? • Were financial allocations changed? • Were implementation methods changed? • Are the evaluations freely available? • Has there been public debate? • Is there any intention to further consider findings? • Did any factors militate against use?

  24. Conclusions

  25. STRENGTHS & WEAKNESSES • Strengths: • Organisation and planning • Partnership • Allocation of Resources • Increased evaluation capacity • Weaknesses • Rigid deadline • Breadth of requirements • ToR not sufficiently adapted • Quality concerns

  26. CHALLENGES FOR THE FUTURE • Further Strengthening Evaluation Capacity • Managing evaluations – human and financial resources • Evaluators – commitment to quality and more targeted methodologies • New Member States – building on experiences • Improved Monitoring Systems • Creating a Platform for exchange of experience

More Related