1 / 18

Session 2 Exploring the Issues Group Work

Session 2 Exploring the Issues Group Work. Exploring the Issues - Retrospective. Group Work: To what extent have the standards led to better quality evaluations, in particular.. What have been the strengths & success in using the EQS?

faunia
Download Presentation

Session 2 Exploring the Issues Group Work

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Session 2 Exploring the Issues Group Work

  2. Exploring the Issues - Retrospective Group Work: To what extent have the standards led to better quality evaluations, in particular.. What have been the strengths & success in using the EQS? What have been the limitations & expectations of the EQS?

  3. EQS The French speaking group Justine, M, Modibo Makalou, Gerhard Siegfried, Benoît Chervalier, Dominique de Crombrugghe

  4. EQS strenghts Used at several steps of evaluation: • Reference framework, also with other standards (eg SEVAL) • Selection of consultants • Report reading tool : grid in 10 points (Fr); quality grid in BMZ, GTZ, KfW • Makes comparative reading of reports possible and helps capitalise on consultants’ work

  5. EQS Strenghts (continued) • Helps overcoming desks’ resistance on evaluations • Builds confidence with partners • Helps convincing policy level • Reference framework for meta evaluations

  6. EQS Weaknesses • Not yet systematically incorporated in tor • Uneasy access for operational desks • Dissemination and training on ESQ insufficient (both for evaluators and decision makers) • Standards not known (Senegal, Benin) • Failing standards, evaluation design and implementation is scattered • Use of standards, among other tools such as guidances, still confused

  7. EQS limitations • Utility of evaluations: good standards are not a sufficient condition for useful evaluations • There is littel feed back by consultants on the standards • Partner appropriation of standards still weak • Some rules are still difficullt to accept in other cultural environments such as disenting opinions • They are donor standards

  8. EQS Attention points • Make it clear who the users are • Training and dissemination on the standards

  9. DAC EVALUATIONS QUALITY STANDARDS WORKSHOP GROUP ONE (1) PRESENTATION

  10. Background: • Mix of Players in the group • Some have used and others have not • Recognition of importance of M&E • Other Standards – UNEG, Bilateral, IFIs • Viet Nam further ahead

  11. Strengths and Successes in using EQS??? • Some common core items help to avoid multiplicity of requirements • Facilitates donor/partner country efforts • Some Partner’s EQS (tools, techniques, standards) must be built, strengthened and sustained • Current EQS forming the basis of Uniform EQS (must be encouraged) • Joint Evaluations must be encouraged (harmonisation) • Potential use at front end to encourage results management • Potential for professionalisation

  12. Limitations and Expectations with the EQS??? • Long and can be reduced/simplified • Standards are mixed of guidance and quality criteria (need to determine focus) • Methodological aspects can be improved • Cultural context and mechanism for feedback not adequately covered • Paris Declaration considerations are not covered • Ethical aspects are not adequately covered • Challenges for smaller partner countries - capacity, data availability and quality problematic

  13. Group 2:Widjanarko Soebhadi, Miranda Cahn, Ginny Chapman, Ted Kliest, Gunila Tornquist Strengths -Informs how people should work -Consistent standards across countries -Can assist in development of country standards -DAC provides credibility -Improves probability of higher quality evaluations

  14. Recommendations • Stress and institutionalize constructive nature of evaluation • Emphasize partnership with partner country • Evaluations should cover role of both donor/partner • Emphasize capacity building of partners • Need separate section on management and governance (6.1-8.2) • Include but do not specify cross cutting issues • Review Methodology section for level of specificity and clarity • Rearrange so as to link the sections on objectives to evaluation questions • Better promotion, awareness and dissemination of EQS

  15. Group 3 • To what extent have the EQS led to better quality evaluations, in particular? • maybe, but it is hard to tell as implementation is occurring progressively • help to develop a common language and shared understanding • potential to develop increased co-ordination

  16. What have been the strengthens and success in using the EQS? • do seem to be providing a basis for shared understanding and potential for better co-ordination • successes are not the same for donors and partner counties • self-reflective tool - this seems to be leading to clarity for improvement areas • treated as flexible therefore allow for the needs of individual partner and agency variation

  17. What have been the limitations and expectations with the EQS? • Donor focused - Language and emphasis - written from a donor perspective – e.g. Refers to AID policy not development policy • Partner country view - the EQS aren’t well know, including across a range of Departments - some have their own standards, therefore EQS is in addition (local systems, including evaluation, need to be recognised) • EQS might be too ambitious already – will the EQS become too complex • might be simplified to be more principle-based • Capacity building

  18. Group 4Evaluation Quality Standards + - Part of bigger picture Logic missing – structure could be change to cover: design; conduct; product. Dissemination • Generic • Flexible • Checklist • Useful - Developing evaluation policy

More Related