1 / 20

Independent Evaluation « Quality Assessment of SDC’s External Evaluation Reports »

Independent Evaluation « Quality Assessment of SDC’s External Evaluation Reports ». Anne Bichsel Evaluation + Controlling Division. Content Objectives of the Quality Assessment Procedures and Method Findings Recommendations. 1. Objectives ot the Quality Assessment

ilana
Download Presentation

Independent Evaluation « Quality Assessment of SDC’s External Evaluation Reports »

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Independent Evaluation « Quality Assessment of SDC’s External Evaluation Reports » Anne BichselEvaluation + Controlling Division

  2. Content • Objectives of the Quality Assessment • Procedures and Method • Findings • Recommendations

  3. 1. Objectives ot the Quality Assessment • To assess the quality of SDC‘sexternal evaluation reports • To contribute to improving future performance and achieving a higher evaluation quality of SDC‘s external evaluations

  4. 2. Procedures and Method (I) • Step I: Random Sample of 12 external evaluation reports • Step 2: devise a list of evaluation standards on the basis of- „DAC Minimum Sufficient Evaluation Standards“- SEVAL Standards- Key questions in Approach Paper

  5. 2. Procedures and Method (II) • List of evaluation standards with 4 categories • Utility (9 standards): readable, accessible, timely evaluations with a good summary • Feasibility (3 standards): realistic well thought-out evaluations • Propriety (4 standards): ethical aspect of evaluations • Accuracy (7 standards): proper scientific methods and procedures

  6. 2. Procedures and Method (III) • Step 3: review documents, conduct interviews with desk managers and evaluators • Step 4: work through all the standards for the sample of evaluations. Result: 12 fact sheets • Step 5: analysis and comparison • Step 6: conclusions and recommendations

  7. 3. Findings (I) • Strengths of external evaluation reports • Identification and participation of stakeholders • Timely reporting • Cost effectiveness • Complete and balanced assessment

  8. 3. Findings (II) • Weaknesses of external evaluations reports • Selection procedures of evaluation team • Formal written agreement • Comprehensive and clear reporting • Description of evaluation purpose and objectives • Description of evaluation procedures and methods • Making findings available

  9. 4. Recommendations (I)  Draft good, realistic and comprehensive TORs • Clear purpose • Clear, focused, concise and understandable objectives with three to four questions • Agreement on TORs among SDC divisions • Realistic expectations in terms of resources and timeframe • Beware of questions at the impact level

  10. 4. Recommendations (II)  Ensure more competitive and open selection procedures • Aim at balance in desired knowhow and independence of evaluators • Place more weight on evaluation knowhow (methods) • More open and transparent selection procedures • More competition • Careful team building in matching personalities, skills, division of labor

  11. 4. Recommendations (III)  Improve the conditions for the utilisation of external evaluations • Accompany evaluations closely for maximum learning potential • Continuity on the part of desk managers • Reports need to be more accessible to a wider audience • Enhance the quality of evaluation reports • Make results of external evaluations more widely available

  12. 4. Recommendations (IV)  Enhance the conditions for high quality external evaluations Evaluations are a challenging business, hence:Provide support and training for desk managers • Training in evaluation • Support in drafting TORs and commissioning evaluations • Support in obtaining high quality reports

  13. In sum: less is more !

  14. Direktor DEZA Stellv. Direktor Gerhard Siegfried Samuel Wälty Anne Bichsel Sekretariat (je 50%) Regula Herlan Christa Rohner jeweilige Bereichsleitung H-Bereich O-Bereich E-Bereich M-Bereich F-Bereich A-Bereich Christoph Jakob Kuno Schläfli Peter Meier Regula Bäbler MarkusGlatz Ivo Angehrn Alexandre Kobel Follow-up by the Core Learning Group: Evaluation & Controlling Net

  15. TORs&contracts • Implementation • Draft Report • Debriefing • Final Report 0. • COSTRA Decision Steps of an Independent EvaluationCOSTRAComité Stratégique I. • Approach Paper • Core Learning Partners (CLP) II. III. • Agreement at Completion Point • Dissemination to Broad Learning Partners (BLP) IV. • Senior Management Standpoint discussed in COSTRA

  16. Measures to improve quality • Communicated evaluation results / recommendations in all departments • Clarified and communicated the palette of instruments • Developing toolkit for commissioners of evaluations • Intraweb: Description of Process / examples • Principles to ensure successful Use of Evaluations • Improving training courses: curriculum and availability (1 day training Ext. Evals) • Backstopping and Quality Control by the E&C Net

  17. Palette of Instruments Independent Evaluation External Evaluation External Review Self-Evaluation Expert Opinion

  18. ExternalEvaluations Independent Evaluationssince 2001 ExternalReviews according to operational needs 5 – 10 annually 2 – 5 annually outside line management( E&C Division) within line management - issues of interest across departments- selected country programs - to generate knowledge within the department- of interest across divisions - ongoing operations - in the context of program cycle management aim for the same standards DAC Executive Summary Registration in data management system (DMS)

More Related