1 / 23

Measuring Outreach Effectiveness

Measuring Outreach Effectiveness. Web4Dev – November 2006 Alex McKenzie - amckenzie@worldbank.org Knowledge & Evaluation Capacity Development Independent Evaluation Group – World Bank. Key Points:. Share IEGWB’s approach for aligning web/online practice with mandate and business strategy

berny
Download Presentation

Measuring Outreach Effectiveness

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Measuring Outreach Effectiveness Web4Dev – November 2006 Alex McKenzie - amckenzie@worldbank.org Knowledge & Evaluation Capacity Development Independent Evaluation Group – World Bank

  2. Key Points: • Share IEGWB’s approach for aligning web/online practice with mandate and business strategy • Share our experience running several outreach campaigns, discuss metrics • Measuring effectiveness and impact • Would appreciate hearing about your own experience and/or feedback

  3. Business Strategy • IEG is an independent unit that assesses the effectiveness of World Bank Group’s interventions • Well regarded for its evaluation capabilities and reports • Combines Accountability & Learning • “To be effective, IEG want’s to be an influential and reliable source of knowledge” • New Outreach and Communications Strategy: • Target key audiences • Embed outreach throughout the evaluation process • Accelerate learning, encourage utilization • Measure results • Decided on a campaign-based approach, 10 to 12 major evaluation products per year. • Multi-pronged outreach campaigns: media, print, online, in-person, translations, etc. • Piloted many new ideas, mainstreamed what worked

  4. Key Audiences • Systematic effort to identify audiences has led to two sets: “core” & campaign-specific (specialists) • Identify decision-makers & influencers • Identify key network nodes • Evolving audience criteria: from “static” classifications, to others based on opportunity or time-sensitive demand • Customize products to audience whenever economically feasible (e.g. conference CDROMs, translations, etc.)

  5. Embed in Evaluation Process

  6. Encourage Utilization

  7. Encourage Utilization

  8. Measuring Results • Results-Based Management causal chain

  9. Measuring Results – Causal Chain Source: “Strengthening EAP’s Results Orientation”, April 2004

  10. Measuring Results - Outputs

  11. Measuring Results - Outputs

  12. Measuring Results - Outcomes • Intermediate Outcomes: perception survey* of IEG evaluation products, covering: • Readership • Quality • Influence on Bank processes and policies (similar for external clients) • Utilization • Annual Review of Operations Evaluation (AROE) 2006, to be released in November/December. *Source: IEG’s Annual Review of Operations Evaluation (AROE). See http://www.worldbank.org/ieg/aroe

  13. Measuring Results - Outcomes • Survey targeted individuals defined as intended audience: 4285 internal (WBG) and 2759 external contacts. Results are indicative for respondents (22%) but are not generalizable to population

  14. Perception of Quality – Staff/EDs (E4)

  15. Perception of Quality – External Clients (E13)

  16. Influence – Staff (E8)

  17. Influence – External Clients (E15)

  18. Staff Use of Evaluations (E9)

  19. External Clients Use of Evaluations (E16)

  20. Measuring Results – Survey Implications • IEG provides good services to its main client, the Board. External audiences are also highly satisfied with the quality of IEG evaluations and considered them to be influential. • However external audiences would like IEG to broaden its external consultation during evaluations. • Bank staff see IEG products useful with deepening their understanding of a given subject. • However they don’t see them as useful in day-to-day work. IEG needs to focus on influencing ongoing and future operations.

  21. Measuring Results – Survey Implications • All audiences suggest: • High value placed on timeliness to provide information when decisions need to be made • Deepening analysis through more consultation and attention to context • Focus on operational value of findings • A call for action for IEG’s outreach program • 4-page briefs • Customized “findings” brief, upstream, via email • More use of non-print media • Balance “lecturer” approach with conversations

  22. Thoughts Going Forward • "What information consumes is rather obvious; it consumes the attention of its recipients. Hence, a wealth of information creates a poverty of attention." Herbert Simon, Nobel laureate

  23. Your Experience/Feedback • What has worked in your organization? (audience analysis; relevance/timeliness; measuring outputs, outcomes, impacts, etc.) • Choices related to channels? High quality content vs. open/collaborative models • Outreach models used? Worthwhile consultants & advice? • What challenges do you see ahead?

More Related