Bringing evaluation in moving from evaluation to evidence based decision support
This presentation is the property of its rightful owner.
Sponsored Links
1 / 13

Bringing Evaluation In: Moving from Evaluation to Evidence-based Decision Support PowerPoint PPT Presentation


  • 46 Views
  • Uploaded on
  • Presentation posted in: General

Bringing Evaluation In: Moving from Evaluation to Evidence-based Decision Support. Ben Pollard Director, VPS Portfolio Initiatives, UBC Former Director, Program Management and Evaluation, Immigrant Integration Branch. Immigrant Integration Overview.

Download Presentation

Bringing Evaluation In: Moving from Evaluation to Evidence-based Decision Support

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Bringing evaluation in moving from evaluation to evidence based decision support

Bringing Evaluation In: Moving from Evaluation to Evidence-based Decision Support

Ben Pollard

Director, VPS Portfolio Initiatives, UBC

Former Director, Program Management and Evaluation, Immigrant Integration Branch


Immigrant integration overview

Immigrant Integration Overview

  • All facets of settlement and integration of newcomers

    • Settlement and orientation

    • Language

    • Labour market and Foreign Credential Recognition

    • Community integration

    • Community based, school-based, etc.

  • Diversity of immigrants (~40,000 annually)

  • 25+ program

  • 300 contracts with 100+ service providers

  • $140m annual programming budget


Context for evaluation

Context for Evaluation

Significant Investment

Multiple complex, diverse lines of business

Multiple stakeholders

High visibility

Fed/Prov Agreement and Intergovernmental Relations

Program review and re-procurement of all programming


Initial evaluation model

Initial Evaluation model

  • Contracted out evaluation

  • Driven by external pressures

  • One at a time and intermittent

  • Minimal internal expertise to guide

  • Minimal linkage to decision-making processes


Challenges

Challenges

  • One-offs

    • Lack of ability to compare

    • Lack of consistency

    • Lack of strategic or system evaluation

  • Simple evaluations

    • Didn’t address complexity

  • Funding bias

  • Consultant-driven methodology

  • Lack of internal guidance/overview/quality control

  • Lack of direct link between evaluation and decisions


New approach

New approach

Internal evaluation team

Reporting to same director as program managers

External supports

Transparency of methodology


Program decision support

Program Decision Support

  • Evaluations designed specifically to answer program and costing questions

  • Embedded evaluators - working closely with program managers

  • Triangulated recommendations – program managers and evaluators putting forward recommendations

    • Supports evidence-based decision-making


Strategic evaluation

Strategic Evaluation

Beyond program-specific

How do these programs in total work to achieve our strategic goals?

How do these programs work together?

What are the broader pieces/horizontal approaches that impact on this suite of programs?

How do those external partnerships work?


Building the evidence base

Building the Evidence base

Testing service methodologies to be portable to other program areas/providers

Comparative – this works better than that

For which populations?

Value for money

Advanced analysis – e.g. what are the most important drivers of the desired outcome?


Putting down roots model

Putting Down Roots Model

  • Three key pathways have the strongest impact on recent immigrants’ sense of belonging and desire to stay in their communities.

  • Inclusive Community Path

  • Welcoming Community Path

  • Employment Equity Path

(Recent Immigrants, n = 999) 54% of Variance Explained


Lessons learned

Lessons Learned

  • Very effective at improving decision-making

  • Builds internal resources

  • Forces the internal conversation of evidence

  • Need to ensure transparency

  • Bring in outside evaluators where necessary

    • Skill sets

    • Data collection

  • Won’t work for all organizations


Wider implications

Wider implications

What decisions do the clients need to be able to make?

How can you support comparative funding decisions?

What are the opportunities for building a wider evidence-base?


Bringing evaluation in moving from evaluation to evidence based decision support

Questions?

Ben Pollard

[email protected]


  • Login