Bringing evaluation in moving from evaluation to evidence based decision support
Sponsored Links
This presentation is the property of its rightful owner.
1 / 13

Bringing Evaluation In: Moving from Evaluation to Evidence-based Decision Support PowerPoint PPT Presentation


  • 54 Views
  • Uploaded on
  • Presentation posted in: General

Bringing Evaluation In: Moving from Evaluation to Evidence-based Decision Support. Ben Pollard Director, VPS Portfolio Initiatives, UBC Former Director, Program Management and Evaluation, Immigrant Integration Branch. Immigrant Integration Overview.

Download Presentation

Bringing Evaluation In: Moving from Evaluation to Evidence-based Decision Support

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Bringing Evaluation In: Moving from Evaluation to Evidence-based Decision Support

Ben Pollard

Director, VPS Portfolio Initiatives, UBC

Former Director, Program Management and Evaluation, Immigrant Integration Branch


Immigrant Integration Overview

  • All facets of settlement and integration of newcomers

    • Settlement and orientation

    • Language

    • Labour market and Foreign Credential Recognition

    • Community integration

    • Community based, school-based, etc.

  • Diversity of immigrants (~40,000 annually)

  • 25+ program

  • 300 contracts with 100+ service providers

  • $140m annual programming budget


Context for Evaluation

Significant Investment

Multiple complex, diverse lines of business

Multiple stakeholders

High visibility

Fed/Prov Agreement and Intergovernmental Relations

Program review and re-procurement of all programming


Initial Evaluation model

  • Contracted out evaluation

  • Driven by external pressures

  • One at a time and intermittent

  • Minimal internal expertise to guide

  • Minimal linkage to decision-making processes


Challenges

  • One-offs

    • Lack of ability to compare

    • Lack of consistency

    • Lack of strategic or system evaluation

  • Simple evaluations

    • Didn’t address complexity

  • Funding bias

  • Consultant-driven methodology

  • Lack of internal guidance/overview/quality control

  • Lack of direct link between evaluation and decisions


New approach

Internal evaluation team

Reporting to same director as program managers

External supports

Transparency of methodology


Program Decision Support

  • Evaluations designed specifically to answer program and costing questions

  • Embedded evaluators - working closely with program managers

  • Triangulated recommendations – program managers and evaluators putting forward recommendations

    • Supports evidence-based decision-making


Strategic Evaluation

Beyond program-specific

How do these programs in total work to achieve our strategic goals?

How do these programs work together?

What are the broader pieces/horizontal approaches that impact on this suite of programs?

How do those external partnerships work?


Building the Evidence base

Testing service methodologies to be portable to other program areas/providers

Comparative – this works better than that

For which populations?

Value for money

Advanced analysis – e.g. what are the most important drivers of the desired outcome?


Putting Down Roots Model

  • Three key pathways have the strongest impact on recent immigrants’ sense of belonging and desire to stay in their communities.

  • Inclusive Community Path

  • Welcoming Community Path

  • Employment Equity Path

(Recent Immigrants, n = 999) 54% of Variance Explained


Lessons Learned

  • Very effective at improving decision-making

  • Builds internal resources

  • Forces the internal conversation of evidence

  • Need to ensure transparency

  • Bring in outside evaluators where necessary

    • Skill sets

    • Data collection

  • Won’t work for all organizations


Wider implications

What decisions do the clients need to be able to make?

How can you support comparative funding decisions?

What are the opportunities for building a wider evidence-base?


Questions?

Ben Pollard

[email protected]


  • Login