Telling the Difference between Snake Oil and Good Oil; Evaluation Capacity Building in the public se...
Sponsored Links
This presentation is the property of its rightful owner.
1 / 51

Bronte Koop, Principal Consultant, Strategy and Evaluation PowerPoint PPT Presentation


  • 87 Views
  • Uploaded on
  • Presentation posted in: General

Telling the Difference between Snake Oil and Good Oil; Evaluation Capacity Building in the public sector. Bronte Koop, Principal Consultant, Strategy and Evaluation John Collins, Manager, Program Evaluation. National Partnerships Literacy and Numeracy Teacher Quality Low SES communities.

Download Presentation

Bronte Koop, Principal Consultant, Strategy and Evaluation

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Telling the Difference between Snake Oil and Good Oil; Evaluation Capacity Building in the public sector

Bronte Koop, Principal Consultant, Strategy and Evaluation

John Collins, Manager, Program Evaluation


National Partnerships

Literacy and Numeracy

Teacher Quality

Low SES communities

COAG Reform Agenda: Human Capital Development


Not getting what you expected from an evaluation


What do we mean by Snake Oil

  • Staff as passive recipients

  • Evaluation questions not addressed

  • Use lots of words

  • Recommendations to continue the program and evaluate it again

  • Lack of evidence based recommendations for improvement

  • No discussion of data or design limitations

  • No discussion of alternative explanations for program effect

  • Evaluators reluctant to address client concerns about clarity or focus


Useful information to guide decision making


So what’s the Good Oil

  • Staff critical participants

  • Provides answers to specific questions

  • Balances qualitative with quantitative data

  • Reports are just sufficient in length

  • Conclusions follow from the evidence

  • Has logical recommendations

  • Data or design limitations are discussed

  • Alternative explanations for program effect are discussed


How do we get there?

teamwork

commitment

learning

proactivity


Attitudes and values

What are we trying to change?

Knowledge and skills

Organisational behaviour


Overcoming your fears is the first step


Principles for evidence based program management

  • Demand evidence

  • Examine the logic

  • Treat the program as an unfinished prototype

  • Embrace an attitude of wisdom

  • Jeffrey Pfeffer and Robert Sutton on evidence based management


CREATINGCHANGE

Where should be put our efforts?

  • Executive leadership

  • Consulting Services

  • Resources

  • Professional development

  • Policy and operations

  • Networks

  • Strategic Oversight


Locate near the top

Recruit the knowledge, skills and motivation

Build credibility

Provide an ‘evaluation perspective’

Consulting Services


Resources

If you build it, they will come


What have others done?


Accessible Resources


Practical Resources


Practical Resources

Provide a starting point


Project Management


Evaluation & planning


  • Professional Development

  • Time to stretch your thinking

    • Induction

    • Customized training to needs


Policy and Operations

Think outside the square

Procurement:

Expression of Interest

Two part contracts:

plan + do

Executive summaries

Interviews

Hard questions


Network, network, network


Strategic objective 2011

“We will foster a culture of evaluation and review to better direct our services.”


Strategic objective 2011

“We will foster a culture of evaluation and review to better direct our services.”


Strategic Plan 2012-2016

We take seriously our responsibility to invest public funds effectively and efficiently.

Our funding decisions will be guided by the potential of our policies and programs to improve learning outcomes . . .


Strategic Plan 2012-2016

We take seriously our responsibility to invest public funds effectively and efficiently.

Our funding decisions will be guided by the potential of our policies and programs to improve learning outcomes . . . current investments will be evaluated and we will reinvest where appropriate.


Strategic Plan 2012-2016

We take seriously our responsibility to invest public funds effectively and efficiently.

Our funding decisions will be guided by the potential of our policies and programs to improve learning outcomes . . . current investments will be evaluated and we will reinvest where appropriate.

DRAFT


  • Represents a major cultural change.


  • Represents a major cultural change

  • From an ‘accountability’ model to one centred on the corporate goal.


  • Represents a major cultural change

  • From an ‘accountability’ model to one centred on the corporate goal.

A tough call


How?


A marketing

approach


A marketing approach

Product life-cycle


A marketing approach

Product life-cycle


A marketing approach

Product life-cycle

Customer


A marketing approach

Product life-cycle

Customer

Needs and wants


A marketing approach

Product life-cycle

Customer

Needs and wants


A marketing approach

Product life-cycle

Customer

Needs and wants

The grudge purchase


A marketing approach

Product life-cycle

Customer

Needs and wants

The grudge purchase


A marketing approach

Product life-cycle

Customer

Needs and wants

The grudge purchase


A marketing approach

Product life-cycle

Customer

Needs and wants

The grudge purchase


A marketing approach

Product life-cycle

Customer

Needs and wants

The grudge purchase

Brands and brand equity


A marketing approach

Product life-cycle

Customer

Needs and wants

The grudge purchase

Brands and brand equity


A marketing approach

Product life-cycle

Customer

Needs and wants

The grudge purchase

Brands and brand equity

Competition


A marketing approach

Product life-cycle

Customer

Needs and wants

The grudge purchase

Brands and brand equity

Competition


A marketing approach

Product life-cycle

Customer

Needs and wants

The grudge purchase

Brands and brand equity

Competition

4 Ps – Profit Product Place Promotion


Priorities?

To continue development & refinement of our ‘system’

To address changing thefocus of our evaluation culture

To move from ‘good oil’ to ‘best value’ oil


Disclaimer

The views and interpretationsexpressed in this presentation are those of the authors anddo not necessarily reflect those of the Department of Education and Children's Services or the Government of South Australia


Contact

  • [email protected]

  • [email protected]


  • Login