Telling the Difference between Snake Oil and Good Oil; Evaluation Capacity Building in the public se...
This presentation is the property of its rightful owner.
Sponsored Links
1 / 51

Bronte Koop, Principal Consultant, Strategy and Evaluation PowerPoint PPT Presentation


  • 80 Views
  • Uploaded on
  • Presentation posted in: General

Telling the Difference between Snake Oil and Good Oil; Evaluation Capacity Building in the public sector. Bronte Koop, Principal Consultant, Strategy and Evaluation John Collins, Manager, Program Evaluation. National Partnerships Literacy and Numeracy Teacher Quality Low SES communities.

Download Presentation

Bronte Koop, Principal Consultant, Strategy and Evaluation

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Bronte koop principal consultant strategy and evaluation

Telling the Difference between Snake Oil and Good Oil; Evaluation Capacity Building in the public sector

Bronte Koop, Principal Consultant, Strategy and Evaluation

John Collins, Manager, Program Evaluation


Coag reform agenda human capital development

National Partnerships

Literacy and Numeracy

Teacher Quality

Low SES communities

COAG Reform Agenda: Human Capital Development


Bronte koop principal consultant strategy and evaluation

Not getting what you expected from an evaluation


Bronte koop principal consultant strategy and evaluation

What do we mean by Snake Oil

  • Staff as passive recipients

  • Evaluation questions not addressed

  • Use lots of words

  • Recommendations to continue the program and evaluate it again

  • Lack of evidence based recommendations for improvement

  • No discussion of data or design limitations

  • No discussion of alternative explanations for program effect

  • Evaluators reluctant to address client concerns about clarity or focus


Bronte koop principal consultant strategy and evaluation

Useful information to guide decision making


Bronte koop principal consultant strategy and evaluation

So what’s the Good Oil

  • Staff critical participants

  • Provides answers to specific questions

  • Balances qualitative with quantitative data

  • Reports are just sufficient in length

  • Conclusions follow from the evidence

  • Has logical recommendations

  • Data or design limitations are discussed

  • Alternative explanations for program effect are discussed


How do we get there

How do we get there?

teamwork

commitment

learning

proactivity


What are we trying to change

Attitudes and values

What are we trying to change?

Knowledge and skills

Organisational behaviour


Overcoming your fears is the first step

Overcoming your fears is the first step


Bronte koop principal consultant strategy and evaluation

Principles for evidence based program management

  • Demand evidence

  • Examine the logic

  • Treat the program as an unfinished prototype

  • Embrace an attitude of wisdom

  • Jeffrey Pfeffer and Robert Sutton on evidence based management


Bronte koop principal consultant strategy and evaluation

CREATINGCHANGE

Where should be put our efforts?

  • Executive leadership

  • Consulting Services

  • Resources

  • Professional development

  • Policy and operations

  • Networks

  • Strategic Oversight


Consulting services

Locate near the top

Recruit the knowledge, skills and motivation

Build credibility

Provide an ‘evaluation perspective’

Consulting Services


Bronte koop principal consultant strategy and evaluation

Resources

If you build it, they will come


What have others done

What have others done?


Accessible resources

Accessible Resources


Practical resources

Practical Resources


Practical resources1

Practical Resources

Provide a starting point


Project management

Project Management


Evaluation planning

Evaluation & planning


Bronte koop principal consultant strategy and evaluation

  • Professional Development

  • Time to stretch your thinking

    • Induction

    • Customized training to needs


Bronte koop principal consultant strategy and evaluation

Policy and Operations

Think outside the square

Procurement:

Expression of Interest

Two part contracts:

plan + do

Executive summaries

Interviews

Hard questions


Network network network

Network, network, network


Strategic objective 2011

Strategic objective 2011

“We will foster a culture of evaluation and review to better direct our services.”


Strategic objective 20111

Strategic objective 2011

“We will foster a culture of evaluation and review to better direct our services.”


Strategic plan 2012 2016

Strategic Plan 2012-2016

We take seriously our responsibility to invest public funds effectively and efficiently.

Our funding decisions will be guided by the potential of our policies and programs to improve learning outcomes . . .


Strategic plan 2012 20161

Strategic Plan 2012-2016

We take seriously our responsibility to invest public funds effectively and efficiently.

Our funding decisions will be guided by the potential of our policies and programs to improve learning outcomes . . . current investments will be evaluated and we will reinvest where appropriate.


Strategic plan 2012 20162

Strategic Plan 2012-2016

We take seriously our responsibility to invest public funds effectively and efficiently.

Our funding decisions will be guided by the potential of our policies and programs to improve learning outcomes . . . current investments will be evaluated and we will reinvest where appropriate.

DRAFT


Bronte koop principal consultant strategy and evaluation

  • Represents a major cultural change.


Bronte koop principal consultant strategy and evaluation

  • Represents a major cultural change

  • From an ‘accountability’ model to one centred on the corporate goal.


Bronte koop principal consultant strategy and evaluation

  • Represents a major cultural change

  • From an ‘accountability’ model to one centred on the corporate goal.

A tough call


Bronte koop principal consultant strategy and evaluation

How?


Bronte koop principal consultant strategy and evaluation

A marketing

approach


A marketing approach

A marketing approach

Product life-cycle


A marketing approach1

A marketing approach

Product life-cycle


A marketing approach2

A marketing approach

Product life-cycle

Customer


A marketing approach3

A marketing approach

Product life-cycle

Customer

Needs and wants


A marketing approach4

A marketing approach

Product life-cycle

Customer

Needs and wants


A marketing approach5

A marketing approach

Product life-cycle

Customer

Needs and wants

The grudge purchase


A marketing approach6

A marketing approach

Product life-cycle

Customer

Needs and wants

The grudge purchase


A marketing approach7

A marketing approach

Product life-cycle

Customer

Needs and wants

The grudge purchase


A marketing approach8

A marketing approach

Product life-cycle

Customer

Needs and wants

The grudge purchase


A marketing approach9

A marketing approach

Product life-cycle

Customer

Needs and wants

The grudge purchase

Brands and brand equity


A marketing approach10

A marketing approach

Product life-cycle

Customer

Needs and wants

The grudge purchase

Brands and brand equity


A marketing approach11

A marketing approach

Product life-cycle

Customer

Needs and wants

The grudge purchase

Brands and brand equity

Competition


A marketing approach12

A marketing approach

Product life-cycle

Customer

Needs and wants

The grudge purchase

Brands and brand equity

Competition


A marketing approach13

A marketing approach

Product life-cycle

Customer

Needs and wants

The grudge purchase

Brands and brand equity

Competition

4 Ps – Profit Product Place Promotion


Priorities

Priorities?

To continue development & refinement of our ‘system’

To address changing thefocus of our evaluation culture

To move from ‘good oil’ to ‘best value’ oil


Disclaimer

Disclaimer

The views and interpretationsexpressed in this presentation are those of the authors anddo not necessarily reflect those of the Department of Education and Children's Services or the Government of South Australia


Contact

Contact

  • [email protected]

  • [email protected]


  • Login