Telling the Difference between Snake Oil and Good Oil; Evaluation Capacity Building in the public se...
Download
1 / 51

Bronte Koop, Principal Consultant, Strategy and Evaluation - PowerPoint PPT Presentation


  • 118 Views
  • Uploaded on

Telling the Difference between Snake Oil and Good Oil; Evaluation Capacity Building in the public sector. Bronte Koop, Principal Consultant, Strategy and Evaluation John Collins, Manager, Program Evaluation. National Partnerships Literacy and Numeracy Teacher Quality Low SES communities.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Bronte Koop, Principal Consultant, Strategy and Evaluation' - italia


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Telling the Difference between Snake Oil and Good Oil; Evaluation Capacity Building in the public sector

Bronte Koop, Principal Consultant, Strategy and Evaluation

John Collins, Manager, Program Evaluation


Coag reform agenda human capital development

National Partnerships Evaluation Capacity Building in the public sector

Literacy and Numeracy

Teacher Quality

Low SES communities

COAG Reform Agenda: Human Capital Development


Not getting what you expected from an evaluation Evaluation Capacity Building in the public sector


What do we mean by Snake Oil Evaluation Capacity Building in the public sector

  • Staff as passive recipients

  • Evaluation questions not addressed

  • Use lots of words

  • Recommendations to continue the program and evaluate it again

  • Lack of evidence based recommendations for improvement

  • No discussion of data or design limitations

  • No discussion of alternative explanations for program effect

  • Evaluators reluctant to address client concerns about clarity or focus


Useful information to guide decision making Evaluation Capacity Building in the public sector


So what’s the Good Oil Evaluation Capacity Building in the public sector

  • Staff critical participants

  • Provides answers to specific questions

  • Balances qualitative with quantitative data

  • Reports are just sufficient in length

  • Conclusions follow from the evidence

  • Has logical recommendations

  • Data or design limitations are discussed

  • Alternative explanations for program effect are discussed


How do we get there
How do we get there? Evaluation Capacity Building in the public sector

teamwork

commitment

learning

proactivity


What are we trying to change

Attitudes and values Evaluation Capacity Building in the public sector

What are we trying to change?

Knowledge and skills

Organisational behaviour


Overcoming your fears is the first step
Overcoming your fears is the first step Evaluation Capacity Building in the public sector


Principles for evidence based program management Evaluation Capacity Building in the public sector

  • Demand evidence

  • Examine the logic

  • Treat the program as an unfinished prototype

  • Embrace an attitude of wisdom

  • Jeffrey Pfeffer and Robert Sutton on evidence based management


CREATING Evaluation Capacity Building in the public sectorCHANGE

Where should be put our efforts?

  • Executive leadership

  • Consulting Services

  • Resources

  • Professional development

  • Policy and operations

  • Networks

  • Strategic Oversight


Consulting services

Locate near the top Evaluation Capacity Building in the public sector

Recruit the knowledge, skills and motivation

Build credibility

Provide an ‘evaluation perspective’

Consulting Services


Resources Evaluation Capacity Building in the public sector

If you build it, they will come


What have others done
What have others done? Evaluation Capacity Building in the public sector


Accessible resources
Accessible Resources Evaluation Capacity Building in the public sector


Practical resources
Practical Resources Evaluation Capacity Building in the public sector


Practical resources1
Practical Resources Evaluation Capacity Building in the public sector

Provide a starting point


Project management
Project Management Evaluation Capacity Building in the public sector


Evaluation planning
Evaluation & planning Evaluation Capacity Building in the public sector


  • Professional Development Evaluation Capacity Building in the public sector

  • Time to stretch your thinking

    • Induction

    • Customized training to needs


Policy and Operations Evaluation Capacity Building in the public sector

Think outside the square

Procurement:

Expression of Interest

Two part contracts:

plan + do

Executive summaries

Interviews

Hard questions


Network network network
Network, network, network Evaluation Capacity Building in the public sector


Strategic objective 2011
Strategic objective 2011 Evaluation Capacity Building in the public sector

“We will foster a culture of evaluation and review to better direct our services.”


Strategic objective 20111
Strategic objective 2011 Evaluation Capacity Building in the public sector

“We will foster a culture of evaluation and review to better direct our services.”


Strategic plan 2012 2016
Strategic Plan 2012-2016 Evaluation Capacity Building in the public sector

We take seriously our responsibility to invest public funds effectively and efficiently.

Our funding decisions will be guided by the potential of our policies and programs to improve learning outcomes . . .


Strategic plan 2012 20161
Strategic Plan 2012-2016 Evaluation Capacity Building in the public sector

We take seriously our responsibility to invest public funds effectively and efficiently.

Our funding decisions will be guided by the potential of our policies and programs to improve learning outcomes . . . current investments will be evaluated and we will reinvest where appropriate.


Strategic plan 2012 20162
Strategic Plan 2012-2016 Evaluation Capacity Building in the public sector

We take seriously our responsibility to invest public funds effectively and efficiently.

Our funding decisions will be guided by the potential of our policies and programs to improve learning outcomes . . . current investments will be evaluated and we will reinvest where appropriate.

DRAFT




A tough call


How? Evaluation Capacity Building in the public sector


A marketing Evaluation Capacity Building in the public sector

approach


A marketing approach
A marketing approach Evaluation Capacity Building in the public sector

Product life-cycle


A marketing approach1
A marketing approach Evaluation Capacity Building in the public sector

Product life-cycle


A marketing approach2
A marketing approach Evaluation Capacity Building in the public sector

Product life-cycle

Customer


A marketing approach3
A marketing approach Evaluation Capacity Building in the public sector

Product life-cycle

Customer

Needs and wants


A marketing approach4
A marketing approach Evaluation Capacity Building in the public sector

Product life-cycle

Customer

Needs and wants


A marketing approach5
A marketing approach Evaluation Capacity Building in the public sector

Product life-cycle

Customer

Needs and wants

The grudge purchase


A marketing approach6
A marketing approach Evaluation Capacity Building in the public sector

Product life-cycle

Customer

Needs and wants

The grudge purchase


A marketing approach7
A marketing approach Evaluation Capacity Building in the public sector

Product life-cycle

Customer

Needs and wants

The grudge purchase


A marketing approach8
A marketing approach Evaluation Capacity Building in the public sector

Product life-cycle

Customer

Needs and wants

The grudge purchase


A marketing approach9
A marketing approach Evaluation Capacity Building in the public sector

Product life-cycle

Customer

Needs and wants

The grudge purchase

Brands and brand equity


A marketing approach10
A marketing approach Evaluation Capacity Building in the public sector

Product life-cycle

Customer

Needs and wants

The grudge purchase

Brands and brand equity


A marketing approach11
A marketing approach Evaluation Capacity Building in the public sector

Product life-cycle

Customer

Needs and wants

The grudge purchase

Brands and brand equity

Competition


A marketing approach12
A marketing approach Evaluation Capacity Building in the public sector

Product life-cycle

Customer

Needs and wants

The grudge purchase

Brands and brand equity

Competition


A marketing approach13
A marketing approach Evaluation Capacity Building in the public sector

Product life-cycle

Customer

Needs and wants

The grudge purchase

Brands and brand equity

Competition

4 Ps – Profit Product Place Promotion


Priorities
Priorities? Evaluation Capacity Building in the public sector

To continue development & refinement of our ‘system’

To address changing thefocus of our evaluation culture

To move from ‘good oil’ to ‘best value’ oil


Disclaimer
Disclaimer Evaluation Capacity Building in the public sector

The views and interpretationsexpressed in this presentation are those of the authors anddo not necessarily reflect those of the Department of Education and Children's Services or the Government of South Australia


Contact
Contact Evaluation Capacity Building in the public sector

  • [email protected]

  • [email protected]


ad