Evaluation on shifting ground
1 / 16

Evaluation on shifting ground - PowerPoint PPT Presentation

  • Uploaded on

Evaluation on shifting ground. Sonia Barnes, NZ Police Alison Chetwin, Independent Evaluator (formerly NZ Police). Presentation overview. Context Neighbourhood Policing – evaluation approach and changes to it, specific learning and contributions

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Evaluation on shifting ground' - rosine

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Evaluation on shifting ground
Evaluation on shifting ground

Sonia Barnes, NZ Police

Alison Chetwin, Independent Evaluator (formerly NZ Police)

Presentation overview
Presentation overview

  • Context

  • Neighbourhood Policing – evaluation approach and changes to it, specific learning and contributions

  • Mobility – evaluation approach and changes to it, specific learning and contributions

  • What we learnt


Policing Excellence

  • Suite of 9 initiatives aiming to improve efficiency and effectiveness of NZ Police and shift more effort into prevention

  • Implementation of change programme began in 2010 with formal project management processes and evaluation

  • Implementation completed in 2013, targets to be met by 2014/15


  • 2014/15: achievement of Policing Excellence aims

    • Increase preventative policing by 4%

    • Reduce crime and crash by 13%

    • Reduce prosecutions by 19%

  • 2017: contribute to Better Public Services targets

    • Reduce crime by 15%

    • Reduce violent crime rate by 20%

    • Reduce youth crime by 5%

Neighbourhood policing
Neighbourhood Policing

  • Neighbourhood Policing Teams:

    • Sergeant + 4-6

    • Vulnerable neighourhoods

    • Up to 9,000 residents

    • Community engagement, partnership working, problem solving

  • Aim to:

    • Reduce crime and crash and re-victimisation

    • Increase trust and confidence, satisfaction, feeling safe

Original evaluation approach
Original evaluation approach

  • South Auckland – to inform national roll-out

  • Desire for outcome evaluation

    • Two years before measure

    • Crime reduction

    • Perceptions of Police

  • Formative evaluation

    • Identification of outcomes and theory of action

    • Activities based on good practice

    • Early operation of teams

Adapting the evaluation to implementation changes
Adapting the evaluation to implementation changes

  • Accelerated implementation

    • Implementation in South Auckland late 2010 – early 2011

    • Evaluation commenced early 2011

    • Directive mid-2011 for 32 NPTs by end 2011

  • Adaptation of evaluation approach

    • Emphasis on formative evaluation and learning to inform further implementation

    • Advisory role on implementation team

    • Responsive to ad hoc evaluative requests

What the evaluation contributed
What the evaluation contributed

  • Wider awareness and utility of findings

  • Evidence based development of initiative:

    • International research evidence

    • Learning from local implementation

  • Implementation team sought advice from evaluation team on aspects of development

  • Education/ awareness raising of utility of evaluative work – raised profile of value of evaluation

Mobility initiative
Mobility initiative

  • Mobile devices to access and record into police systems while out of station

  • Aim to save time, improve safety, make police work more effective

  • Trial in four police areas to be evaluated

  • Emphasis on summative evaluation

Mobility trial
Mobility trial

Model of evaluator/implementer collaboration

  • Early involvement as independent expert adviser

  • Input into trial design to ensure evaluability

  • Reporting to allow time for outcomes to emerge

  • Integration between project management and evaluation

  • Formal agreement to evaluation framework

  • Investment decision to draw on evaluation evidence

Shifting timeline
Shifting timeline

Oct 2012 Business case for national roll-out

Jan 2013 National roll-out commences

Options changed timeframe
Options – changed timeframe

  • Continue with agreed evaluation plan

  • Seek reversion to agreed timeframe

  • Abandon evaluation

  • Adapt evaluation to new needs

Adapted design
Adapted design

  • Need to refine the trial

    • Formative real time information – surveys at completion of each roll-out

    • Informal verbal reports – surveys and observations

  • Need to inform national roll-out

    • Strengthen and bring forward qualitative component

    • Formal interim report and presentation

  • Need to inform investment decision

    • Retain summative evaluation

What we learnt
What we learnt

  • Importance and value of working with project management methodologies

  • Flexibility expands evaluation uses

    • Being prepared to circulate emerging findings leads to greater utility

    • Being willing to assist in developing the initiative results in initiative being more grounded in evidence

  • Summative evaluation needed for large investment decisions

  • Accelerated timeframes not always achievable

  • Demonstrate value of formative evaluation

Contact details
Contact details

Sonia Barnes

Senior Evaluator, NZ Police

[email protected]

Alison Chetwin

Independent Researcher & Evaluator

[email protected] clear.net.nz