criminal justice evaluation in the uk l.
Skip this Video
Loading SlideShow in 5 Seconds..
Criminal justice evaluation in the UK PowerPoint Presentation
Download Presentation
Criminal justice evaluation in the UK

Loading in 2 Seconds...

play fullscreen
1 / 24

Criminal justice evaluation in the UK - PowerPoint PPT Presentation

  • Uploaded on

Criminal justice evaluation in the UK. Chris Fox Dan Ellingworth. Evidence-based policy. ‘Modern policy making’. Forward looking: Defining policy outcomes and taking a long term view

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Criminal justice evaluation in the UK' - maree

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
modern policy making
‘Modern policy making’
  • Forward looking: Defining policy outcomes and taking a long term view
  • Outward looking: Take account of national, European and international situation; learning from experience of other countries; recognising regional variations.
  • Innovative, flexible: Questioning established ways of dealing with things, encouraging new and creative ideas, identifying and managing risk.
  • Joining up: Joining up the work of different government departments; ensuring that implementation is part of the policy process.
  • Inclusive: Consulting those responsible for implementation and those affected by the policy; carrying out an impact assessment
  • Evidence based: Basing policy decisions and advice upon the best available evidence; ensuring evidence is available in an accessible and meaningful form.
  • Evaluated: Systematic evaluation of the effectiveness of policy.
  • Reviews: Policy constantly reviewed to ensure it is dealing with problems it was designed to solve.
  • Lessons learned: Learning from experience of what works and what does not

Cabinet Office (1999) Professional policy making for the twenty-first century London: Cabinet Office

what is evidence based policy
What is evidence-based policy?
  • Evidence-based policy (EBP). An approach that:
    • “helps people make well informed decisions about policies, programmes and projects by putting the best available evidence from research at the heart of policy development and implementation” (Davies 1999 quoted in Davies 2004)
  • Contrasted with opinion-based policy which:
    • “relies heavily on either the selective use of evidence (e.g. on single studies irrespective of quality) or on the untested views of individuals or groups, often inspired by ideological standpoints, prejudices, or speculative conjecture.” (Davies 2004)
opinion based policy
Opinion-based policy

Taken from Chalmers, I. (2003) ‘Campbell and Cochrane: the need for generosity of spirit and mutual support’ Jerry Lee Lecture: 3rd Annual Campbell Colloquiem

opinion based policy6
Opinion based policy

Taken from Chalmers, I. (2003) ‘Campbell and Cochrane: the need for generosity of spirit and mutual support’ Jerry Lee Lecture: 3rd Annual Campbell Colloquiem

what factors influence policy
What factors influence policy?

Taken from Davies, P. (2004) Is evidence-based government possible Jerry Lee Lecture 2004

an evaluation framework
An evaluation framework
  • Should it work?(Theory of change)
    • What is the underlying ‘theory of change’ which explains why the intervention will make an impact?
  • Can it work?(Implementation evaluation)
    • Has the project been properly implemented? What were the challenges to implementation and how were they overcome?
  • Does it work? (Impact evaluation)
    • What is the impact of the intervention?
  • Is it worth it?(Economic evaluation)
    • What are the resource implications of implementing the intervention and what benefits will it deliver?

Adapted from Haynes, B., 1999. BMJ; 319:652-653 ( 11 September )

should it work theories of change
Should it work? Theories of change
  • Sometimes programmes and projects:
    • don’t have clear aims or change their aims over time
    • are ‘shoe horned’ into a set of funding criteria to access that funding
    • don’t have clearly defined processes and structures
    • are implemented in a multi-agency context where different agencies will have different ideas about what the project is trying to achieve
theories of change
Theories of change
  • “What is the conceptual link from an intervention's inputs to the production of its outputs and, subsequently, to its impacts on society in terms of results and outcomes?”

(United Kingdom Evaluation Society Glossary of evaluation terms

scared straight
Scared straight
  • Started as US programme in 1970s
  • Target group: At risk or delinquent young people
  • Prison visits including tour and confrontational meeting with serving prisoners
  • Graphic depiction of life in prison
  • TV documentary in US extols its virtues
  • Also used in US
can it work process evaluation
Can it work? Process evaluation
  • Identify whether the intervention has been implemented as intended
  • Explore how a service or policy is delivered and experienced in practice
  • Identify the mechanisms by which it can produce the desired effects, potential barriers and facilitators
  • Identify circumstances under which successful operation might be replicated
  • James Lind (1716 – 1794): Ship’s Surgeon
  • Lind selected 12 men, all suffering from scurvy
  • Divided them into six pairs, giving each group different additions to their basic diet.
    • Cider
    • Seawater
    • A mixture of garlic, mustard and horseradish
    • Spoonfuls of vinegar
    • Oranges
    • Lemons.
  • Those fed citrus fruits experienced a remarkable recovery
  • In 1753, he published 'A Treatise of the Scurvy‘

does it work impact evaluation
Does it work? Impact evaluation

Sherman et al. (1998) Preventing Crime. What works, what doesn’t, what’s promising. National Institute of Justice.

measuring impact
Measuring impact
  • Choice of impact measures is often limited by availability of data or resources to collect primary data.
    • In the UK, re-offending data can be accessed via the Police National Computer (PNC) or the Offender Index
    • For less tangible outcomes such as family reconciliation or increased self-confidence the only option is often expensive face-to-face interviews/surveys.
finding a comparator
Finding a comparator
  • A comparator group or area may not be readily available.
    • While a programme or pilot area may be willing to cooperate with an evaluation an organisation that is not part of the evaluation but is being asked to provide data for a comparator area may be hard to persuade to cooperate.
    • A regional or national roll-out may limit the number of potential comparator areas
  • Programmes and projects often struggle to generate sufficient throughputs to allow for robust statistical analysis.
    • Set-up times are often under-estimated
    • The need for inter-agency cooperation is often under-estimated
  • Often evaluation timescales preclude appropriate follow-up periods to identify the long-term impact of interventions.
      • The Home Office recommends that reconviction studies use a 2 year follow-up period.
      • By the time a cohort to study has been generated this will often require an evaluation period of 3 – 4 years.
systematic reviews
Systematic Reviews
  • Systematic reviews are overviews of the existing research literature on a topic
  • A comprehensive search of print, electronic, and unpublished sources is made.
  • Sources identified are screened to see if they are relevant.
  • The quality of the sources (the strength of the evidence) is then assessed.
  • Sometimes the findings from individual studies are pooled in a process called meta-analysis
scared straight systematic review
Scared straight: systematic review
  • The systematic review shows that:
    • “The program increases the percentage of the treatment group committing new offences anywere from 1% to 30%.” (365) (emphasis added)

Petrosino, A., Turpin-Petrosino, C., and Finckenauer, J. (2000) ‘Well-Meaning Programs Can Have Harmful Effects! Lessons from Experiments of Programs Such as Scared Straight’ Crime Delinquency 46; 354


If so, Chris and Dan run a Third Year Module called ‘What works in social and criminal policy’ that looks in more detail at how policy is made and the role of evaluation in supporting the policy-making process.