real world approaches to assessing the impact of research on policy n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Real world approaches to assessing the impact of research on policy PowerPoint Presentation
Download Presentation
Real world approaches to assessing the impact of research on policy

Loading in 2 Seconds...

play fullscreen
1 / 21

Real world approaches to assessing the impact of research on policy - PowerPoint PPT Presentation


  • 176 Views
  • Uploaded on

Real world approaches to assessing the impact of research on policy. Annette Boaz King’s College London 23 rd September 2010. Outline. Models of the relationship between research policy and practice Rationales for assessing impact What can we learn from the literature?

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Real world approaches to assessing the impact of research on policy' - klarika


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
real world approaches to assessing the impact of research on policy

Real world approaches to assessing the impact of research on policy

Annette Boaz

King’s College London

23rd September 2010

outline
Outline
  • Models of the relationship between research policy and practice
  • Rationales for assessing impact
  • What can we learn from the literature?
  • 10 research impact assessment case studies
  • Consider the implications for SSH research
models
Models

Relationship – two communities

Linear – relay race

Engineering

Diffusion

Systems – co-production

activities designed to promote research impact
Activities designed to promote research impact

Research briefings

Advisory groups

Knowledge brokers

Co-locating researchers and policy makers

Participation in Networks

Joint research projects

Seminars/ presentations

models1

Advisory groups

Models

Research briefings

Relationship – two communities

Linear – relay race

Joint research projects

Systems – co-production

effective knowledge transfer
Effective knowledge transfer

depends on

  • the nature of the research and its utility
  • the personal characteristics of both researchers and users
  • the links between researchers and users
  • the context for the application of the research

See Nutley, Walter and Davies, 2007, pp 81-83,

See also Overseas Development Institute

slide7

Moderating between stakeholders

Learning

Accountability

Why assess the impact of our research?

In its quest for excellence, the European Commission must and will encourage, promote

and support every effort to understand and monitor the quality of research at universities. 1

Promoting organizational achievements

Performance

1Commissioner Janez Potočnik, Assessing Europe’s University-Based Research, 2010

review question how do you best evaluate the impact of research on policy
Review question: How do you (best) evaluate the impact of research on policy?
  • Explored the websites of 30 organizations
  • Hand search of selected journals and reference lists
  • Search of SPRU and IDS library catalogues
  • Search of 10 research databases
  • 1987-2007
  • UK and international sources
slide9

Single study

Policy makers

Research programmes

Systematic reviews

Focusing the review on policy use

Research Users

Research

Initiatives to increase research use

slide10

Other includes: Brazil,

Denmark, Spain and Norway

how do people evaluate the impact of research programmes
How do people evaluate the impact of research programmes?

Wide range of methods and approaches (often in combination):

· Qualitative methods: semi-structured interviews, documentary analysis, field visits and observations

· Quantitative methods: surveys, bibliometrics and patent/new technology tracking

· Panels and peer review

· Workshops and focus groups

· Process tracking

ten impact assessment case studies

The Foundation for Research, Science and Technology (New Zealand)

French Ministry of Ecology, Energy, Sustainable Development and the Sea

Ten impact assessment case studies

Land and Water Australia

CGIAR

Stockholm Environment Institute

The Irish Environmental Protection Agency

Defra

Environment Agency (England and Wales)

Finnish Environment Institute

Swedish Environmental Protection Agency

finnish environmental institute
Finnish Environmental Institute

Embedded approach: Ten year independent evaluation, four year strategic evaluations, internal annual evaluations, annual researcher self evaluation

Goal: how to increase impact

Methods: multi-method

key themes
Key themes:
  • Diverse of objectives, scope and scale
  • Types of approach
  • Generic challenges
  • Challenges specific to environmental research
slide19

8 key questions (when designing an impact evaluation)

  • 1. What is your conceptual framework?
  • . What are the outcomes of interest?
  • . What methods will best explore the outcomes of interest?
  • . How do you address attribution?
  • . What is the direction of travel for the evaluation?
  • . Is this a mixed method approach, providing scope for triangulation?
  • . Will the methods selected capture context and the complexity?
  • . When might be the best time to conduct the evaluation?
  • These need to be considered in the light of the time, skills and resources available for the evaluation.
  • See Boaz et al 2009
my contact details annette boaz@kcl ac uk
My contact details: Annette.boaz@kcl.ac.uk

Some useful references and websites:

  • Boaz, A, Fitzpatrick, S and Shaw, B (2009) Assessing the impact of research on policy: a literature review. Science and Public Policy, 36(4): 255-270
  • Nutley S, Walter I, and Davies H (2007) Using evidence: How research can inform public services. Bristol: The Policy Press.
  • O’Leary, G, Boyle, R, Donlon, B and Sheils, L (2008) Providing a framework for accountability and learning in environmental research. Administration. 5 (4): 159-170

-The Overseas Development Institute RAPID programme: http://www.odi.org.uk/programmes/rapid/

- Wooding S, Nason E, Klautzer L, Rubin J, Hanney S and Grant J (2007) Policy and practice impacts of research funded by the ESRC. A case study of the Future of Work programme, approach and analysis. Cambridge: RAND Europe. http://www.rand.org/pubs/technical_reports/2007/RAND_TR435.pdf

slide21

Managing Editors: Annette Boaz, Division of Health and Social Care Research, King's College London, UK and David Gough, Institute of Education, University of London, UK Volume 6, 2010, 4 issues: January, May, August and November http://www.policypress.co.uk/journals_eap.asp