philip davies international initiative for impact evaluation 3ie n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Philip Davies International Initiative for Impact Evaluation [3ie] PowerPoint Presentation
Download Presentation
Philip Davies International Initiative for Impact Evaluation [3ie]

Loading in 2 Seconds...

play fullscreen
1 / 19

Philip Davies International Initiative for Impact Evaluation [3ie] - PowerPoint PPT Presentation


  • 75 Views
  • Uploaded on

3ie-LIDC Seminar Series 'What Works In International Development’ London School of Hygiene and Tropical Medicine 20 th February 2013. Getting Evidence Into Policy. Philip Davies International Initiative for Impact Evaluation [3ie]. Outline of Presentation. How evidence is used in policy

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Philip Davies International Initiative for Impact Evaluation [3ie]' - istas


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
philip davies international initiative for impact evaluation 3ie

3ie-LIDC Seminar Series 'What Works In International Development’

London School of Hygiene and Tropical Medicine

20th February 2013

Getting Evidence Into Policy

Philip Davies

International Initiative for Impact Evaluation [3ie]

slide2

Outline of Presentation

  • How evidence is used in policy
  • Factors other than evidence
  • Some features of evidence
  • Different notions of evidence
  • Barriers to getting evidence into policy
  • Overcoming barriers to getting evidence into policy
instrumental use involves acting on research results in specific direct ways

How Evidence is Used in Policy Making

Conceptual Use Involves using research results for general enlightenment; results influence actions, but in less specific, more indirect ways than in instrumental use

Instrumental UseInvolves acting on research results in specific, direct ways.

Symbolic Use Involves using research results to legitimate and sustain pre-determined positions.

Source: Carol Weiss, 1982

slide4

How Evidence is Used in Policy Making

“Rarely does research supply an “answer” that policy actors employ to solve a policy problem. Rather, research provides a background of data, empirical generalisations, and ideas that affect the way that policy makers think about a problem.”

“But to acknowledge this is not the same as saying that research findings have little influence on policy.”

Source: Carol Weiss, 1982

slide5

Factors Other Than Evidence

Values and Decision Making Context

Pragmatics & Contingencies

Experience & Expertise

Lobbyists & Pressure Groups

Evidence

Judgement

Bureaucratic Culture

Resources

slide6

Some Features of Evidence

  • Evidence is almost always probabilistic
  • Evidence is often context specific
  • Often disagreement agreement on what counts as ‘evidence’
  • Evidence is always contestable/contested
  • Evidence is rarely self-evident
  • Not all research is of equal value/sufficient quality
  • Single studies can misrepresent the balance of evidence
  • Hence, the need for systematic reviews/synthesis of evidence
slide7

Different Notions of Evidence

Knowledge Translation and Transfer

Policy Makers’ Evidence

Researchers’ Evidence

  • Colloquial (Narrative)
  • Anything that seems reasonable
  • Policy relevant
  • Timely
  • Clear Message
  • ‘Scientific’ (Generalisable)
  • Proven empirically
  • Theoretically driven
  • As long as it takes
  • Caveats and qualifications

Source: J. Lomas et al, 2005

slide8

UK Policymakers’ Views of Evidence*

  • Focus on the ‘end product’, rather than how the information was either collected or analysed
  • Use of ‘anecdotal’ evidence
  • Drawing on such things as ‘real life stories’, ‘fingers in the wind’, ‘local’ and ‘bottom-up’ evidence
  • But:
  • “If we try and move anywhere without having the scientific basis to do so we get fleeced in the House”

*Source. Campbell, S. et al, 2007: Analysis for Policy, London, GSR.

slide9

Where Do UK Civil Servants Go For Evidence?

Sharks

Plankton

Academic/Evaluation Research?

slide10

Barriers to the Use of Evidence

  • Policymakers’ lack of familiarity with the research process
  • Researchers’ lack of familiarity with the policy process
  • Trust (lack of trust) of policymakers in researchers (vice versa)
  • Physical access to evidence
  • Cognitive access to evidence (i.e. lack of understanding)
  • Lack of clarity in the presentation of evidence
  • Timeliness and availability of evidence

Sources: Lomas, 2000; Petticrew et al, 2004; Lavis et al, 2005; Dobbins et al, 2007; Ouimet, et al, 2009; Rosenbaum, 2010

slide11

Overcoming Barriers to the Use of Evidence

    • Early and ongoing involvement of relevant decision makers
    • Interactions between researchers and policymakers increases the prospects for research use by policymakers
    • Identify and use interpersonal networksand face-to-face interactions
  • Identify willing and able knowledge brokers
  • Separate strategic from operational demands for evidence
  • Get policy makers to own the evidence – not just the policy

Sources: Lomas, 2000; Petticrew et al, 2004; Lavis et al, 2005; Dobbins et al, 2007; Ouimet, et al, 2009; Rosenbaum, 2010

slide12

Improving Communication of Evidence

  • Establish what the evidence says, and does not say
  • Establish the policy messages and policy implications
  • But avoid ‘recommendations’
  • Provide information about the costs and benefits, harms and risks of interventions/policies
  • Present contextual factors that affect local applicability
  • Be clear - plain English summary, unclouded by jargon
  • Find the ‘influencing moment’
  • Aligned to decision making timescales
  • Use a 1:3:25 format
the one in the 1 3 25 format
The ‘One’ in the 1:3:25 Format
  • A one page of main message bullets
  • The lessons decision makers can take from the research
  • Not a summary of findings
  • Suggest implications of findings
  • No details of methodology
the three in the 1 3 25 format
The ‘Three’ in the 1:3:25 Format
  • These are the key findings of the study
  • The classic Executive Summary
  • Condensed to serve the needs of the busy decision maker
  • Focus on how the study may be useful for policy
  • Some brief mention of methodology
  • Some implications of policy and practice
the 25 in the 1 3 25 format
The ‘25’ in the 1:3:25 Format

This should include:

  • Context/Background
  • Approach (Methodology in Appendices, not text)
  • Results
  • Implications
  • Knowledge gaps
  • References
  • Additional resources
  • Appendices
slide16

Summary

  • The routes of evidence to policy are usually indirect and delayed
  • There are factors other than evidence
  • Evidence is seldom definitive or invariant
  • Nor is it self-evident
  • There are different notions of evidence
  • Physical and cognitive access to evidence is important
  • Evidence has to be ‘worked’ into policy/practice
  • Clear communication evidence is essential
slide17

References

Campbell, S., et al, 2007

Analysis for Policy, London, GSR

Dobbins et al, 2007

Public Health Decision-Makers’ Informational Needs and Preferences for Receiving Research Evidence, Worldviews on Evidence-Based Nursing, 3rd Quarter, 156-163, New York, Wiley Inc.

Lavis et al, 2005

Towards systematic reviews that inform health care management and policy-making, Journal of Health Services Research and Policy, 10, Suppl 1 July 2005 S1:35.

Lomas, J., 2000

Connecting Research and Policy, Canadian Journal of Policy Research, Spring, 140-144

Lomas, J. Culyer, T., McCutcheon, C., McAuley, L., and Law, S., 2005

Conceptualizing and Combining Evidence for Health System Guidance, Final Report, Canadian Health Services Research Foundation, Ottawa.

.

slide18

References (continued)

Ouimet, M., Landry, R., Ziam, S., and Bédard, P., 2009

The absorption of research knowledge by public servants, Evidence and Policy, 5, 4, 331-350.

Petticrew, M., et al, 2004

Evidence for public health policy on inequalities: 1: The reality according to policymakers, Journal of Epidemiology and Community Health 2004;58:811-816

Rosenbaum, S.E., et al, 2010

Evidence summaries tailored to health policy-makers in low- and middle-income countries, Bulletin of the World Health Organization, 89, 1, 54-61.

Weiss, C.A., 1982

Policy research in the context of diffuse decision making, Journal of Higher Education, 53, 6, 619-639.

slide19

Thank you

Philip Davies

Email: pdavies@3ieimpact.org

+44 (0)207 958 8350

Visit www.3ieimpact.org