15 10 planning your review
1 / 32

15-10 Planning Your Review - PowerPoint PPT Presentation

  • Uploaded on

15-10 Planning Your Review. Andrew Booth, Reader in Evidence Based Information Practice, Co-Convenor – Cochrane Collaboration Qualitative Methods Group. Process of Qualitative Evidence Synthesis (Major & Savin-Baden 2010). Identify Studies related to research question ↓

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' 15-10 Planning Your Review' - tudor

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
15 10 planning your review

15-10 Planning Your Review

Andrew Booth, Reader in Evidence Based Information Practice, Co-Convenor – Cochrane Collaboration Qualitative Methods Group

Process of qualitative evidence synthesis major savin baden 2010
Process of Qualitative Evidence Synthesis (Major & Savin-Baden 2010)

Identify Studies related to research question

Collate Qualitative Studies across chosen topic

Examine theories and methods used in each study in-depth

Compare and Analyse findings for Each Study

Synthesise findings for each study

Undertake interpretation of findings across studies

Present interpretive narrative about synthesis of findings

Provide series of recommendations

Stages of a qualitative evidence synthesis
Stages of a Qualitative Evidence Synthesis? Savin-Baden 2010)

Formulating the review question

Conducting a systematic literature search

Screening and selecting appropriate research articles

Analyzing and synthesizing qualitative findings

Maintaining quality control

Presenting findings

(Sandelowski & Barroso, 2007)

Garside, 2010 Savin-Baden 2010), PhD Thesis

Eight key questions
Eight Savin-Baden 2010)Key Questions

  • THE QUESTION: Starting Point or Early Outcome of Review?

  • THE QUESTION: Comprehensive Searching versus Sampling

  • THE DATA: Participants’ Comments and/or Authors’ Interpretations

  • THE DATA: Qualitative Data Versus Qualitative Research

  • STUDY QUALITY: Rich and Thick?

  • STUDY QUALITY: Appraisal for Exclusion or Moderation

  • ROLE OF THEORY: Theory Secure or Evolving

  • ROLE OF THEORY: Theory Generating/Theory Validating/Other

The question
The Question Savin-Baden 2010)

Is your question
Is Your Question…… Savin-Baden 2010)

  • Fixed? – Pre-defined as a PICO (Population-Intervention-Comparison-Outcome) or SPICE (Setting-Perspective- Interest, Phenomenon of – Comparison- Evaluation) – Question is an “Anchor”

  • (e.g. attached to an Effectiveness review)

  • Negotiable? – To be explored as part of initial review process – Becomes clearer as you examine data (cp. Grounded theory approaches) – Question is a “Compass”

  • NB. Even answering a fixed PICO question for HTAs may require exploration of phenomenon of untreated/pretreated condition (Lorenc et al, 2012)

Will you describe or interpret
Will You Describe or Interpret? Savin-Baden 2010)

All Reviews figure on continuum between Description and Interpretation

  • Description – What does the data say? – factual reporting of “epidemiology” of studies, themes etc…

  • Reader does work of interpretation

  • Interpretation – What does the data mean? – “diagnosis” – subjective interpretation of “signs and symptoms” from data and themes etc…

  • Reviewer does work of interpretation – may be contested

The data study quality
The Data/STUDY QUALITY Savin-Baden 2010)

A caution
A Caution! Savin-Baden 2010)

  • “it was found to be necessary…to include evidence not relating directly to interventions. In this respect, these reviews appear to be generally representative of the field of public health, where relatively little substantive qualitative evidence on specific interventions is available…..this is probably the case in many areas of social and health research. If so, limiting inclusion to qualitative studies of interventions alone will not be a practicable course of action, due to the lack of data (cf. Garside et al., 2009a)” (Lorenc et al, 2012).

Data study quality
Data/Study Quality Savin-Baden 2010)

  • How Rich (“Thick”) is Your Data?

    • Qualitative data from “thin” studies (or textual responses to surveys) will not sustain interpretive approaches

    • Rich/“Thick” reports will sustain interpretive approaches – may allow selective sampling/ theoretical saturation

  • How Will You Use Quality Assessment?

    • To Exclude Studies? (May be a luxury you cannot afford)

    • To Moderate Study Findings? (Will you examine which findings are supported by which quality studies? – Qualitative Sensitivity Analysis)

Role of theory
ROLE OF THEORY Savin-Baden 2010)

Will you generate explore test theory gough et al 2012
Will You Generate, Explore, Test Theory (Gough et al, 2012)? Savin-Baden 2010)

  • Generate – may require “suspension of disbelief” – quality assessment/value judgement may come later (cp. Brainstorming)-

  • Explore – looking for patterns

  • Test – quality assessment differentiates well-supported and unsupported data

  • NB. We (Carroll & Booth) are currently conducting empirical work on systematic identification of Theories

  • Pragmatic outputs e.g. HTAs or Clinical Guidelines may not require theory

  • Other outputs may have theory generation as a major objective

What are your choices
What Are Your Choices? Savin-Baden 2010)

Dixon Woods et al, 2004

Booth et al 2011

Ring et al, 2011

Gough et al, 2012

Hannes & Lockwood, 2011

Pope et al, 2007

Toolbox texts
Toolbox Texts Savin-Baden 2010)

  • Popay J, Roberts H, Sowden A et al. (2006) Guidance on the Conduct of Narrative Synthesis in Systematic Reviews: a Product from the ESRC Methods Programme.

  • Barnett-Page E, Thomas J. Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol. 2009 Aug 11;9:59. http://www.biomedcentral.com/1471-2288/9/59

  • Barnett-Page, E and Thomas, J (2009) Methods for the synthesis of qualitative research: a critical review. NCRM Working Paper. NCRM. (Unpublished) http://eprints.ncrm.ac.uk/690/1/0109%2520Qualitative%2520synthesis%2520methods%2520paper%2520NCRM.pdf

  • Dixon-Woods M, Agarwal S, Young B et al (2006). Integrative approaches to qualitative and quantitative evidence. Health Development Agency http://www.nice.org.uk/niceMedia/pdf/Integrative_approaches_evidence.pdf

  • Pope C, Mays N & Popay J.Synthesizing Qualitative and Quantitative Health Evidence: A Guide to Methods. ISBN: ISBN: 033521956X Open University Press.

  • Thomas J, Harden A (2008) Methods for the thematic synthesis of qualitative research in systematic reviews BMC Medical Research Methodology 8:45http://www.biomedcentral.com/1471-2288/8/45

Decisions decisions
Decisions? Decisions! Savin-Baden 2010)

Hannes, K., & Lockwood, C. (2011). Synthesizing qualitative research: choosing the right approach. BMJ Books.

What expertise can you access
What Expertise Can You Access? Savin-Baden 2010)

  • Expertise in Qualitative Research Methods (e.g. Grounded Theory; Framework Analysis, Thematic Analysis)

  • Expertise in Synthesis Methods (incl. Searching, Data Extraction, Quality Assessment, Interpretation)

  • Knowledge of Topic Area


How long how much have you got
How Long?/How Much Have You Got? Savin-Baden 2010)

  • “Richer” approaches make fuller use of data – require fewer studies

  • Meta-Aggregation, Thematic Synthesis can handle large numbers of studies

32 papers (775 patients and carers) reporting help-seeking experiences for at least 20 different types of cancer.

Other considerations
Other Considerations Savin-Baden 2010)

  • Relationship with Quantitative Syntheses

    • Pre-existing Review

    • Sequentially

    • In Parallel

    • [Iteratively]

    • [Combined Methods versus Separate Methods]

  • To include: Any Qualitative Research? OR Specific methods? OR only well-described methods and thick detail of findings?

What does the field look like
What does the Field look like? Savin-Baden 2010)

  • Some Tools

    • PubMed Health Services Research Special Queries


    • PubMed Reminer

    • http://bioinfo.amc.uva.nl/human-genetics/pubreminer/

Similar reviews http www mendeley com groups 518691 cochrane qes register
Similar Reviews? Savin-Baden 2010)http://www.mendeley.com/groups/518691/cochrane-qes-register/

Some practicalities
Some Practicalities Savin-Baden 2010)

  • Use of/Selection of Methodological Filters

  • Key studies for Citation Searching

  • Examples of Data Extraction Forms

  • Selection of Critical Appraisal Checklist

  • Innovative Ways of Presentation

Conclusions Savin-Baden 2010)

  • Question may involve experience of Condition as well as Intervention studies

  • Scoping is Time Well-Spent

  • Searching will be more challenging

  • Sifting will be more time-consuming

  • Allow extra time for Interpretation (Synthesis is not an End but a Means!)

References 1
References - 1 Savin-Baden 2010)

  • Barnett-Page E, Thomas J. Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol. 2009 Aug 11;9:59.

  • Booth, A, Papaioannou, D and Sutton, A J (2011). Systematic Approaches to a Successful Literature Review. SAGE publications

  • Candy B, King M, Jones L, Oliver S. Using qualitative synthesis to explore heterogeneity of complex interventions. BMC Med Res Methodol. 2011 Aug 26;11:124.

  • Dixon-Woods M, Agarwal S, Young B, Jones D, Sutton A. (2004) Integrative approaches to qualitative and quantitative evidence. London: Health Development Agency

  • Gough, D, Oliver, S, Thomas J (2012) An Introduction to Systematic Reviews. London: Sage Publications.

  • Lorenc, T., Pearson, M., Jamal, F., Cooper, C. and Garside, R. (2012), The role of systematic reviews of qualitative evidence in evaluating interventions: a case study. Res. Synth. Method, 3: 1–10.

References 2
References - 2 Savin-Baden 2010)

  • PopayJ, Roberts H, Sowden A, Pettticrew M, Arai L, Rodgers M, Britten N: Guidance on the conduct of narrative synthesis in systematic reviews. http://www.lancs.ac.uk/fass/projects/nssr/2007 .

  • Pope C, Mays N, Popay J: Synthesizing Qualitative and Quantitative Health Evidence: a Guide to Methods. Maidenhead: Open University Press; 2007.

  • Ring N., Ritchie K, Mandava L, Jepson R. (2011) A guide to synthesising qualitative research for researchers undertaking health technology assessment and systematic reviews. NHS Quality Improvement Scotland and University of Stirling, Edinburgh.  

  • Snilstveit, B., Oliver, S., & Vojtkova, M. (2012). Narrative approaches to systematic review and synthesis of evidence for international development policy and practice. Journal of development effectiveness, 4(3), 409-429.

  • Thomas J, Harden A (2009) Methods for the thematic synthesis of qualitative research in systematic reviews, BMC Medical Research Methodology 8:45