Systematic review module 3 study eligibility criteria
This presentation is the property of its rightful owner.
Sponsored Links
1 / 32

Systematic Review Module 3: Study Eligibility Criteria PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

Systematic Review Module 3: Study Eligibility Criteria. Melissa McPheeters , PhD, MPH Associate Director Vanderbilt University Evidence-based Practice Center. Learning Objectives. To understand the role of selection criteria in framing a systematic review

Download Presentation

Systematic Review Module 3: Study Eligibility Criteria

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Systematic review module 3 study eligibility criteria

Systematic Review Module 3: Study Eligibility Criteria

Melissa McPheeters, PhD, MPH

Associate Director

Vanderbilt University Evidence-based Practice Center

Learning objectives

Learning Objectives

  • To understand the role of selection criteria in framing a systematic review

  • To know when and how to set selection criteria

  • To understand the effect of selection criteria on interpretation of a review

Cer process overview

CER Process Overview

Study selection criteria

Study Selection Criteria

  • Function the same in systematic reviews as in primary research

  • Should reflect the analytic framework and key questions

  • Are powerful tools for widening or narrowing the scope of a review

  • Provide information to determine whether reviews can be compared or combined

Some example criteria

Some Example Criteria

  • Adult, community-dwelling females

  • Study of a screening tool for depression

  • United States only

  • Hospital-based studies only

  • N > 200

  • Randomized controlled trials

Using broad criteria

Using Broad Criteria

  • Can be as broad as all related studies

  • Helpful for exploring “what is known”

  • May result in too much literature to feasibly review or disparate literature that cannot be compared

Using narrow criteria

Using Narrow Criteria

  • May return too little literature

  • Can result in inability to answer the intended question

  • Helpful in culling homogenous literature

  • Can reduce size of the literature to a manageable scope

Bias in this context

Bias in this Context

  • Distortion of the estimate of effect that comes from how studies are selected for inclusion

  • Affects the applicability or “external validity” of the review itself

Examples of bias in this context

Examples of Bias in this Context

  • Included studies may not have been conducted in the patient population whose care the review is intended to affect

    • e.g., the use of studies of twin pregnancies in a review of preterm labor management for low-risk women

  • Selection criteria may be set to include more of a certain study type that either overestimates or underestimates effectiveness

Selecting criteria

Selecting Criteria

  • Review study goals

  • Assess analytic framework and key questions

  • Tie criteria to PICOTS

  • Set criteria before beginning abstract review

Basic questions

Basic Questions

  • What is the relevant population?

  • What is the intervention of interest?

  • To what exposure is the intervention being compared?

  • What outcomes are relevant?

  • Should time to outcome be limited?

  • In what setting should the results be applicable?

Exercise 1

Exercise 1

  • What would you do if you were asked to review the literature on transition support for adolescents with autism spectrum disorders (ASD) entering adulthood?

  • Before seeing the key questions, consider the categories of criteria that we will want to apply.

Apply picots


  • Population—condition, disease severity and stage, comorbidities, patient demographics

  • Intervention—dosage, frequency, method of administration

  • Comparators—placebo, usual care, or active control

  • Outcomes—health outcomes, morbidity, mortality, quality of life (QoL)

  • Timing—Duration of followup

  • Setting—Primary, specialty, in-patient, cointerventions



  • Population

  • Intervention

  • Comparators

  • Outcomes

  • Timing

  • Setting

  • What constitutes an adolescent? What constitutes a diagnosis of ASD?

  • How is transition support defined?

  • Do we compare to no transition support or directly compare types of support?

  • What are the goals for adolescents with ASD as they transition to adulthood? Should they be individually focused?

  • How quickly should the outcomes be apparent?

  • Is transition support provided in multiple settings, such as schools, clinics, and the community?

What would you do with

What Would You Do with…

  • A paper that was about “individuals over age 10”?

  • A paper that was about an intervention for individuals with a range of developmental disabilities?

  • Or, conversely, a paper that was specifically about children with Asperger’s syndrome but not other ASDs?

Example of a narrow scope

Example of a Narrow Scope

  • What is the efficacy of home uterine activity monitoring for preventing preterm birth among women at low risk of a preterm birth?

Implications of a question with narrow scope

Implications of a Question with Narrow Scope

  • Efficacy: RCTs only

  • Low risk: no prior preterm birth

  • No. of studies: 11

Overactive bladder study

Overactive Bladder Study

  • Management of OAB among women

  • Considerations

    • OAB is a fairly difficult condition to define

    • Treatments include pharmacologic, behavioral, complementary and alternative medicine (CAM) and procedural—and each area includes very different types of studies

    • Study of OAB is often combined with other types of urological conditions, such as stress incontinence or prostate issues

Exercise 2

Exercise 2

  • Set two criteria and consider the potential implications

    • Minimum study size

    • Gender of study participants

Study size

Study Size

  • 50 at study start

  • Implications

    • Excluded for size only:79

    • Excluded for N < 20:36

    • Excluded for N 20 to 29:23

    • Excluded for N 30 to 39:8

    • Excluded for N 40 to 49:12



  • Studies had to include at least 75% women

  • This decision was based on expert opinion and the size and scope of the literature

  • 40 studies were excluded with less than 75% women

  • 27 additional studies would have been excluded had the review been limited to studies of only women

Other considerations

Other Considerations

  • What study designs should be included?

  • Include foreign studies? Other languages? Studies conducted in specific parts of the world?

  • Include “grey” or “fugitive” literature?

Types of studies

Types of Studies?

  • Limit to RCTs?

  • Include observational studies?

    • If so, which kinds?

  • What is the value of a case series?

  • What counts as a case series?

Example of maternal fetal surgery technical brief

Example of Maternal-fetal Surgery Technical Brief

  • Included case series with N ≥ 2

  • Only 3 of 169 studies were RCTs, and 122 were case series

  • Because of the relative newness of this area of research, it was important to capture data even from studies without comparison groups

Observational studies

Observational Studies

  • Types

    • Cohorts (with comparisons)

    • Case controls

    • Case series

    • Registries/databases

Observational studies1

Observational Studies

  • Well-done observational studies can address issues of applicability and the need for longer-term outcomes if they:

    • Include more representative patient populations

    • Have relevant comparators

    • Report more meaningful clinical outcomes over longer time frames

  • Observational studies may be a better source of information about harms

Foreign literature

Foreign Literature

  • Positive findings may be more likely to be published in high-profile journals published in English

  • Therefore, to include only English-language journals may overestimate the positive effect of an intervention

  • Empirically, the bias associated with limiting one’s review to English has been shown to be small (Moher et al., 2000; Gregoire et al., 1995)

Systematic review on cesarean delivery

Systematic Review on Cesarean Delivery

  • Systematic review on outcomes of cesarean delivery on maternal request

  • Conducted for the National Institutes of Health-Office of Medical Applications of Research State-of-the-Science conference

Exercise 3

Exercise 3

  • Define the appropriate population group and comparator.

  • What other limitations would you put on included literature?

The challenge

The Challenge

  • No evidence on outcomes of CDMR vs. other modes of delivery

  • Urgent need for actionable evidence

  • Need to recognize and account for confounders 



  • Expand search to include proxies

  • Weight rungs of evidence to account for confounding

    • Highest rung: Trials of breech delivery, but only for maternal outcomes

    • Next rung: Planned cesarean vs. planned vaginal

    • Lowest rung: Comparisons of maternal and neonatal outcomes from actual modes of delivery



  • Selection criteria are essential for setting the scope of the review

  • They should be tied to the analytic framework, key questions, and PICOTS

  • When properly applied, selection criteria can reduce bias and support the applicability of the review

  • Login