systematic reviews of health promotion and public health interventions l.
Skip this Video
Loading SlideShow in 5 Seconds..
Systematic reviews of health promotion and public health interventions PowerPoint Presentation
Download Presentation
Systematic reviews of health promotion and public health interventions

Loading in 2 Seconds...

play fullscreen
1 / 91

Systematic reviews of health promotion and public health interventions - PowerPoint PPT Presentation

  • Uploaded on

Systematic reviews of health promotion and public health interventions. Rebecca Armstrong Elizabeth Waters Cochrane Health Promotion & Public Health Field. Overview. Overview of systematic reviews Outline of The Cochrane Collaboration Role of the HPPH Field

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Systematic reviews of health promotion and public health interventions' - tino

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
systematic reviews of health promotion and public health interventions

Systematic reviews of health promotion and public health interventions

Rebecca Armstrong

Elizabeth Waters

Cochrane Health Promotion & Public Health Field

  • Overview of systematic reviews
  • Outline of The Cochrane Collaboration
  • Role of the HPPH Field
  • Function of systematic reviews in informing policy and practice
  • Key elements of systematic reviews
    • Asking answerable questions
    • Searching for evidence
    • Assessing quality
    • Synthesising results
    • Applicability and transferability
types of reviews

Types of reviews




Systematic reviews


narrative reviews
Narrative reviews
  • Usually written by experts in the field
  • Use informal and subjective methods to collect and interpret information
  • Usually narrative summaries of the evidence

Read: Klassen et al. Guides for Reading and Interpreting Systematic Reviews. Arch Pediatr Adolesc Med 1998;152:700-704.

what is a systematic review
What is a systematic review?

A review of the evidence on a clearly formulated question that uses systematic and explicit methods to identify, select and critically appraise relevant primary research, and to extract and analyse data from the studies that are included in the review*

*Undertaking Systematic Reviews of Research on Effectiveness. CRD’s Guidance for those Carrying Out or Commissioning Reviews. CRD Report Number 4 (2nd Edition). NHS Centre for Reviews and Dissemination, University of York. March 2001.

key elements of a systematic review
Key elements of a systematic review

Structured, systematic process involving several steps :

  • Formulate the question
  • Plan the review
  • Comprehensive search
  • Unbiased selection and abstraction process
  • Critical appraisal of data
  • Synthesis of data (may include meta-analysis)
  • Interpretation of results

All steps described explicitly in the review

systematic vs narrative reviews
Scientific approach to a review article

Criteria determined at outset

Comprehensive search for relevant articles

Explicit methods of appraisal and synthesis

Meta-analysis may be used to combine data

Depend on authors’ inclination (bias)

Author gets to pick any criteria

Search any databases

Methods not usually specified

Vote count or narrative summary

Can’t replicate review

Systematic vs. Narrative reviews
advantages of systematic reviews
Advantages of systematic reviews
  • Reduce bias
  • Replicability
  • Resolve controversy between conflicting studies
  • Identify gaps in current research
  • Provide reliable basis for decision making
limitations of systematic reviews specific to health promotion
Limitations of systematic reviews specific to health promotion
  • Results may still be inconclusive
  • There may be no trials/evidence
  • The trials may be of poor quality
  • The intervention may be too complex to be tested by a trial
  • Practice does not change just because you have the evidence of effect/effectiveness
consider these interventions
Consider these interventions…
  • Interventions to promote smoke alarm ownership and function
  • School-based driver education for the prevention of traffic crashes
  • Helmets for preventing head and facial injuries in bicyclists

Do you think the results identified in SRs will be good, promising or absent (and potentially harmful)?

results from systematic reviews
Results from systematic reviews
  • Helmets reduce bicycle-related head and facial injuries for bicyclists of all ages involved in all types of crashes including those involving motor vehicles.
  • The results provide no evidence that drive education reduces road crash involvement, and suggest that it may lead to a modest but potentially important increase in the proportion of teenagers involved in traffic crashes.
  • Results from this review suggest that area-wide traffic calming in towns and cities may be a promising intervention for reducing the number of road traffic injuries and deaths. However, further rigorous evaluations of this intervention are needed.
the cochrane collaboration
The Cochrane Collaboration

International non-profit organisation that prepares, maintains, and disseminates systematic up-to-date reviews of health care interventions

cochrane collaboration
Cochrane Collaboration

Named in honour of Archie Cochrane, a British researcher

In 1979:

“It is surely a great criticism of our profession that we have not organised a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomised controlled trials”


The Cochrane Library

  • Cochrane Systematic reviews : Cochrane reviews and protocols
  • Database of Reviews of Effects: Other systematic reviews appraised by the Centre for Reviews and Dissemination.
  • Cochrane Central Register of Controlled Trials:

Bibliography of controlled trials (some not indexed in MEDLINE).

  • Health Technology Assessment Database: HTA reports
  • NHS Economic evaluation database:

Economic evaluations of health care interventions.

the cochrane library
The Cochrane Library

cochrane hpph field
Cochrane HPPH Field
  • Represent the needs and interests of those in health promotion and public health in Cochrane matters
  • Represent Cochrane in health promotion and public health forums
  • In transition from Field to Review Group
      • Will edit PH reviews for the Cochrane Library
cochrane collaboration structure
Cochrane Collaboration Structure

Steering Group

Review Groups



Methods Groups

Consumer Network

collaborative review groups 50
Collaborative Review Groups (50)
  • Produce systematic reviews relevant to a particular disease or health issue for inclusion in the Cochrane Library
  • Examples
    • Airways Group
    • Drug and Alcohol Group
    • Heart Group
    • Injuries Group
    • Skin Group
    • Pregnancy and Childbirth Group
    • Stroke Group
    • Breast Cancer Group
methods groups 12
Methods Groups (12)
  • Provide advice and support in the development of the methods of systematic reviews
  • Examples
    • Non-Randomised Studies
    • Screening and Diagnostic Tests
    • Empirical Methodological Studies
    • Qualitative Methods
cochrane centres 14
Cochrane Centres (14)
  • Work to assist all Cochrane entities within a specific geographical area
  • Examples
    • Australasian Cochrane Centre (at Monash)
    • South African Cochrane Centre
    • Italian Cochrane Centre
    • Chinese Cochrane Centre
cochrane fields networks 9
Cochrane Fields/Networks (9)
  • Represent an area of interest which spans a number of health problems - and hence a number of Review Groups
  • Examples
    • Health Promotion and Public Health Field
    • Primary Health Care Field
    • Cancer Network
    • Child Health Field
cochrane hpph field23
Cochrane HPPH Field
  • Cochrane Fields represent a population group, or type of care that overlaps multiple Review Group areas
  • HPPH Field
    • Registered in 1996
    • Administered from Melbourne
    • Funded by VicHealth
    • Over 400 members on contact database across >30 countries
  • Elizabeth Waters (Director)
  • Jodie Doyle (Coordinator)
  • Rebecca Armstrong (Senior Research Fellow)
  • Naomi Priest (Research Fellow)
questions of interest
Questions of interest


  • Does the intervention work/not work?
  • Who does it work/not work for?

Other important questions:

  • How does the intervention work?
  • Is the intervention appropriate?
  • Is the intervention feasible?
  • Is the intervention and comparison relevant?
answerable questions
Answerable questions


A description of the populations P

An identified interventionI

An explicit comparison C

Relevant outcomes O

a pico question
A PICO question

Time-consuming question:

What is the best strategy to prevent smoking in young people?

an answerable question
An answerable question

Q. Are mass media (or school-based or community-based) interventions effective in preventing smoking in young people?

systematic review process
Systematic review process
  • Well formulated question
  • Comprehensive data search
  • Unbiased selection and abstraction process
  • Critical appraisal of data
  • Synthesis of data
  • Interpretation of results
a good search
A good search
  • Clear research question
  • Comprehensive search
    • All domains, no language restriction, unpublished and published literature, up-to-date
  • Document the search (replicability)
components of electronic searching
Components of electronic searching
  • Describe each PICO component
  • Start with primary concept
  • Find synonyms
    • Identify MeSH / descriptors / subject headings
    • Add textwords
  • Add other components of PICO question to narrow citations (may use study filter)
  • Examine abstracts
  • Use search strategy in other databases (may need adapting)
so you want to do a quick dirty
So you want to do a ‘quick & dirty’?
  • DARE
  • PubMed (clinical queries, related records)
  • CDC
  • NICE
  • Organisations who do work in your area
  • …google
the guide to community preventive services
The Guide to Community Preventive Services

national institute for health and clinical excellence
National Institute for Health and Clinical Excellence

evidence for policy and practice information and co ordinating centre eppi centre
Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre)

effective public health practice project ephpp
Effective Public Health Practice Project (EPHPP)

centre for reviews and dissemination
Centre for Reviews and Dissemination

  • The order of terms will effect the results so start with the obvious or key concept
  • No need for ‘and’
  • Google will ignore common words
    • If they are important use + (e.g. policy + 3)
  • Phrase searching is useful eg “suicide prevention”
  • Google searches for variations on words eg diet, dietary
  • Where terms have multiple meanings you can direct google to remove sites you want to avoid (e.g. bass –music)
  • Keep your search strings brief
    • Mental health promotion initiatives to prevent suicide in young people
  • Compartmentalise your search strings
    • “mental health promotion” suicide
    • “Suicide prevention” “young people”
    • Prevention and suicide and youth
  • Use the same principles for google – keep it short and sweet.
  • Key features
    • Journals DatabaseMeSH DatabaseSingle Citation MatcherClinical Queries

Select the Limits tab – just under the search string

These next few slides show you how to search MeSH terms in PubMed. Useful if you don’t have access to electronic databases. You combine this method with the one for text words outlined above. The process for combining text words and MeSH terms is outlines below.

Mass media interventions to prevent smoking in young people

P= Young people


Find MeSH and textwords to

describe young people


Mass media interventions to prevent smoking in young people

P= Young people

MeSH: Adolescent




Mass media interventions to prevent smoking in young people

P= Young people


Adolescent Girl

Child Boy

Juvenile Teenager

Young people Young adult

Student Youth


Truncation $:

To pick up various forms of a word

Teen$.tw Smok$.tw Teenage Smoke

Teenager Smoking

Teenagers Smokes

Teens Smoker

Teen Smokers


Wild cards ? and #:

To pick up different spellings

Colo? (? Can be substituted for one or no characters)


Color (# Substitutes for one character)




Adjacent ADJn:

  • retrieves two or more query terms within n words of each other, and in any order
  • Great when you are not sure of phraseology

Eg sport adj1 policy

Sport policy

Policy for sport

Eg mental adj2 health

Mental health

Mental and physical health

example continued
Example continued

Mass media interventions to prevent smoking in young people

I = Mass media interventions


Find MeSH and textwords to

describe mass media interventions

example continued64
Example continued
  • MeSH
    • Mass media
    • Audiovisual aids
    • Television
    • Motion pictures
    • Radio
    • Telecommunications
    • Newspapers
    • Videotape recording
    • Advertising
example continued65
Example continued

Mass media interventions to prevent smoking in young people

O = Prevention of smoking


Find MeSH and textwords to

describe prevention of smoking

example of search
Example of search


MeSH Textwords

………………………. ………………………. ………………………. ……………………….

………………………. ……………………….


MeSH Textwords

………………………. ………………………. ………………………. ……………………….

………………………. ……………………….

C = (if required)


MeSH Textwords

………………………. ………………………. ………………………. ……………………….

………………………. ……………………….











different bibliographic databases
Different bibliographic databases
  • Databases use different types of controlled vocabulary
    • Same citations indexed differently on different databases
    • Medline and EMBASE use a different indexing system for study type
    • PsycINFO and ERIC do not have specific terms to identify study types

Need to develop search strategy for each database

study design filters
Study design filters
  • RCTs
    • See Cochrane Reviewer’s Handbook
  • Non-RCTs
    • Not yet developed, research in progress
  • Qualitative research
    • Specific subject headings used in CINAHL, ‘qualitative research’ used in Medline
    • CINAHL Filter: Edward Miner Library

  • Systematic reviews/meta-analyses
    • CINAHL: as above
    • Medline
    • Medline and Embase

    • PubMed
2 unpublished literature
2. Unpublished literature
  • Not all known published trials are identifiable in Medline (depending on topic)
  • Only 25% of all medical journals in Medline
  • Non-English language articles are under-represented in Medline (and developing countries)
  • Publication bias – tendency for investigators to submit manuscripts and of editors to accept them, based on strength and direction of results (Olsen 2001)
2 unpublished literature70
2. Unpublished literature
  • Hand searching of key journals and conference proceedings
  • Scanning bibliographies/reference lists of primary studies and reviews
  • Contacting individuals/agencies/ academic institutions

Neglecting certain sources may result in reviews being biased

critical appraisal
Critical appraisal

The process of systematically examining research evidence to assess its validity, results and relevance before using it to inform a decision.

Alison Hill, Critical Appraisal Skills Programme, Institute of Health

Sciences, Oxford

why appraise validity
Why appraise validity?
  • Not all published and unpublished literature is of satisfactory methodological rigour
    • Just because it is in a journal does not mean it is sound!
    • Onus is on you to assess validity!
  • Quality may be used as an explanation for differences in study results
  • Guide the interpretation of findings and aid in determining the strength of inferences
bias quality assessment tool
Bias – quality assessment tool
  • Selection bias
  • Allocation bias
  • Confounding
  • Blinding (detection bias)
  • Data collection methods
  • Withdrawals and drop-outs
  • Statistical analysis
  • Intervention integrity

Selection bias

Recruit participants

Allocation of concealment





Exposed to intervention

Not exposed to intervention

Integrity of intervention





Blinding of outcome assessment



Data collection methods



Statistical analysis

critical appraisal tools
Critical appraisal tools
  • RCTs
    • The Quality Assessment Tool for Quantitative Studies (
  • Non-RCTs
    • Cochrane Effective Practice and Organisation of Care Group (
    • The Berkeley Systematic Reviews Group (
qualitative research
Qualitative research
  • … explores the subjective world. It attempts to understand why people behave the way they do and what meaning experiences have for people.
  • Qualitative studies of experience
  • Process evaluation

Undertaking Systematic Reviews of Research on Effectiveness. CRD’s Guidance for those Carrying Out or Commissioning Reviews. CRD Report Number 4 (2nd Edition). NHS Centre for Reviews and Dissemination, University of York. March 2001.

casp appraisal checklist
CASP appraisal checklist
  • Clear aims of research (goals, why it is important, relevance)
  • Appropriate methodology (what, how, why)
  • Sampling strategy
  • Data collection
  • Relationship between researcher and participants
  • Ethical issues
  • Data analysis
  • Findings
  • Value of research (context dependent)
other qualitative checklist
Other qualitative checklist
  • Quality framework
    • Government Chief Social Researcher’s Office, UK
  • 19 question checklist for process evaluations (EPPI-Centre)
appraisal of a systematic review
Appraisal of a systematic review
  • 10 questions
    • Clearly-focused question
    • The right type of study included
    • Identifying all relevant studies
    • Assessment of quality of studies
    • Reasonable to combine studies
    • What were the results
    • Preciseness of results
    • Application of results to local population
    • Consideration of all outcomes
    • Policy or practice change as a result of evidence


factors influencing effectiveness
Factors influencing effectiveness
  • Type III error (integrity of intervention)
  • Theoretical framework of intervention
  • Context
  • For whom did the intervention work, why, in what circumstances, at what cost
difficulties addressing inequalities
Difficulties addressing inequalities
  • Studies rarely present information on differential effects of interventions
  • Cannot locate studies addressing inequalities
  • May need original data from authors
  • Low power to detect subgroup differences
assessing the applicability and transferability of interventions
Assessing the applicability and transferability of interventions
  • Applicability – whether the intervention process could be implemented in the local setting, no matter what the outcome is.
    • Is it possible to run this intervention in this local setting?
    • Eg. provision of condoms in area where they are not acceptable for religious reasons

Wang et al 2005

assessing the applicability and transferability of interventions89
Assessing the applicability and transferability of interventions
  • Transferability – if the intervention were to be implemented in the local setting, would the effectiveness of the program be similar to the level detected in the study setting?
    • E.g. if the interventionists lack experience and have few skills in delivering the intervention then its effectiveness in the local setting may be lower than that demonstrated in the study setting

Wang et al 2005

contact details
Contact details
  • Rebecca Armstrong
    • 03 9667 1336
    • If I can’t help you I might be able to point you in the right direction.
    • If you are interested in training and support for conducting SRs or increasing uptake within your organisation, region, state please let me know