keeping up with the evidence l.
Skip this Video
Loading SlideShow in 5 Seconds..
Keeping up with the evidence: PowerPoint Presentation
Download Presentation
Keeping up with the evidence:

Loading in 2 Seconds...

play fullscreen
1 / 25

Keeping up with the evidence: - PowerPoint PPT Presentation

  • Uploaded on

Keeping up with the evidence:. Systematic searching vs literature surveillance (or Why Librarians Do it Better…). Sarah Greenley Information Specialist BMJ Knowledge, London, UK (With thanks to Mick Arber and Alex McNeil). Overview of presentation. Background to project

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Keeping up with the evidence:' - Mia_John

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
keeping up with the evidence

Keeping up with the evidence:

Systematic searching vs literature surveillance

(or Why Librarians Do it Better…)

Sarah Greenley

Information Specialist

BMJ Knowledge, London, UK

(With thanks to Mick Arber and Alex McNeil)

overview of presentation
Overview of presentation
  • Background to project
  • Methods of evidence-based resources
  • Literature review of existing research
  • Clinical Evidence (CE) search and appraisal
  • BMJ Updates
  • Phase I: 10 CE topics
  • Phase II: 217 topics
  • Implications of missing studies
pre appraised resources
Pre-appraised resources
  • Traditional “full blown” EBM impractical for many health professionals
  • Move towards “pre-appraised” evidence
  • Searching, critical appraisal, summary of evidence performed by others
  • Examination of methods necessary
methods of evidence based resources
Methods of Evidence-based resources

Not all “evidence-based” resources are the same

  • Systematic searches and explicit selection criteria for initial search and updates e.g. Clinical Evidence, Cochrane systematic reviews
  • Systematic literature surveillance for updating– defined set of journals/resources e.g. UptoDate, Dynamed, Infopoems, EB journals
  • Collection of other resources e.g. TRIP, Sumsearch
  • Citing references
literature review
Literature review
  • Examination of methods of EB resources
  • Search on Medline, Embase, The Cochrane Library, additional web-based resources
  • “Secondary journals/literature” “literature surveillance/scanning”
  • Studies comparing yield of search, time to appraise, speed of incorporation of evidence, missing data
challenge of updating systematic searching
Challenge of updating: systematic searching
  • Databases index thousands of journals, Medline c4500, Embase c6500
  • Includes foreign language material
  • Studies unlikely to be missed
  • Time consuming to incorporate new evidence, costly
  • Indexing delays?
  • Authoring, editing, peer review
challenge of updating surveillance
Challenge of updating: surveillance
  • Scanning defined list of resources 85-500
  • Newly published material incorporated quickly
  • “Added value” of commentaries, ratings
  • Possibility of bias: reporting positive trials
  • Language restrictions, missing information
  • Missed studies?
ce information specialists
CE information specialists

Scoping searches

Topic planning

Foundation literature searches

Systematic annual update searches

Critical appraisal of all search results

Checking of author inclusion/exclusion forms

  • Ear wax: 18 update refs
  • Secondary prevention of MI: 5285
  • Additional searches: adverse events, observational studies, new comparisons, new options
  • Ordering papers, editing, production, peer review
  • Subsequently published studies – is there a better way to identify them?
bmj updates
BMJ Updates
  • McMaster University
  • Alerting services
  • Over 110 premier clinical journals
  • Articles rated for quality and relevance
  • Searchable database from 2002
  • Also basis for EB Journals
our study using bmjupdates vs searching
Phase 1

10 CE topics

Compare time to search

Compare time to appraise

Compare yield of results

Phase 2

All CE topics

Highlight missing studies

Examine implications

Our study: using BMJupdates vs searching
phase 1
Sickle cell










Volume of last CE annual update

(retrospective checking of search assessment form)

Volume of BMJUpdates published in same period

(Search on BMJupdates database limited to therapy/prevention)

Phase 1
key issues
Key issues
  • Surveillance ongoing not retrospective
  • BMJUpdates found 1 extra study in chickenpox – delay on database searching
  • CE update search: 237selected results compared with 18 for BMJUpdates
  • Some topics yielded no BMJUpdates results
  • Only around 10% of CE results selected for authors
  • CE search and appraisal estimate for TB=18 hours
  • BMJ Updates search and appraisal estimate for TB=3 hours
phase ii objectives
Phase II objectives

Check every reference cited in CE (excluding background references) against list of 110 BMJUpdates journals

How many studies would have been missed in total?

How many studies would have been missed for updates?

Did missing studies change CE categorisation?

phase ii methods
Phase II methods
  • Print out each CE chapter
  • Exclude background references
  • Check cited reference against BMJUpdates list of journals
  • Identify new references (substantive changes)
  • Identify changes in categorisation
phase ii
Total number of topics checked:217

Total number of references checked:6794

123 topics with new evidence

Results tabulated on Excel spreadsheet

Phase II
phase ii results
Phase II results
  • 60% of all references were missing
  • 54% of “new evidence” references missing
  • 27 topics where all “new evidence” references were on BMJUpdates journal list
  • 32 topics where 100% of “new evidence” references were not on BMJUpdates journal list
best bmjupdates coverage
Best BMJUpdates coverage

Section %of refs missing

Diabetes 28%

CVD 31%

Sleep 33%

Pregnancy 40%

Musculoskeletal 47%

Respiratory 47%

worst bmjupdates coverage
Worst BMJUpdates coverage

Section % refs missing

ENT 71

Poisoning 71

Cancer 72

Skin 78

Kidney 81

Men’s health 83

Oral 84

Eye 86

categorisation changes
Across 16 updated topics 30 instances where “missing” references led to change in categorisation

“Taxane based combination chemotherapy One systematic review and three RCTs added; categorisation changed from Likely to be beneficial to Trade off between benefits and harms.”

Categorisation changes
english language bias
English language bias
  • Exclusion of foreign language material can be important.
  • Translated Turkish trial changed categorisation in Chronic Suppurative Otitis Media
  • German language ENT topics
  • Literature surveillance is a useful method to stay aware of new evidence published in major journals in a digest format
  • Limited scope of their surveillance means not suitable for full updating of evidence-based resources but could be useful “extra”
  • Evidence-based resources should use explicit and reproducible methods
contact information
Contact information

Sarah Greenley