Towards evidence based discovery informatics tools for synthesis guest speaker tim cary
Sponsored Links
This presentation is the property of its rightful owner.
1 / 22

Towards Evidence-Based Discovery Informatics Tools for Synthesis Guest Speaker : Tim Cary PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

Towards Evidence-Based Discovery Informatics Tools for Synthesis Guest Speaker : Tim Cary. Catherine Blake School of Information and Library Science University of North Carolina at Chapel Hill Systematic Review Process.

Download Presentation

Towards Evidence-Based Discovery Informatics Tools for Synthesis Guest Speaker : Tim Cary

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Towards Evidence-Based DiscoveryInformatics Tools for Synthesis Guest Speaker : Tim Cary

Catherine Blake

School of Information and Library Science

University of North Carolina at Chapel Hill

Systematic Review Process

  • Formulate the problem

  • Locate and select studies

  • Assess quality of studies

  • Collect data

  • Analyze and present results

  • Interpret results

  • Improve and update review

28 months from

initial idea to


Increased demand due to evidence-based medicine

Guesswork guided

by scientifically

trained intuition

Rescher (1978)

Manual Synthesis





Cochrane - RevMan

  • Review Manager (RevMan) is the software used for preparing and maintaining Cochrane reviews.

  • You can use RevMan for protocols and full reviews. It is most useful when you have formulated the question for the review, and allows you to prepare the text, build the tables showing the characteristics of studies and the comparisons in the review, and add study data. It can perform meta-analyses and present the results graphically.

  • Source:

Cochrane - GRADEpro

  • GRADEpro (GRADEprofiler) is the software used to create Summary of Findings (SoF) tables in Cochrane systematic reviews. It can retrieve data of the systematic review and meta-analyses from a Review Manager 5 file, combine these data with user-entered data, and then export a Summary of Findings table ready for import into Review Manager 5. It performs many of the calculations necessary to present the key results of systematic reviews in a table format and guides users through the process of grading the quality of the evidence using the GRADE approach.

  • Source:

Reporting Guidelines

  • CONSORT - reporting of RCTs

  • PRISMA (formerly QUOROM) [PDF document] - preferred reporting items for systematic reviews and meta-analyses

  • STROBE - reporting of observational studies in epidemiology

  • EQUATOR Network - collection of reporting guidelines

  • Source:

Selection Step

  • Typical information retrieval framing

    • Input: MEDLINE

    • Output: Articles included in previous studies

    • Goal: identify weighting schemes that identify only articles included in a traditional analysis

  • Examples

    • Cohen AM, Hersh WR, Peterson K, Yen PY. Reducing Workload in Systematic Review Preparation Using Automated Citation Classification. JAMIA 2006;13(2):206-219.

    • Demner-Fushman D, Seckman C, Fisher C, Hauser S, Clayton J, Thoma G. Prototype System To Support Evidence-based Practice. AMIA AnnuSymp Proc. November 2008:151-5.

Context Information

  • Study Information

    • e.g. date, location, ...

  • Population Information

    • e.g. gender, age, ...

  • Risk Factor or Intervention

    • e.g. duration of exposure, confounders

  • Disease

    • e.g. stage, confounders

Loosely coupled

to review focus

Tightly coupled

to review focus

Collaborative Information Synthesis

Key: Estimate Missing Information



What are people with Breast Cancer exposed to?

What are people in a similar population exposed to?

  • Facts for each study

  • number of patients

  • age of patients

  • geographic location

  • risk-factor exposure …

  • Codebook

  • question asked

  • age, gender

  • % responses

Database of risk factors


Studies with Breast Cancer patients


Are these rates significantly different?

T. Tengs & N. D. Osgood (2001) “The link between smoking and Impotence: Two Decades of Evidence”, Preventive Medicine, 32:447-52

Information Synthesis

Information Synthesis

More than Automated Meta-Analysis

  • Traditional analysis

    • same study design

    • medicine = RCT

    • epidemiology = cohort

  • Information Synthesis

    • any study that includes required information

    • augment missing information

Systematic Review


Main topic

Entire study

Secondary Information

External database

Natural Language


Human-assisted Discovery and Synthesis

Natural Language





Discovery Science

Evidence-based Practice


Human Discovery and Synthesis



Breast Cancer

Synthesis and

Discovery Work Practices

Heterogeneous Literature

METIS Information Extractor

  • Semantic Grammar

  • Features: words, numbers, and semantic types in the Unified Medical Language System (UMLS)

  • Information extracted :

  • risk factor exposure (tobacco and alcohol )  gender

  • age (min, max, mean)  start and end dates

  • number of subjects with medical condition geographical location

{term;’age’} {term:’of’} {number;10<n2<110}{term;’to’}{number;10<n2<110}

The age of breast cancer subjects ranged between 20 to 64 years old.

{semantic type: neoplastic process, or disease}

METIS Info Extractor – Evaluation

  • Diverse text corpus

    • epidemiology, surgery, biology, ...

    • cohort studies, case-control trials, ...

  • Evaluation

    • Metrics (precision, recall)

    • Annotators (developer, domain expert, expert annotator, novice)

    • Primary topic (breast cancer, impotence)

    • Secondary information (tobacco and alcohol consumption)

METIS Info Extractor – Recall

METIS Info Extractor – Precision

METIS Verifier

Converted Article

Electronic version of article

Verify information extracted

METIS Verifier

METIS Analyzer

  • Meta-Analysis

    • Developed for agricultural application

    • Requires empirical studies with a quantitative outcome

    • Unit of study is an article - not a person

    • Result – a unitless metric called an effect size

  • Two common meta-analysis techniques

    • Fixed effects

    • Randomized-effects model

Evaluation: Compared generated effect size with examples in text books and published articles


Result: Same effect size



Synthetic Estimate Evaluation



  • Login