Towards evidence based discovery informatics tools for synthesis guest speaker tim cary
1 / 22

Towards Evidence-Based Discovery Informatics Tools for Synthesis Guest Speaker : Tim Cary - PowerPoint PPT Presentation

  • Uploaded on

Towards Evidence-Based Discovery Informatics Tools for Synthesis Guest Speaker : Tim Cary. Catherine Blake School of Information and Library Science University of North Carolina at Chapel Hill [email protected] Systematic Review Process.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Towards Evidence-Based Discovery Informatics Tools for Synthesis Guest Speaker : Tim Cary' - colt-logan

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Towards evidence based discovery informatics tools for synthesis guest speaker tim cary

Towards Evidence-Based DiscoveryInformatics Tools for Synthesis Guest Speaker : Tim Cary

Catherine Blake

School of Information and Library Science

University of North Carolina at Chapel Hill

[email protected]

Systematic review process
Systematic Review Process

  • Formulate the problem

  • Locate and select studies

  • Assess quality of studies

  • Collect data

  • Analyze and present results

  • Interpret results

  • Improve and update review

28 months from

initial idea to


Increased demand due to evidence-based medicine

Manual synthesis

Guesswork guided

by scientifically

trained intuition

Rescher (1978)

Manual Synthesis





Cochrane revman
Cochrane - RevMan

  • Review Manager (RevMan) is the software used for preparing and maintaining Cochrane reviews.

  • You can use RevMan for protocols and full reviews. It is most useful when you have formulated the question for the review, and allows you to prepare the text, build the tables showing the characteristics of studies and the comparisons in the review, and add study data. It can perform meta-analyses and present the results graphically.

  • Source:

Cochrane gradepro
Cochrane - GRADEpro

  • GRADEpro (GRADEprofiler) is the software used to create Summary of Findings (SoF) tables in Cochrane systematic reviews. It can retrieve data of the systematic review and meta-analyses from a Review Manager 5 file, combine these data with user-entered data, and then export a Summary of Findings table ready for import into Review Manager 5. It performs many of the calculations necessary to present the key results of systematic reviews in a table format and guides users through the process of grading the quality of the evidence using the GRADE approach.

  • Source:

Reporting guidelines
Reporting Guidelines

  • CONSORT - reporting of RCTs

  • PRISMA (formerly QUOROM) [PDF document] - preferred reporting items for systematic reviews and meta-analyses

  • STROBE - reporting of observational studies in epidemiology

  • EQUATOR Network - collection of reporting guidelines

  • Source:

Selection step
Selection Step

  • Typical information retrieval framing

    • Input: MEDLINE

    • Output: Articles included in previous studies

    • Goal: identify weighting schemes that identify only articles included in a traditional analysis

  • Examples

    • Cohen AM, Hersh WR, Peterson K, Yen PY. Reducing Workload in Systematic Review Preparation Using Automated Citation Classification. JAMIA 2006;13(2):206-219.

    • Demner-Fushman D, Seckman C, Fisher C, Hauser S, Clayton J, Thoma G. Prototype System To Support Evidence-based Practice. AMIA AnnuSymp Proc. November 2008:151-5.

Context information
Context Information

  • Study Information

    • e.g. date, location, ...

  • Population Information

    • e.g. gender, age, ...

  • Risk Factor or Intervention

    • e.g. duration of exposure, confounders

  • Disease

    • e.g. stage, confounders

Loosely coupled

to review focus

Tightly coupled

to review focus

Key estimate missing information
Key: Estimate Missing Information



What are people with Breast Cancer exposed to?

What are people in a similar population exposed to?

  • Facts for each study

  • number of patients

  • age of patients

  • geographic location

  • risk-factor exposure …

  • Codebook

  • question asked

  • age, gender

  • % responses

Database of risk factors


Studies with Breast Cancer patients


Are these rates significantly different?

T. Tengs & N. D. Osgood (2001) “The link between smoking and Impotence: Two Decades of Evidence”, Preventive Medicine, 32:447-52

More than automated meta analysis

Information Synthesis

Information Synthesis

More than Automated Meta-Analysis

  • Traditional analysis

    • same study design

    • medicine = RCT

    • epidemiology = cohort

  • Information Synthesis

    • any study that includes required information

    • augment missing information

Systematic Review


Main topic

Entire study

Secondary Information

External database

Natural Language


Human-assisted Discovery and Synthesis

Natural Language





Discovery Science

Evidence-based Practice


Human Discovery and Synthesis



Breast Cancer

Synthesis and

Discovery Work Practices

Heterogeneous Literature

Metis information extractor
METIS Information Extractor

  • Semantic Grammar

  • Features: words, numbers, and semantic types in the Unified Medical Language System (UMLS)

  • Information extracted :

  • risk factor exposure (tobacco and alcohol )  gender

  • age (min, max, mean)  start and end dates

  • number of subjects with medical condition  geographical location

{term;’age’} {term:’of’} {number;10<n2<110}{term;’to’}{number;10<n2<110}

The age of breast cancer subjects ranged between 20 to 64 years old.

{semantic type: neoplastic process, or disease}

Metis info extractor evaluation
METIS Info Extractor – Evaluation

  • Diverse text corpus

    • epidemiology, surgery, biology, ...

    • cohort studies, case-control trials, ...

  • Evaluation

    • Metrics (precision, recall)

    • Annotators (developer, domain expert, expert annotator, novice)

    • Primary topic (breast cancer, impotence)

    • Secondary information (tobacco and alcohol consumption)

Metis verifier
METIS Verifier

Converted Article

Electronic version of article

Verify information extracted

Metis analyzer
METIS Analyzer

  • Meta-Analysis

    • Developed for agricultural application

    • Requires empirical studies with a quantitative outcome

    • Unit of study is an article - not a person

    • Result – a unitless metric called an effect size

  • Two common meta-analysis techniques

    • Fixed effects

    • Randomized-effects model

Evaluation: Compared generated effect size with examples in text books and published articles


Result: Same effect size

Synthetic estimate evaluation



Synthetic Estimate Evaluation