RCT Schema
This presentation is the property of its rightful owner.
Sponsored Links
1 / 43

Ida Sim, MD, PhD Associate Professor of Medicine PowerPoint PPT Presentation


  • 44 Views
  • Uploaded on
  • Presentation posted in: General

RCT Schema The Trial Bank Project. Ida Sim, MD, PhD Associate Professor of Medicine Director, Center for Clinical and Translational Informatics University of California, San Francisco, CA Supported by The Trial Bank Project R01-LM-06780

Download Presentation

Ida Sim, MD, PhD Associate Professor of Medicine

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Ida sim md phd associate professor of medicine

RCT Schema

The Trial Bank Project

Ida Sim, MD, PhD

Associate Professor of Medicine

Director, Center for Clinical and Translational Informatics

University of California, San Francisco, CA

Supported by

The Trial Bank Project R01-LM-06780

National Center for Biomedical OntologyU54 HG004028-01


Outline

Outline

  • Background

  • RCT Schema

    • modeling approach

    • the class structure

    • evaluation

  • Relationship to CTO

  • Summary


Goals of the cto

Goals of the CTO

  • (1) fully and faithfully capture the types of entities and relationships involved in clinical trials

  • (2) comprehend terms like: cohort, randomization, placebo, etc., including ... statistical terms and terms for ... meta-analysis;

  • (3) organize these terms in a structured way, providing definitions and logical relations designed to enhance retrieval of, reasoning with, and integration of the data annotated in its terms

  • ...

  • (6) draw on and seek maximal alignment with existing clinical trial ontologies, including:

    • RCT Schema ontology used by theTrial Bank Project


Major axes for aligning models

Major Axes for Aligning Models

  • Domain of clinical trials

    • design: (non-)randomized, crossover, cluster randomized, factorial...

    • objective: interventional, diagnostic, preventive...

    • clinical domain: drugs, procedures, organizational change...

  • Task

    • trial design, execution, reporting, analysis, application

    • for individual trials, sets of clinically related trials

  • Purpose (application vs. domain ontology)

    • to support accomplishment of domain task(s)

    • to define shared meaning for “integration of the data annotated in its terms”


Trial tasks

Trial Tasks


Trial bank definition

Trial Bank Definition

  • Computable repository of RCT information sufficiently detailed to support scientific analysis for

    • designing future clinical trials

    • evidence-based practice and policy making

  • Detailed information on

    • study design

    • study execution

    • summary and individual participant-level results

  • Trial Bank is NOT for running a trial


Trial bank target uses

Trial Bank Target Uses


Trial bank software

Bank-a-Trial

RCT Presenter

RCT Bank

RCT Schema

Trial Bank Software

  • RCT Bank built on RCT Schema

    • Ocelot frame-based ontology

  • Bank-a-Trial

    • web-based program for trialists to enter trial instances into RCT Bank

    • clinical descriptions of trial features (slot values) are in UMLS

  • RCT Presenter

    • web-based browser of individual trials


Major axes for rct schema

Major Axes for RCT Schema

  • Domain of clinical trials

    • design: (non-)randomized, crossover, cluster randomized, factorial...

    • objective: interventional, diagnostic, preventive...

    • clinical domain: drugs, procedures, organizational change...

  • Task

    • trial design, execution, reporting, analysis, application

    • for individual trials, sets of clinically related trials

  • Purpose

    • to support accomplishment of domain task(s) [application ontology]

    • to define shared meaning for “integration of the data annotated in its terms”


Outline1

Outline

  • Background

  • RCT Schema

    • modeling approach

    • the class structure

    • evaluation

  • Relationship to CTO

  • Summary


Entity specification problem

Entity Specification Problem

  • What RCT aspects to model in RCT Schema? What not to model?

    • multiple users (e.g., trialists, systematic reviewers)

    • multiple tasks (e.g., analysis, interpretation)

      • multiple methods

    • no one correct RCT ontology

  • Need principled, systematic approach

    • to specifying, documenting, and evaluating


Competency decomposition method

Competency Decomposition Method

(Sim, et al, JBI 2004: 37(2):108-119)

  • To define the entities that must be in a conceptual model

  • General approach

    • specify a task hierarchy of target tasks and subtasks

    • specify methods for each task

    • specify entities required for completing each task using each method

  • Generates a specification of required entities

    • the information requirements for the competencies (tasks and subtasks) that the model/knowledge base is to support


Target task for rct bank

Target Task for RCT Bank

  • Using RCTs for trial design or

  • clinical application requires

  • synthesizing evidence across

  • all trials on a topic


Systematic reviewing

Systematic Reviewing

  • Canonical method for synthesizing evidence across trials

  • Major steps are

    • retrieve related RCTs (e.g., 32 trials of metformin for diabetes)

    • analyze how comparable the trials are

    • statistically combine data if appropriate

      • combining smaller trials increases statistical power to detect effects


Target task systematic review

Target Task = Systematic Review

  • RCT Bank entities = sys. review information needs

    • identified all systematic review tasks

      • review of literature and personal experience conducting 3 systematic reviews

    • identified methods for completing these tasks

    • organized tasks and methods into a task hierarchy

    • derived RCT entities necessary and sufficient for each (sub)task


Top level sys review tasks

Top-Level Sys Review Tasks

  • Trial retrieval

  • Trial critiquing

    • judging internal validity

    • judging generalizability

  • Meta-analysis of quantitative results

    • analysis of clinical and statistical heterogeneity

  • Contextual interpretation

    • scientific, socio-economic, and ethical

http://rctbank.ucsf.edu/tasks/tasks.html


Judgment of generalizability

Judgment of Generalizability

  • Were the people enrolled in the trial representative and unbiased?

    • were eligible patients randomly selected from the source population?

    • were enrolled subjects a random subset of those eligible?

  • Are the trial subjects similar to mine?

  • Do I have the tested intervention available here?


Method in dependent entity specification

Method-(In)dependent Entity Specification

  • Were the people enrolled in the trial representative and unbiased?

    • were eligible patients randomly selected from the source population?

      • method: no computable algorithm available

        • recruitment method

    • were enrolled patients a random subset of those eligible?

      • method: using standard statistics

        • number and clinical characteristics of enrolled subjects

        • number and clinical characteristics of eligible but non-enrolled subjects


Results entity requirements

External

Validity

. . . . .

3

. . .

2

1

4

1

. . .

. . .

. . .

. . .

. . .

1

2

4

7

2

9

. . .

. . .

. . .

39

22

13

Results: Entity Requirements

Quantitative

Synthesis

Contextual

Interpretation

Critiquing

High-level

Tasks

Retrieval

SubTask I

Internal

Validity

Methods

SubTask II

11

(n=35)

SubTask III

(n=74)

112

29

30

171 Unique Entities Required


Entity specification evaluation

Entity Specification Evaluation

  • Evaluated match between

    • the 171 information items

    • 388 requirements in 18 published trial-critiquing instruments

  • Results

    • entity specification is comprehensive

    • entity specification is reasonable

Sim, et al, KR-MED 2004; JBI 2004: 37(2):108-119


Benefits of approach

Benefits of Approach

  • Task hierarchy understandable by domain experts

    • identifies which information items are required for which tasks

  • Provides an evaluation “yardstick”

    • if an ontology contains all the information requirements for a task

      • then is it “competent” for that task

    • can evaluate and compare application ontologies

  • Documents an (application/domain) ontology

    • states which tasks an ontology is competent for, and why

    • cross-indexes tasks and entities in the ontology


Outline2

Outline

  • Background

  • RCT Schema

    • modeling approach

    • the class structure

    • evaluation

  • Relationship to CTO

  • Summary


Implementation of 171 items

Bank-a-Trial

RCT Presenter

RCT Bank

RCT Schema

Implementation of 171 Items

  • Purpose of RCT Bank KB is to support scientific analysis of trial evidence

    • needed a “data-schema” or “instance-style” ontology

  • RCT Schema implemented the 171 information items in a frame-based ontology


Rct schema ontology

RCT Schema Ontology

  • Ocelot frame-based model

    • 7 levels deep

  • 192 frames, 607 unique slots

    • avg. 9.8 slots/frame

    • 3 frames (1.6%) have multiple parents

    • 193/607 slots (32%) take other frames as values

  • Available at http://rctbank.ucsf.edu/


Ida sim md phd associate professor of medicine

  • RCT Schema displayed in GKB Editor

    • classes are red boxes

    • instances are blue boxes

  • Class hierarchy organizes entities as

    • trial concepts

    • trial descriptions (details)

  • Not fully compliant with “Werner’s Rules”


Includes is a hierarchies

Includes IS-A Hierarchies


Instantiating rct schema

Instantiating RCT Schema

  • Clinical content described by terms from a clinical vocabulary, e.g.,

    • for a breast cancer trial, instance of BASELINE-CHARACTERISTIC is described by

      • term “menopause” from UMLS preferred term

      • the UMLS CUI

    • Trial Bank software supports any vocabulary in UMLS (e.g., SNOMED)

  • Each trial is a collection of instances of classes


Ida sim md phd associate professor of medicine

  • 518-TRIAL

    • 518-BACKGROUND-DETAILS

    • 518-ADMIN-DETAILS

    • 518-EXECUTED-PROTOCOL

      • 518-ALL-SUBJECTS

      • 518-PRIMARY-OUTCOME-1

      • etc.

    • 518-ERRATUM

    • 518-CONCLUSION-DETAILS


Ida sim md phd associate professor of medicine

  • 518-PRIMARY-OUTCOME-1(e.g., all-cause mortality)

    • 518-STAT-ANALYSIS-AND-RESULTS-1(e.g. t-test)

      • 518-ALL-COMPARISONS-AT-TIME-X-1(e.g., at 6 months)

        • 518-SINGLE-TIME-X-COMPARISON-1 (e.g., between PCI and thrombolysis groups)


Ida sim md phd associate professor of medicine

  • 518-SINGLE-TIME-X-COMPARISON-1

    • datapoint for PCI group

      • numerator (all-cause deaths at 6 months)

      • denominator (had all-cause death outcome assessed at 6 months)

      • 518-STUDY-ARM-POPULATION-1 (the PCI group)

    • datapoint for thrombolysis group

    • summary odds ratio under intention-to-treat analysis

    • summary odds ratio under efficacy analysis


Outline3

Outline

  • Background

  • RCT Schema

    • modeling approach

    • the class structure

    • evaluation

  • Relationship to CTO

  • Summary


Expressivity evaluation

Expressivity Evaluation

  • Captured 17 full and 20+ partial trials


Modeling challenges met

Modeling Challenges Met

  • Multi-armed, crossover, cluster randomized studies

  • Many variations of patient drop-out, loss to followup (e.g., excluded after randomization)

  • For each outcome, the # of subjects assessed at each timepoint in each subgroup

  • Blinding efficacy: did subjects know which arm they were assigned to?

  • etc.


Modular extensible

Modular, Extensible

  • Extensions possible with only minimal changes to 192 existing classes

  • Extensible to new clinical domains (e.g., genomics) via clinical vocabularies

    • no clinical terms in RCT Schema


Limitations of representation

Limitations of Representation

  • Modeled, not yet tested

    • participant-level data

    • factorial designs

    • designs with run-in and washout periods

  • In development

    • computable eligibility rules

  • Not yet modeled

    • genomic data

    • nested subgroups

    • secondary studies (e.g,. followup studies)


Trial bank publishing

Trial Bank Publishing

  • How to get trials into RCT Bank?

  • Collaborated with JAMA and Ann Int Med to explore co-publishing trials as articles and RCT Bank entries

    • authors submit manuscripts for peer review as usual

    • trial-bank staff enter accepted trials into trial bank

    • co-published 14 trials in RCT Presenter

  • Evaluation

    • 83 respondents evaluated a trial using both RCT Presenter and the Journal Article

      • mostly trialists and meta-analysts


Rct presenter evaluation

RCT Presenter Evaluation

  • 70% of respondents rated RCT Presenter as good as or better than the Journal Article for all attributes

N=30


Outline4

Outline

  • Background

  • RCT Schema

    • modeling approach

    • the class structure

    • evaluation

  • Relationship to CTO

  • Summary


Alignment to cto

Alignment to CTO

  • Have participated in CTO working group from inception (Simona Carini)

  • Contributed RCT Schema classes and definitions to list of terms for consideration

  • Contributed to draft of high-level concept hierarchy

    • http://www.bioontology.org/wiki/index.php/High-level_Concepts_v0.2

  • Contributed to creation of the draft CTO presented this morning


  • How would trial bank use cto

    How Would Trial Bank Use CTO?

    • Use CTO as common index into RCT Bank and into other clinical trial data and information systems

    • Map RCT Schema class and slot names to terms in CTO

      • so that RCT Bank instances can be made available to machines and humans who wish to

        • query

        • reason, or

        • integrate

      • clinical trial information using CTO

      • e.g., trials co-published with PLoS, etc. into RCT Bank


    Example use ncbo

    Example Use (NCBO)

    • Map class names from any trial bank to terms in CTO

      • e.g., in RCT Bank or “European Trial Bank”

        • PRIMARY-OUTCOME to CTO term for this

        • BASELINE-CHARACTERISTIC to CTO term for this

    • CTeXplorer an ontology-driven tool for visualizing complex design differences across trials [MA Storey, et al UVic, Canada]

      • given variable HgbA1C from any trial bank

      • would know how to handle and display if annotated as

        • PRIMARY-OUTCOME [CTO] or

        • BASELINE-CHARACTERISTIC [CTO]


    Cto for integration with trial bank collaborators

    CTO for Integration with Trial Bank Collaborators...

    • National Center for Biomedical Ontology

    • Electronic Primary Care Research Network

      • Primary Care Research Object Model

    • Global Trial Bank, with AMIA

      • trial-bank publishing

      • “dedicated to assuring the implementation and maintenance of an open global infrastructure for computable clinical trial results information”

    • European Clinical Trial Data Repository

      • submitted to Framework Programme 7

    • with BRIDG, clinical trial management systems, etc?


    Summary

    Summary

    • Trial banks are computable repositories of trial information for analysis, interpretation, and application of trial evidence to research and care

    • We specified the entities for RCT Bank using competency decomposition method

    • RCT Schema implemented entities specification as an “instance-style” frame-based ontology

      • expressive, extensible, useful for reporting and interpretation

    • Mapping of RCT Schema terms to CTO terms makes RCT Bank instances available for CTO-driven integration and reasoning


  • Login