The dimacs working group on disease and adverse event surveillance
This presentation is the property of its rightful owner.
Sponsored Links
1 / 34

The DIMACS Working Group on Disease and Adverse Event Surveillance PowerPoint PPT Presentation


  • 105 Views
  • Uploaded on
  • Presentation posted in: General

The DIMACS Working Group on Disease and Adverse Event Surveillance. Henry Rolka and David Madigan. Background. WG Objective: Bring together researchers in adverse event monitoring and disease surveillance Part of a 5-year special focus on computational and mathematical epidemiology

Download Presentation

The DIMACS Working Group on Disease and Adverse Event Surveillance

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


The dimacs working group on disease and adverse event surveillance

The DIMACS Working Group on Disease and Adverse Event Surveillance

Henry Rolka and David Madigan


Background

Background

  • WG Objective: Bring together researchers in adverse event monitoring and disease surveillance

  • Part of a 5-year special focus on computational and mathematical epidemiology

  • 50+ WG members: epidemiologists, public health professionals, biostatisticians, etc.

  • Focus on analytic/statistical methods

  • Two WG meetings plus week-long tutorial (02-03)

  • Coordinated closely with National Syndromic Surveillance Conferences


The dimacs working group on disease and adverse event surveillance

Areas of Common Interest


Representation

Carnegie-Mellon University

FDA

Quintiles Inc.

CDC

Rutgers University

Emergint, Inc.

AT&T Labs

NJ State

NYC Dept. of Health

University of Pennsylvania

Aventis

ATSDR

University of Connecticut

Los Alamos National Lab

Lincoln Technologies

SAS Institute

Representation


Background cont

Background, cont.

  • WG conceived before September 11, 2001

  • Surveillance landscape has changed drastically

  • Major public health effort directed at bioterrorism detection

  • Proliferation of novel surveillance projects in response to national threat

  • “Good for detecting outbreaks of various kinds”


New data types for public health surveillance

New Data Types for Public Health Surveillance

  • Managed care patient encounter data

  • Pre-diagnostic/chief complaint (text data)

  • Over-the-counter sales transactions

    • Drug store

    • Grocery store

  • 911-emergency calls

  • Ambulance dispatch data

  • Absenteeism data

  • ED discharge summaries

  • Prescription/pharmaceuticals

  • Adverse event reports


New analytic methods and approaches

New Analytic Methods and Approaches

  • Spatial-temporal scan statistics

  • Statistical process control (SPC)

  • Bayesian applications

  • Market-basket association analysis

  • Text mining

  • Rule-based surveillance

  • Change-point techniques


The dimacs working group on disease and adverse event surveillance

ANALYTIC METHODS IN USE

  • Scan statistics (e.g., Kulldorff’s SaTScan)

  • Statistical process control (e.g., Hutwagner’s EARS)

  • Association rule mining (e.g., Moore’s WSARE)

  • Bayesian shrinkage (e.g., DuMouchel’s MGPS)

  • Generalized linear mixed models (e.g., Kleinman)

  • Sequential probability ratio tests (e.g., Spiegelhalter, Evans)


The dimacs working group on disease and adverse event surveillance

SCAN STATISTICS

  • Martin Kulldorff’s SaTScan - Spatial and Space-Time Scan Statistics - software.

  • e.g., spatial scan – using Poisson model computes likelihood of all possible circles compared with likelihood under the null distribution

  • Picks the circle with the biggest likelihood ratio

  • P-value computed via Monte Carlo

  • Big literature on disease clustering: Besag & Newell, Diggle, Moran test, Turnbull’s method, Cuzick & Edwards, etc.

  • Need methodology for multiple sources


The dimacs working group on disease and adverse event surveillance

  • Farzad Mostashari


The dimacs working group on disease and adverse event surveillance

BAYESIAN SHRINKAGE ESTIMATION

  • DuMouchel’s GPS/MGPS

  • Compares observed counts of “market baskets” to expected counts under some (simple) model. For example, saw 30 cases in the ER today with G.I. syndrome AND fever AND work in Newark compared with an expectation of 3 cases

  • 30-to-3 is more convincing than 3-to-0.3 but less convincing that 300-to-30. Idea: shrink the smaller ones towards one.


The dimacs working group on disease and adverse event surveillance

GPS SHRINKAGE – AERS DATA

number of reports


The dimacs working group on disease and adverse event surveillance

BAYESIAN SHRINKAGE ESTIMATION

  • Issues:

  • Appropriate amount of shrinkage?

  • Where do the expected values come from?

  • Temporal dimension?

  • Covariate information

  • Simpson’s paradox (“innocent bystander”)


The dimacs working group on disease and adverse event surveillance

SEQUENTIAL PROBABILITY RATIO TESTS

  • Classical much-studied statistical method dating back to Wald (1948)


The dimacs working group on disease and adverse event surveillance

NATURAL LANGUAGE

  • Important sources of health data begin life as free text “chief complaints” (ED visits, primary care encounters, adverse event reports, e-mail, etc.)

  • “Approximately 5 minutes after receiving flu and pneumonia vaccine pt began hollering, "Oh, Oh my neck is hurting. Feels like a knot in my throat, a medicine taste." Complained of chest pain moving to back and leg numbness.”

  • Some (successful) work on automated coding of free text.

  • Little work on direct surveillance of text data


The dimacs working group on disease and adverse event surveillance

CONCLUSION

  • Analytic methods for surveillance have a long history in Statistics but currently attract substantial new interest from researchers in both CS and Statistics

  • Urgently need new methods for multivariate, multi-data type streams

  • Data availability a bottleneck; simulation non-trivial.

  • DARPA currently staging a competition


The dimacs working group on disease and adverse event surveillance

THE IDEAOF A COMPETITION

Thesis: Rapid growth in the number of deployed health surveillance systems and increasing complexity require new analytic methodologies

Goal: Stimulate mainstream Computer Science and Statistics researchers to focus on this area

How: A signal detection competition

Examples: the Message Understanding Conferences (MUC), Text Retrieval Conferences (TREC), KDD Cup, M3 Time Series competition


The dimacs working group on disease and adverse event surveillance

COMPETITION STATUS

  • DIMACS Working Group on Adverse Event and Disease Reporting, Surveillance, Analysis

  • Subgroup focused on competition; applied for funding; identified data sources

  • Key challenge: appropriate methods for inserting signals into real data (“spiking”)

  • Other groups face the same challenge (e.g. BioStorm)


The dimacs working group on disease and adverse event surveillance

ANALYTIC METHODS IN USE

  • Scan statistics (e.g., Kulldorff’s SaTScan)

  • Statistical process control (e.g., Hutwagner’s EARS)

  • Association rule mining (e.g., Moore’s WSARE)

  • Bayesian shrinkage (e.g., DuMouchel’s MGPS)

  • Generalized linear mixed models (e.g., Kleinman)

  • Sequential probability ratio tests (e.g., Spiegelhalter, Evans)


The dimacs working group on disease and adverse event surveillance

SCAN STATISTICS

  • Martin Kulldorff’s SaTScan - Spatial and Space-Time Scan Statistics - software.

  • e.g., spatial scan – using Poisson model computes a likelihood ratio for all possible circles comparing event counts inside and outside

  • Picks the circle with the biggest likelihood ratio

  • P-value computed via Monte Carlo

  • Big literature on disease clustering: Besag & Newell, Cuzick & Edwards, Diggle, Moran test, Pagano, Turnbull’s method,, etc.

  • Need methodology for multiple sources


The dimacs working group on disease and adverse event surveillance

  • Farzad Mostashari


The dimacs working group on disease and adverse event surveillance

BAYESIAN SHRINKAGE ESTIMATION

  • DuMouchel’s GPS/MGPS

  • Compares observed counts of “market baskets” to expected counts under some (simple) model. For example, saw 30 cases in the ER today with G.I. syndrome AND fever AND work in Newark compared with an expectation of 3 cases

  • 30-to-3 is more convincing than 3-to-0.3 but less convincing that 300-to-30. Idea: shrink the smaller ones towards one.


The dimacs working group on disease and adverse event surveillance

GPS SHRINKAGE – AERS DATA

number of reports


The dimacs working group on disease and adverse event surveillance

BAYESIAN SHRINKAGE ESTIMATION

  • Issues:

  • Appropriate amount of shrinkage?

  • Where do the expected values come from?

  • Temporal dimension?

  • Covariate information


The dimacs working group on disease and adverse event surveillance

SEQUENTIAL PROBABILITY RATIO TESTS

  • Classical much-studied statistical method dating back to Wald (1948). Mostly univariate.


The dimacs working group on disease and adverse event surveillance

NATURAL LANGUAGE

  • Important sources of health data begin life as free text “chief complaints” (ED visits, primary care encounters, adverse event reports, e-mail, etc.)

  • “Approximately 5 minutes after receiving flu and pneumonia vaccine pt began hollering, "Oh, Oh my neck is hurting. Feels like a knot in my throat, a medicine taste." Complained of chest pain moving to back and leg numbness.”

  • Some (successful) work on automated coding of free text.

  • Little work on direct surveillance of text data


The dimacs working group on disease and adverse event surveillance

THE IDEAOF A COMPETITION

Thesis: Rapid growth in the number of deployed health surveillance systems and increasing complexity require new analytic methodologies

Goal: Stimulate mainstream Computer Science and Statistics researchers to focus on this area

How: A signal detection competition

Examples: the Message Understanding Conferences (MUC), Text Retrieval Conferences (TREC), KDD Cup, M3 Time Series competition


The dimacs working group on disease and adverse event surveillance

HOW CAN THIS BE ACCOMPLISHED

  • Definitions of signals.

  • Test data sets for refining signal detection procedures.

  • Modular, interoperable signal generation algorithms.

  • Computing efficiencies for Monte Carlo simulations of signal detection events in large complex data.

  • Multidimensional graphical displays to interpret results and evaluate algorithms.

  • Multivariate statistical techniques for evaluating signal detection profiles across multiple data sources.


The dimacs working group on disease and adverse event surveillance

COMPETITION STATUS

  • DIMACS Working Group on Adverse Event and Disease Reporting, Surveillance, Analysis

  • Subgroup focused on competition; applied for funding; identified data sources

  • Key challenge: appropriate methods for inserting signals into real data (“spiking”)

  • Other groups face the same challenge (e.g. BioStorm)


The dimacs working group on disease and adverse event surveillance

CONCLUSION

  • Short-term goals/benefits:

    • Promote coordination and collaboration

  • Long-term goals/benefits

    • Stimulate methodological research

    • Provide objective evaluation of competing algorithms

    • Produce high quality spiking algorithms


The dimacs working group on disease and adverse event surveillance

ANALYTICAL METHODSFOR HEALTH SURVEILLANCE

DAVID MADIGAN

DEPARTMENT OF STATISTICS

RUTGERS UNIVERSITY


Novel surveillance applications methodologies

Novel Surveillance Applications Methodologies

  • Early Aberration Reporting System (EARS), CDC

  • What’s Strange About Recent Events? (WSARE), U of Pittsburgh and Carnegie-Mellon U

  • Spatial and Space-Time Scan Statistics (SaTScanTM – Kulldorff)

  • Web Visual Data Mining Environment (WebVDME), Lincoln Technologies, Inc.


Novel surveillance applications projects

Novel Surveillance Applications Projects

  • Electronic Surveillance System for the Early Notification of Community-based Epidemics (ESSENCE I&II), DOD

  • Real-time Outbreak and Disease Surveillance (RODS), U of Pittsburgh

  • Biological Spatio-Temporal Outbreak Reasoning Module (BioSTORM), Stanford U

  • Rapid Syndrome Validation Project (RSVP), Sandia NL, NM

  • Alternative Surveillance Alert Program (ASAP), Health Canada

  • Syndromic Surveillance Project, NYC

  • Bioterrorism Syndromic Surveillance Demonstration Program, CDC/Harvard


Conceptual taxonomy

Conceptual Taxonomy

Public Health Surveillance

Adverse event

(to intervention exposure)

Disease

Drug

Vaccine

Syndromic

Traditional

Infectious disease

Other

Birth defect

Injuries

Etc.


  • Login