Using hmo claims data and a tree based scan statistic for drug safety surveillance
Download
1 / 43

Using HMO Claims Data and a Tree-Based Scan Statistic for - PowerPoint PPT Presentation


  • 322 Views
  • Uploaded on

Using HMO Claims Data and a Tree-Based Scan Statistic for Drug Safety Surveillance. Martin Kulldorff Department of Ambulatory Care and Prevention Harvard Medical School and Harvard Pilgrim Health Care.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Using HMO Claims Data and a Tree-Based Scan Statistic for ' - Jeffrey


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Using hmo claims data and a tree based scan statistic for drug safety surveillance l.jpg

Using HMO Claims Data and a Tree-Based Scan Statistic for Drug Safety Surveillance

Martin Kulldorff

Department of Ambulatory Care and Prevention

Harvard Medical School

and Harvard Pilgrim Health Care


Slide2 l.jpg

Supported by grant HS10391 from the Agency for Healthcare Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

Project Collaborators:

Richard Platt, Parker Pettus, Inna Dashevsky, Harvard Medical School and Harvard Pilgrim Health Care

Robert Davis, CDC

etc


Note of caution l.jpg
Note of Caution Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

  • Methodological Talk

  • Substantive results shown are very preliminary from the very first early testing phase of the project.


Basic idea l.jpg
Basic Idea Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

  • Drug safety surveillance is important, since some drugs may cause unsuspected adverse events (e.g.Thalidomide)

  • Use HMO data on drug dispensings and diagnoses of potential adverse events

    Data mining:

  • For a particular diagnosis, evaluate all drugs

  • For a particular drug, evaluate all diagnoses


Slide5 l.jpg

HMO Research Network: Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .Center for Education and Research in Therapeutics

  • Fallon Community Health Plan (Massachusetts)

  • Group Health Cooperative (Washington State)

  • Harvard Pilgrim Health Care (Massachusetts, grantee organization)

  • Health Partners (Minnesota)

  • Kaiser Permanente Colorado

  • Kaiser Permanente Georgia

  • Kaiser Permanente Northern California

  • Kaiser Permanente Northwest (Oregon)

  • Lovelace (New Mexico)

  • United Health Care


Slide6 l.jpg

HMO Data Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

  • #HMOs: 10

  • Members: ~ 10.7 million

  • Women: 51%

  • Age <25: 34%

  • Age 25-65: 53%

  • Age 65+: 13%

  • One year retention: ~80%


Three major methodological issues l.jpg
Three Major Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .Methodological Issues

  • Granularity: Is increased risk related to a specific drug or a group of related drugs?

  • Adjusting for Multiple Testing

  • Calculating Expected Counts


Outline l.jpg
Outline Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

  • Tree Based Scan Statistic

  • Application to Heart Attacks, Scanning All Drugs

  • Calculating Expected Counts

  • Future Plans


Nested variables l.jpg
Nested Variables Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

ecotrin Ì asprin Ì nonsteoridal

anti-inflammatory drugs Ì analgesic drugs

acute lymphomblastic leukemia Ì acute leukemias Ì leukemia Ì cancer


Drug tree based on american society for health system pharmacists ahfs classification l.jpg
Drug Tree Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .Based on American Society for Health-System Pharmacists (AHFS) Classification

Level 1, with 18 groups:

  • Antihistamine Drugs (04)

  • Anti-infective Agents (08)

  • Antineoplastic Agents (10)

  • Autonomic Drugs (12)

  • Blood Formation and Coagulation (20)

  • Cardiovascular Drugs (24)

    etc


Drug tree l.jpg
Drug Tree Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

Level 2:

Anti-infective Agents (08)

  • Amebicides (0804)

  • Anthelmintics (0808)

  • Antibacterials (0812)

  • Antifungals (0814)

  • Antimycobacterials (0816)

    etc


Drug tree12 l.jpg
Drug Tree Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

Level 3:

Anti-infective Agents (08)

  • Antibacterials (0812)

    - Aminoglycosides (081202)

    - Antifungal Antibiotics (081204)

    - Cephalosporins (081206)

    - Miscellaneous Lactams (081207)

    etc


Drug tree13 l.jpg
Drug Tree Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

Level 5, generic drugs (1009 total):

Anti-infective Agents (08)

  • Antibacterials (0812)

    - Aminoglycosides (081202)

    - Gentamicin (081202-0002)

    - Geomycin (081202-0004)

    - Tobramycin (081202-0007)


Slide14 l.jpg

A Small Two-Level Tree Variable Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

Root

Node

Branches

Leaf

Drug A1

Drug A2

Drug A3

Drug B1

Drug B2


Granularity problem l.jpg
Granularity Problem Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

Analysis Options

  • Evaluate each of the 1009 generic drug, using a Bonferroni type adjustment for multiple testing.

  • Use a higher group level, such as level 3 with 184 drug groups.

    Problem: We do not know whether a potential adverse event is due to a smaller or larger drug group.


Analysis options the other extreme l.jpg
Analysis Options Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .The Other Extreme

  • Take the 1009 generic drugs as a base, and evaluate all 21009 - 2 = 5.49 ´ 10303 combinations.

Problem: Not all combinations are of interest.


Ideal analytical solution l.jpg
Ideal Analytical Solution Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

  • Use the Hierarchical Drug Tree

  • Evaluate Different Cuts on that Tree


Slide18 l.jpg

Cutting the Tree Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

Cut

Drug A1

Drug A2

Drug A3

Drug B1

Drug B2


Problem l.jpg
Problem Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

How do we deal with the multiple testing?


Proposed solution l.jpg
Proposed Solution Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

Tree-Based Scan Statistic


One dimensional scan statistic studied by naus jasa 1965 l.jpg
One-Dimensional Scan Statistic Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .Studied by Naus (JASA, 1965)


Other scan statistics l.jpg
Other Scan Statistics Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

  • Spatial scan statistics using circles or squares.

  • Space-time scan statistics using cylinders, for the early detection of disease outbreaks.

  • Variable size window, using maximum likelihood rather than counts.

  • Applied for geographical and temporal disease surveillance, and in many other fields.


Tree based scan statistic l.jpg
Tree-Based Scan Statistic Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

H0: The probability of a diagnosis after the dispensing of a drug is the same for all drugs.

HA: There is at least one group of drugs after which the probability of diagnosis is higher

. . . after various adjustments


Tree based scan statistic24 l.jpg
Tree-Based Scan Statistic Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

  • For each generic drug we have:

  • observed number of diagnosed cases

  • expected number of diagnosed cases,

  • adjusted for age and gender


Tree based scan statistic25 l.jpg
Tree-Based Scan Statistic Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

  • 1. Scan the tree by considering all possible cuts on

  • any branch.

  • 2. For each cut, calculate the likelihood.

  • 3. Denote the cut with the maximum likelihood

  • as the most likely cut (cluster).

  • 4. Generate 9999 Monte Carlo replications under H0, conditioning on the observed number of total cases.

  • 5. Compare the most likely cut from the real data set

  • with the most likely cuts from the random data sets.

  • 6. If the rank of the most likely cut from the real data

  • set is R, then the p-value for that cut is R/(9999+1).


Log likelihood ratio l.jpg
Log Likelihood Ratio Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

cG = observed cases in the cut defining drug group G

Ng = expected cases in the cut defining drug group G

C = total number of observed cases = total number of expected cases


Example acute myocardial infarction ami l.jpg
Example: Acute Myocardial Infarction (AMI) Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

  • Sample of Harvard Pilgrim Health Care Data

  • 376,000 patients

  • Years 1999-2003

  • 2755 AMI diagnoses

    [Acute Myocardial Infarction = heart attack]


Results most likely cut l.jpg
Results Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .Most Likely Cut

Drug(s): Nitrates and Nitrites (241208)

Observed: 98 Expected: 7.3 O/E=13.4

LLR = 165.0, p=0.0001


Results second most likely cut l.jpg
Results Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .Second Most Likely Cut

Drug: Nitroglycerin (241208-0004)

Observed: 77, Expected: 6.2, O/E=12.5

LLR = 124.3, p=0.0001


Results top 10 cuts l.jpg
Results: Top 10 Cuts Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

Obs Exp O/E LLR Drug(s) .

98 7.3 13.4 165.0 Nitrates and Nitrites (241208)

77 6.2 12.5 124.3 Nitroglycerin (241208-0004)

110 15.3 7.2 123.4 Vasodilating Agents (2412)

88 11.8 7.4 101.2 Adrenergic Blocking Agents (2424)

88 11.8 7.4 101.2 Adrenergic Blocking Agents (242400)

36 1.3 27.0 84.1 Clopidogrel (920000-0078)

209 74.6 2.8 83.6 Cardiovascular Drugs (24)

28 1.1 24.8 63.1 Isosorbide (241208-0003)

52 7.7 6.8 55.4 Atenolol (242400-0002)

32 2.9 10.9 47.5 Metoprolol (242400-0009) .

p=0.0001, for all cuts


Results tree format l.jpg
Results, Tree Format Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

Obs Exp O/E LLR Drug(s) .

209 74.6 2.8 83.6 Cardiovascular Drugs (24)

110 15.3 7.2 123.4 Vasodilating Agents (2412)

98 7.3 13.4 165.0 Nitrates and Nitrites (241208)

28 1.1 24.8 63.1 Isosorbide (241208-0003)

0 0.0002 0 - Amyl (241208-0001)

77 6.2 12.5 124.3 Nitroglycerin (241208-0004)

5 6.7 0.7 - other 7 VA (2412xx)

88 11.8 7.4 101.2 Adrenergic Block Agents (2424)

88 11.8 7.4 101.2 Adrenergic Block Agents(242400)

52 7.7 6.8 55.4 Atenolol (242400-0002)

32 2.9 10.9 47.5 Metoprolol (242400-0009)

4 1.0 3.9 - other 11 ABA (242400-xxxx)

147 39.8 3.7 - other Cardiovascular Drugs (24xxxx)


Interpretation of results l.jpg
Interpretation of Results Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

People with cardiovascular problems are often taking cardiovascular drugs and they are also at higher risk of AMI.


Observed and expected counts l.jpg
Observed and Expected Counts Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

  • Exposed to drug, had AMI

  • Exposed to drug, no AMI

  • Unexposed to drug, had AMI

  • Unexposed to drug, no AMI


Observed counts l.jpg
Observed Counts Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

  • Use only incident diagnoses

  • Ignore the time after the incident diagnosis

  • New drug users vs. prevalent users

  • Length of drug exposure time window

  • Cover gaps in drug dispensings

  • Use ramp-up period before starting to count


Multiple drugs l.jpg
Multiple Drugs Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

  • Individuals may simultaneously be “exposed” to multiple drugs

  • Observed counts are adjusted for multiple drug use

  • Expected counts are simply added for different drugs, ignoring multiple drug use.

    Alternative

  • Assign each day as exposed to at most one drug, selecting the most uncommon one.


Comparison group l.jpg
Comparison Group Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

  • All non-exposed days

  • Remove days exposed to cardiovascular drugs when evaluating cardiovascular diagnoses

  • Censor individuals the day they start using a cardiovascular drug

  • Other drug users, removing non-drug users


Covariate adjustments l.jpg
Covariate Adjustments Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

  • Age

  • Gender

  • HMO

  • Temporal or seasonal trends

  • Frequency of drug use

  • Disease risk factors (?)


Data mining a cautious approach l.jpg
Data Mining: Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .A Cautious Approach

  • Purpose is to generate unsuspected signals

  • Generated signals that must be interpreted from a clinical perspective.

  • Signals may be unexpected/important or expected/unimportant.

  • If signals are not immediately dismissed, they should be evaluated using standard epidemiological methods.


Tree scan statistics future developments l.jpg
Tree Scan Statistics: Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .Future Developments

  • Simultaneous use of multiple trees

  • Scan diagnoses for a particular drug

  • Simultaneous scanning of drugs and

  • diagnoses using two intersecting trees

  • Drug-drug interaction effects

  • Sequential monitoring of new drugs

  • Development of TreeScan software


Final remarks l.jpg
Final Remarks Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

  • HMO data shows promise for drug safety surveillance

  • The tree scan statistic can be used to solve the problems of granularity and multiple testing

  • Calculating observed and expected counts is complex and critical

  • Data mining generates rather signals that need to be confirmed/rejected using other methods

  • Adopt other data mining methods for HMO data


Reference l.jpg
Reference Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

Kulldorff M, Fang Z, Walsh SJ. A tree-based scan statistic for database disease surveillance. Biometrics, 59:323-331, 2003.


Comparison with computer assisted regression trees cart l.jpg
Comparison with Computer Assisted Regression Trees (CART) Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

Four Similarities: ‘T’, ‘R’, ’E’ and ‘E’


Difference l.jpg
Difference Research and Quality (AHRQ) to the HMO Research Network Center for Education and Research in Therapeutics (CERT) in collaboration with the FDA through Cooperative Agreement FD-U-002068 .

CART: There are multiple continuous or categorical

variables, and a regression tree is constructed by

making a hierarchical set of splits in the multi-

dimensional space of the independent variables.

Tree-Based Scan Statistic: There is only one independent variable (e.g. drug). Rather than using this as a continuous or categorical variable, it is defined as a tree structured variable. That is, we are not trying to estimate the tree, but use the tree as a new and different type of variable.


ad