1 / 13

Clinical Event Classification: Strategies and Practices

This presentation discusses the necessity of clinical event classification, based on the experience in the Women's Health Initiative study. It explores the possible implications and best practices for classification. Comparison of clinical event classification and claims data is also evaluated. The intention to treat comparison and insights from comparing clinical event classification with claims are presented, along with reflections on best practices.

leevasquez
Download Presentation

Clinical Event Classification: Strategies and Practices

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Clinical Event Classification: Strategies and Practices Mark A Hlatky, MD Stanford University School of Medicine September 27, 2018

  2. Outline • Is clinical event classification necessary? • Experience in WHI study • Possible implications • What are best practices when it is done? • Some take-aways from this meeting

  3. Why classify events? • What does a CEC add? • Independent, blinded adjudication of events increases credibility of study results • Standardized data collection and definitions of key outcome events • Satisfying regulatory requirements for new drug approval • Cost, time, and complexity to RCTs • Should test value added by CEC process

  4. Comparison of CEC and claims • Pragmatic clinical trials may use existing data collection processes to simplify studies • The Women’s Health Initiative (WHI), a large NIH RCT, linked the trial database to CMS claims • Opportunity to evaluate the accuracy of hospital claims data in assessing CV outcomes: MI, CABG, and PCI Circ Cardiovasc Qual Outcomes 2014;7:157-162.

  5. Methods • Women ≥65 yo at WHI enrollment • WHI MI = best available adjudication (central > local) • CMS MI = ICD.9 code of 410.x1 (acute MI, initial admission) • Censored at end WHI follow-up, loss of Medicare Part A, or HMO enrollment • Training set = not in hormone RCTs • Application to hormone RCTs Circ Cardiovasc Qual Outcomes 2014;7:157-162

  6. Definition of MI • WHI process • Participant reported a possible event • Records requested and reviewed by CEC • MI Dx by ECG, biomarkers, symptoms • MI categorized as “resulting from a procedure” • CMS Process • 1º = principal reason for admission, • 2º = conditions also present • ICD9 Codes: 410 = acute MI, 411 = other acute CHD, 412 = old MI, 413 = angina, 414 = other chronic CHD Circ Cardiovasc Qual Outcomes 2014;7:157-162

  7. CMS MI vs. WHI MI in Training Set • 37,397 women ≥65 yo w CMS data • 1,345 WHI MI cases (172 procedure related) • 1,195 CMS MI cases (1º dx) • 306 more with acute MI as 2º dx __ WHI MI_ Kappa YesNo CMS (1º) Yes 914 281 0.71 No 431 35,771 CMS (1º, 2º) Yes 1,062 439 0.74 No 283 35,613 Circ Cardiovasc Qual Outcomes 2014;7:157-16

  8. Reasons for CMS Yes/WHI No • 281 with CMS MI = 1º MI dx, no WHI MI • No event reported to WHI (49%) • Inadequate documentation—no records (5%) or key data missing (9%) • Adjudicated, but • No CHD outcomes (16%) • Another CVD outcome (23%) • CHD deaths w/o MI (2%) Circ Cardiovasc Qual Outcomes 2014;7:157-16

  9. Reasons for CMS No/WHI Yes • 431 with WHI MI, no CMS MI (1º diagnosis) • No CMS admission <30 days (15%) • CMS 2º dx of MI (32%) • WHI MI procedure related (13%) • Other CVD 1º dx (39%) • Non-CVD 1º dx (15%) Circ Cardiovasc Qual Outcomes 2014;7:157-16

  10. Intention to treat comparison

  11. Inights from comparing CEC with Claims • Claims data identified events not reported by WHI participants • Agreement between claims and CEC data was very good for MI, and excellent for PCI and CABG (kappa 0.88 to 0.91) • Some disagreements due to: • No or incomplete records for adjudication • Differences in diagnostic criteria applied by CEC adjudicators vs local physicians • MIs due to procedures or non-cardiac illness were often missed in claims

  12. Some reflections on best practices • CEC is more valuable when study intervention is applied during hospitalization for acute illness or procedures • More difficult to identify MI in this setting • CEC more important for drugs in development than for comparative effectiveness studies of well-established drugs • Regulatory requirements to meet

  13. More reflections on best practices • Application of clear, standardized diagnostic criteria for events decreases variation • Interobservervariation is more often due to different dx criteria than to variable application of standardized dx criteria • Standardizing CEC processes, however, could impede innovation • Could test importance of key processes (e.g. meetings vs independent evaluations) • CEC might be applied to a random sample to validate process (e.g. claims) for all events

More Related