Time Series and Risk Adjusted Control Charts. Michael E. Matheny, MD, MS, MPH TVHS Veteran’s Administration Division of General Internal Medicine Department of Biomedical Informatics Department of Biostatistics Vanderbilt University Medical Center Nashville , TN. Objectives.
Michael E. Matheny, MD, MS, MPH
TVHS Veteran’s Administration
Division of General Internal Medicine
Department of Biomedical Informatics
Department of Biostatistics
Vanderbilt University Medical Center
Statistical Process ControlIndustrial Methods
Source: Oakland, JS. Statistical Process Control, 5th Ed. Butterworth-Heinemann, 2003
Source: Matheny et al. Medical Decision Making. 2009;29:247-56.
AUC 0.70 (0.62-0.78)
AUC 0.80 (0.74-0.87)
Source: Austin, PC. Biometrical J. 2009;51:171-184.
Source: Matheny et al. AMIA AnnuSymp Proc. 2007;518-522.
Matched 92.4% (1,144/1,238) by calendar quarter
Critical risk factor was location of access site, which was not captured in the routine ACC-NCDR (or Mass-DAC) datasets.
Automated surveillance is hypothesis generating – all alerts REQUIRE detailed review and confirmation
Source: Tiroch K et al. American Journal of Cardiology. 2008;102:1473-6.
Use of an alpha spending function to adjust per-period alerting boundary thresholds
Massachusetts PCI 2004-2007
Evaluation ofcumulative post-procedure myocardial infarction rate for new drug eluting stent as compared with propensity matched control DES.
Propensity score matching resulted in 81.5% of 18,277 Taxus Express2 devices matched and analyzed.
Source: Resnic et al. JAMA 2010;304(18):2019-2027.
Formal framework for incorporating ά and β error
Specify Odds Ratio of event rate elevation detection desired
Risk Adjustment using a binary outcome risk model to adjust the cumulative log odds
Source: Spiegelhalter, et al. International Journal of Quality Healthcare 2003;15:7-13
Brigham & Women’s Hospital (01/2002 – 10/2006)
All Operators (18) who performed PCI on patients (8750)
Inpatient Mortality (125) (1.4%)
National ACC-NCDR Data
Local BWH Data
Risk Adjusted SPRTInterventional Cardiology Operator Assessment Example
Source: Matheny et al. American Heart Journal. 2008;155:114-20.
Evaluated each fatal case for the operator in question, and noted a high compassionate use rate compared to other operators
After excluding patients that were not considered candidates for CABG or were clearly documented as extremely high-risk compassionate care, operator did not exceed mortality rate expectations
Massachusetts CABG Mortality 2002-2007
2 known institutional outliers in 5 years of data (4 and 1)
Both methods detected all true positives, SPRT had 1 false positive
Source: Matheny et al. BMC Med. Inform. Dec. Mak. 2011;11:75.
Source: Li, Kulldorff.Statist. Med. 2010;29:284-295
Source: Greene et al. Pharmacoepidemiol Drug Saf. 2011 Jun;20(6);583-90