What is the Most Efficient Data Extraction Method for Quality Improvement and Research in Cardiology?: A Comparison of REMIND Artificial Intelligence Software vs. Manual Chart Abstraction for Determining ACC/AHA Guideline Adherence in Non-ST Elevation Acute Coronary Syndromes.
What is the Most Efficient Data Extraction Method for Quality Improvement and Research in Cardiology?:
A Comparison of REMIND Artificial Intelligence Software vs. Manual Chart Abstraction for Determining ACC/AHA Guideline Adherence in Non-ST Elevation Acute Coronary Syndromes
Ali F. Sonel, MD, C. Bernie Good, MD MPH, Harsha Rao, MD, Alanna Macioce, BS, Lauren J. Wall, BS, Radu Stefan Niculescu, MS,
Sahtyakama Sandilya, PhD, Phan Giang, PhD, Sriram Krishnan, PhD, Prasad Aloni, MS, MBA, Bharat Rao, PhD
Center for Health Equity Research and Promotion, VA Pittsburgh Healthcare System and the Cardiovascular Institute, University of Pittsburgh Pittsburgh, PA, Siemens Medical Solutions, USA, Malvern, PA
Introduction: Manual extraction of data for Quality Improvement is tedious, requiring significant individual training and careful attention to the HIPAA Privacy Rule. Automated chart abstraction is an alternative approach that saves time and costs. We compared manual chart abstraction from an electronic medical record (VA CPRS EMR System) to automated extraction using the REMIND artificial intelligence software in 327 consecutive patients admitted with unstable angina or non-ST elevation myocardial infarction.Methods: All patient features required by ACC/AHA guidelines for determining eligibility for class I recommendations to use ACE inhibitors and glycoprotein IIb/IIIa treatment were extracted by both methods. Manual extraction was carried out by well-trained, qualified chart abstractors with prior experience in manual chart abstraction. When both extraction results were identical, the result was assumed correct. Disagreements were manually adjudicated based on pre-determined definitions.Results: Manual extraction and data entry required 136 hours compared to 3 hours using the Siemens REMIND software. A total of 2289 data elements were identified, with agreement in 1912 (84%)and disagreement in 377, involving 2.5-35% of patients for various parameters. REMIND was found to be correct in 215/377 disagreements (57%) and manual extraction was correct in the remaining 43% (162). Based on adjudication, guideline adherence for ACE inhibitor and glycoprotein IIb/IIIa receptor antagonist use were 58.5% and 38.2% respectively. REMIND identified adherence at 55.7% and 38.2% respectively, which was more accurate than guideline adherence determined by manual extraction (64.8% and 33.3%).Conclusions: REMIND can assess ACC/AHA guideline adherence at least as accurately as manual chart abstraction. Use of REMIND for Quality Improvement and research can result in significant savings, better resource utilization, and may improve data extraction quality.
Table 3: Accuracy* of Compliance Assessment with REMIND Compared to Manual Extraction
Table 1: Determination of Contraindications and Eligible Patients for Processes of Care
*Accuracy defined as true positives plus true negatives divided by the total number of patients
Table 2: Assessment of Compliance with Guideline Recommended Therapies