Using Clinical Data to Answer a Research or Quality Improvement Question

1. Using Clinical Data to Answer a Research or Quality Improvement Question Family Medicine Residency Network Scholarship Webinar February 2019 Allison Cole, MD, MPH Director, WWAMI region Practice and Research Network Associate Professor, Family Medicine

2. Learning Objectives Identify sources of clinical data for research and QI projects Understanding how to develop research or QI questions that can be answered using clinical data Understand and plan data collection activities to support research and QI efforts

3. Sources of Data in Clinical Practice • Electronic health records • Patient or provider surveys • Patient, provider or staff interviews

4. Sources of Data in Clinical Practice • Electronic health records • Patient or provider surveys • Patient, provider or staff interviews

5. Card Studies • Brief anonymous questionnaires • Low cost • Observational study • Estimate prevalence of conditions • Get data directly from patients or providers • Assess delivery of care, quality of care, patient or provider preferences Westfall JM et. Al. Card Studies for Observational Research in Practice. Annals of Family Medicine 2011;9:63-68.

6. Card Study Example J Am Board Fam Pract January 1988 1:15-23; doi:10.3122/jabfm.1.1.15

7. Examples of Card-Study Appropriate Research Questions • What proportion of patients in primary care practice experience food insecurity? • What proportion of patients seen for upper respiratory tract infection symptoms request a prescription for an antibiotic medication? • How often do primary care providers make referrals to complimentary and integrative health services and for what conditions?

8. WPRN Card Study: QUESTIONS • What proportion of overweight and obese patients would participate in a comprehensive weight loss program? • What are the most important factors in helping overweight and obese patients decide whether or not to participate?

9. Study Methods • Patient population: • Adult (ages 18+) patients • Pregnant women excluded • Implementation • Administered over 2 week period in June 2013 • Handed out to patients at front desk check-in • Clinics: 12 participating sites across 5 states • Survey characteristics: • Available in English and Spanish • Anonymous • IRB exempt

12. Surveys Returned By Site Surveys returned Percent of population reached

13. Demographics Sex Missing 0.3%

14. Overall Health Missing 1%

15. Insurance Status Missing 4%

16. Race/ethnicity Missing 3.5%

17. Body Mass Index (BMI) Missing 3%

18. Patient reported reasons for weight loss N=1174. 1300+ responses

19. N=1243 Missing 2%

20. EHR Data Query • Widespread adoption of EHRs in primary care practice • EHR-generated data can be helpful for population health management and QI activities • Most clinical practices have HIT analysts who can conduct queries • UW Medicine has created a data repository that can be queried to examine questions relevant to UWNC Northgate and Harborview

21. WHAT EHR DATA CAN AND CAN’T DO Pitfalls to EHR data • Inaccuracy • Incomplete • Badly transformed to make meaningless (e.g., billing ICD-9 codes) • Unrecoverable for research (e.g., stored in text only) • Of unknown provenance (i.e., can’t determine where they came from or what exactly they mean) • Insufficient granularity • Incompatible with research protocols (e.g., no ability to determine accurate onset or treatment change times frames) Hersh et al. (2014). Recommendations for the Use of Operational Electronic Health Record Data in Comparative Effectiveness Research. eGEMs (Generating Evidence & Methods to improve patient outcomes): Vol. 1: Iss. 1, Article 14

22. Recommendations for using EHR data in a study: 4 steps to an evidenced based approach • Ask an answerable question – Ask whether a test or treatment is efficacious is not enough. We need to know at a minimum the effectiveness of the intervention relative to some alternative approach in a particular patient population or setting. • Find the best evidence – The “evidence” / data in EHRs may be incomplete, incorrect, and inconsistent. We need to determine if we have the proper data and, if so, whether they can be applied to answer our question. • Critically appraise the evidence– Can we trust the inferences and conclusions from the data? Are there confounding variables in those data? Have the appropriate methods been used? These may be critical with EHR data where assignment of cause and effect could be difficult, if not impossible. The solution likely comes back to asking the right question, i.e., one for which we can have confidence that we have found the correct answer. • Apply it to the patient situation– Can the evidence be applied in our setting? Just as some RCTs answer questions in patient populations very different from those of the clinician making decisions, we must ascertain if the results obtained from this approach can be applied to a specific clinical context. Hersh et al. (2014)

23. Litmus tests for YOUR research questions • Does your question require data from the “whole picture” of care (care occurring outside your practice and therefore EHR)? Don’t ask about completeness of care ideas, given patients may not get all their care in your clinic. • Will your question be affected over time by practice changes that would significantly affect documentation practices? • If your question requires a start and stop period of time, can you reliably determine these times from the data in the EHR? • Have there been significant coding changes over time for the data you’re interested in that may inadvertently look like a clinical change that doesn’t exist? • Are your data available in coded fields or does it have to be mined out of text notes, etc.? • Do the data provide sufficient granularity that you need for your study? Hersh et al. (2013). Caveats for the Use of Operational Electronic Health Record Data in Comparative Effectiveness Research. Medical Care, 8(3), S30-S37.

24. Research Question: Make it Simple! 1) What proportion of adult patients, seen for a primary care visit in the last year, has been prescribed any opioid medication in the past year? 2) Of adult patients prescribed an opioid medication in the past year, what proportion was prescribed a sedative in the same year?

25. Study Parameters • Study period: October 1, 2015 - September 30, 2016 • Adult Patients: Ages 18 and over • Visit: In-person primary care office visit with a medical provider (MD, DO, NP, PA) • Opioid & Sedative Prescriptions: Opioid and sedative medications on lists provided, that had a “prescription” date in study period, regardless of where the prescription originated (e.g., in clinic or an outside clinic)

26. EHR Data Results • Percent of Adult Patients With an Opioid Prescription in One Year Study Period n= 10,027/ 64,140

27. EHR Data Results • Of Patients with Opioid Prescription: Percent With a Sedative Prescription in One Year Study Period n= 2125/ 10,027

28. A collaborative group of primary care practices in the WWAMI region committed to research and practice improvement WWAMI region Practice and Research Network University of Washington – Seattle, WA Institute of Translational Health Sciences and Department of Family Medicine

29. WPRN Mission and Values • To improve the health and well-being of patients in their communities, through the conduct of collaborative research that informs and enhances primary care clinical practice. • Maintain an equal and bi-directional partnership between academic researchers and community practices • Support practice innovation and quality improvement through research • Conduct research of importance to member practices

30. WWAMI region Practice and Research Network • Over 60 clinics across 28 organizations • 18 primary care EHR-connected clinics

31. WPRN Clinical Practices Serve largely rural and/or underserved populations Annual number of outpatient visits from 6,000-53,000/organization (over 770,000 total) Number of physician providers ranges from 1 to 20 + (over 350 MDs, DOs, NPs, PAs total) • 15 Community Health Centers/FQHCs • 26 Hospital Affiliated Clinics • 17 Rural Health Clinics • 2 Other Diverse Settings • Epic • Centricity • Allscripts • Others Electronic Health Records

32. WPRN Coordinating Center Allison Cole, MD, MPH Associate Professor WPRN Director Angelina Kolomiets, MPH ITHS Program Manager WPRN Coordinator Laura-Mae Baldwin, MD, MPH Professor WPRN Senior Advisor Gina Keppel, MPH Research Scientist WPRN Coordinator

33. Areas of Interest to WPRN members include… Use of the electronic health record (EHR) as a tool for research that improves quality of care Chronic disease management Prevention of adverse drug events Cancer screening and prevention Chronic pain and opioid addiction Behavioral health and primary care integration Oral health in primary care Mobile health technology

34. WPRN Champion Activities

35. Data QUEST An electronic health data-sharing architecture across community-based primary care practices in the WPRN • 18 data-connected clinics in the WPRN • Represents over 150,000 active patients

36. Thank You

37. Questions?