Machine Learning Basics: Advice and Resources for Clinical Researchers

1. Machine Learning Basics: Advice and Resources for Clinical Researchers Michael P. Cary, Jr., PhD, RN Associate Professor, Duke University School of Nursing Interdisciplinary Colloquium for Outcomes Research (ISCORe)

2. Agenda What is machine learning? Why should clinical researchers care? What were some challenges and opportunities to incorporating machine learning into my program of research?

3. What is Machine Learning? learning is the activity or process of gaining knowledge by studying, practicing, being taught, or experiencing something. a mechanically, electrically, or electronically operated device designed to perform a task. https://www.sas.com/content/dam/SAS/en_us/doc/whitepaper1/machine-learning-primer-108796.pdf

4. Machines Learning, Really? Machines learn by studying data to detect patterns or by applying known rules to: • Categorize like people or things • Predict outcomes based on identified patterns • Identify unknown patterns and relationships https://www.sas.com/content/dam/SAS/en_us/doc/whitepaper1/machine-learning-primer-108796.pdf

5. Why should clinicians care? BIG DATA = BIG OPPORTUNITIES

6. As a clinician, what does machine learning mean to me? Big Data ML Algorithms Clinical Insight/Informed Decisions

7. Overview of Research Program • Two interrelated areas: Preventing functional decline and re-hospitalization among vulnerable populations following post-acute rehabilitation. Using quality measures to promote system-level improvements in post-acute care (PAC) settings to achieve better patient outcomes. 1 2

8. Why Hip Fractures? Over90% of older adults admitted to the hospital for a hip fracture use Post-Acute Care for rehabilitation and recovery 29% 70% 25% die within 1 year do not regain independent mobility are permanently admitted to nursing homes

9. Why Inpatient Rehabilitation? Inpatient Rehabilitation Facilities (IRFs) provide intensive rehabilitation services Little evidence about: • Specific individual characteristics of orthopedic patients(demographic and clinical) • Vulnerable subgroups (e.g., racial/ethnic minorities)

10. Study 1: Individual Characteristics and PAC Outcomes Examine the influence of individual characteristics • Predisposing: Age, Race, Sex • Enabling: Social Support (live with others) • Need: Functional Status at Admission, Comorbidities --- ON --- PAC outcomes (patient-level) • Functional Status at Discharge • Community Discharge (home)

11. Methods Design: Retrospective cohort study Setting: 1,112 Medicare certified IRFs in the US Population: Adults 65 yrs. and older, Medicare fee-for-service insurance, with a fractured hip in 2009 Sample: Excluded if: (1) did not live at home prior to acute care hospital admission; (2) stayed in the IRF for <3 days or >30 days after admission; or (3) if they died during the rehabilitation stay. Final sample size was N=34,984.

12. Analysis Multivariate regression analysis was used to determine the relationships between independent variables and functional status at discharge. Logistic regression analysis was used to determine which independent variables were associated with discharged home.

13. Findings PAC outcomes in this national study were predicted by a variety of individual characteristics, most notably enabling and need factors. Subgroups at risk for poorer outcomes: • Older patients • Minorities (Blacks, Hispanics) • Men • Persons with greater disease severity (increase number of comorbidities) Cary MP Jr, et al, (2016). Inpatient rehabilitation outcomes in a national sample of Medicare beneficiaries with hip fracture. Journal of Applied Gerontology, 35(1),62-83.

14. Roadblocks to Improving PAC Hip Fracture Outcomes No real-time data Comorbidity measures included simple counts

15. Next Steps: Hip Fractures, Comorbidities, and PAC Outcomes Poor health outcomes increase with the number of comorbid conditions Little guidance is offered to clinicians in adjusting care for IRF patients with MCCs Readmissions are high among IRF patients; most risk prediction models perform poorly

16. Study 2: MCCs and Hospital Readmission Aim 1 Aim 2 Identify clusters of MCCs associated with hospital readmission among hip fracture patients discharged from IRFs. • Aim 1a: Determine the effects of pre-specified clusters of MCCs on hospital readmission. • Aim 1b: Develop machine learning algorithms to identify computer-generated clusters of MCCs associated with hospital readmission. Assess the performance of the traditional statistical model (Aim 1a) and the machine-learning model (Aim 1b) using receiver operating curves and the area under the curve as well as other measures of accuracy.

17. Challenges Training Infrastructure (Accessing resources and developing a team)

18. Training Opportunities & Resources Short Courses and Workshops Formal Programs Other Resources • + Data Science includes online and in-person learning • Machine Learning Summer & Winter School • Machine Learning for the School of Medicine • Master in Interdisciplinary Data Science • Clinical mentor and/or capstone partner • Duke University Professional Studies Online Program in Big Data and Data Science • Duke Forge • Machine Learning community at Duke • ML Day

19. Training Opportunities & Resources NIH-funded Centers, Workshops, and Fellowships • Big Data 2 Knowledge • Data Science @ NIH • NINR’s Methodologies Boot Camp Other Resources • NC State Data Matters • SAS Academy for Data Science (Big Data + Analytics) • UNC Coding and Data Analytics Bootcamp

20. Build an Interdisciplinary Team CTSA Biostats Core ML Expert Sponsor Sponsor Principal Investigator $$ $$ IT/Database Admin Data Manager/ Programmer Statistician

21. Lessons Learned Along the Way • Secure sufficient funding 2. Understand basic differences between ML & Statisticians 3. Determine desired level of involvement and understanding 4. Build an Interdisciplinary Team 5. Assess IT infrastructure and systems

22. Acknowledgements Duke Clinical & Translational Science Institute • CTSA PI, L. Ebony Boulware, MD, MPH • CTSA KL2 Leaders: Laura Svetkey, MD, Kimberly Johnson, MD, MHS • Rasheed Gbadegesin, MD, MBBS CTSA KL2 Mentoring Team/Collaborators • Cathleen Colón-Emeric, MD, MHS • Elizabeth Merwin, PhD, RN • Wei Pan, PhD • Qing Yang, PhD • Sathya Amarasekara, MS • Lesley Curtis, PhD • Lawrence Carrin, PhD • Sayan Mukherjee, PhD • Rachel Drelos, MD/PhD student Duke University School of Nursing

23. Michael P. Cary, Jr., PhD, RN Associate Professor Duke University School of Nursing Office: Pearson Building, Room 3134 307 Trent Drive, Durham, NC 27710 Phone: (919) 613-6031 Email: michael.cary@duke.edu