Role Prediction Using Electronic Medical Record System Audits

1. Wen Zhang1, Carl Gunter3, David Liebovitz4, Jian Tian1 , Bradley Malin1,2 1Dept. of Electrical Engineering & Computer Science, Vanderbilt University 2Dept. of Biomedical Informatics, Vanderbilt University 3Dept. of Computer Science, University of Illinois at Urbana Champaign 4Dept. of Medicine, Northwestern University Role Prediction Using Electronic Medical Record System Audits 1

2. Misuse of EMR Systems is Real • Medical center employees misuse medical record systems to breach privacy • The problem is not limited to celebrity snooping • HIPAA Security Rule  Access to EMRs should be limited • But how? 2

3. Basic security principle: Least privilege Separation of duty Access control technologies have been around since the 1970’s Information systems often provide role-based access control (RBAC) capability[1] Privileges mapped roles Users mapped to privileges Roles are hard to define, so EMR systems often provide broad access rights Challenges to Security in EMRs [1] R.Sandhu, E.Coyne, H.Feinstein and C.Youman. IEEE computer. 1996. 3

4. In “Rare” Cases – Break the Glass • A user may not sufficient access rights to perform job • This model allows users to temporarily escalate privilege • Access is logged and reviewed by administrator • May require user to specify “reason” for access

5. Rare Cases? • Central Norway Health Region enabled break the glass • 53,000 of 99,000 patients (54.5%)  broken glass • 5,000 of 12,000 users (42.7%)  broke the glass • Over 295,000 logged breakage events in one month [3] L. Røstad and N. Øystein. Proceedings of the 2nd International Conference on Availability, Reliability and Security (ARES)

6. Idea! Refine Access ControlBased on Behavior • Experience-based Access Management (EBAM) • Combine static knowledge (RBAC) with actual actions (access logs) and organizational knowledge for feedback control EMR Access Logs Experience-Based Access Management[2] RBAC Medical Center Knowledge [2] C.Gunter, D.Liebovitz, B.Malin. IEEE Security and Privacy Magazine. 2011. 6

7. The Role Prediction Problem for EBAM • Use audit logs to predict if a user is associated with a role • Goals: • Determine if expert-defined job titles are reasonable • Provide administrators with a better idea of how to refine roles Doctor Role Classifier Nurse …. Access Reason Medical Service Biller Location of Patient 7

8. Evaluation with Cerner EMR of Northwestern Memorial Hospital • Example audit logs • Represent users as <Service, Reason, Location> vectors • Statistics 8

9. Leveraging Role Hierarchies • To assist in role management, we worked with organization experts to build a hierarchy (specialized to Northwestern) • Optimization Tradeoff: • Goal 1: Accuracy (should increase as we step up in hierarchy) • Goal 2: Separation of Duty (will increase as we step down) Employee … Specific Clinician Doctor Conceptual (5 roles) … … Physician Nurse Dietitian General (62 roles) … … … Specific (140 roles) Junior Dietitian Senior Dietitian 9

10. Basis of a “Role-Up” Algorithm • General idea: Audit roles at different levels of the hierarchy • Score each role in conceptual position & general position • Select role with the highest score & generalize its children • Repeat 1 & 2 until a threshold score is reached • Allow administrators to balance between the prediction accuracy and separation of duties (number of roles) 10

11. Balanced Scoring Function • Rmeasures the extent to which specificity could be kept by the node • Ameasures the extent to which predictablity could be achieved by the node 11

12. Employee 0.453 0.0441 Doctor Specific Clinician 0.410 0.476 0.224 Physician Nurse Dietary Physician 1 Physician 2 Nurse 1 Nurse 2 Junior Dietician Senior Dietician α = 0.5, Threshold = 0.4 12

13. Employee 0.0441 0.453 Doctor Specific Clinician 0.224 0.410 Physician Nurse Dietary Nurse 1 Nurse 2 Junior Dietician Senior Dietician α = 0.5, Threshold = 0.4 13

14. After one iteration, the role set is {Doctor, Nurse 1, Nurse 2, Dietary} Employee Doctor Specific Clinician Nurse Dietary Nurse 1 Nurse 2 α = 0.5, Threshold = 0.4 14

15. Training & Testing at the Same Level of the Role Hierarchy Level Accuracy Employee 82.38% Specific Clinician Conceptual 52.45% General Nurse Specific 51.34% Nurse 1 15

16. Distribution of Accuracy Over the Role Hierarchy 16

17. Most Predictable Roles 17

18. Least Predictable Roles 18

19. Number of Users in the Role Can Influence Accuracy 19

20. Case Study: Most Likely Mispredictions for Patient Care Staff Nurse 20

21. Most Likely Mispredictions 21

22. Parameter Bias Trades Between Accuracy and Separation of Duty • Biased toward Accuracy: • number of roles is small (27) • accuracy is highest (63%) • Biased toward Specificity: • number of roles is high (60) • accuracy is lower (52%) 22

23. Conclusion and Future Plans • EHR audit logs can be analyzed to determine if the users’ behaviors are consistent with their designated job titles • Role hierarchies enable automatic discovery of appropriate levels of role management • Plan to expand Role-“up” to allow for Role-“down” and Role-“over” • Need to evaluate Role-up with real hospital administrators, to assess its usability and acceptance of results 23

24. Acknowledgements • National Science Foundation • CCF-024422 • CNS-0964063 • National Library of Medicine • R01-LM010207 • Office of the National Coordinator for HIT • SHARPS (sharps.org) 24

25. Questions? wen.zhang.1@vanderbilt.edu 25