An Empirical User Study of a Smartphone-Based Access-Control System

1. An Empirical User Study of a Smartphone-Based Access-Control System Kami Vaniea Joint work with Lujo Bauer, Lorrie Cranor, Mike Reiter and Rob Reeder

2. Physical access control 2

3. Limitations Must delegate all access tokens in advance Necessary to hide an access token for emergency situations Problems getting access tokens back Once given out key can be copied Requires users to carry additional objects 3

4. Smartphones • What about using smartphones for access control? • Smartphone capabilities • User interface • Computing ability • Communication • Smartphones are increasing in popularity • Computational power of mobile phones also increasing

5. Research questions • What are the usability challenges in building a smartphone-based access-control system? • How well does a deployed smartphone-based access-control system match users’ needs?

6. Outline • Introduction • Grey Overview & Deployment • Study 1: System Acceptance • Study 2: Policy Creation • Related Work • Conclusion

7. Grey • Smartphone based access-control system • Used to open doors in the CIC building • Allows users to grant access to their doors from anywhere at any time

8. Lorrie Grey example Kami Lorrie’s Office

9. Grey advantages Can easily delegate authority In advance of the access At the time of the access Guarantee access is no longer allowed after specified time 9

10. Field trial: environment • 30 doors • Perimeter doors to a large research area • Offices • Storage closets • Conference room • A lab • A machine room

11. Users • Chose participants who work together • Wanted groups of users who share resources • 29 users • 9 faculty • 11 graduate students • 7 technical staff • 2 administrative assistants

12. Interview procedure • Interviewed participants • Security practices • Types of resources managed and needed • Gave participants a smartphone with Grey pre-installed and brief instruction on use • Interviewed one month later • Changes in security practices • General reactions to Grey • Periodically conducted follow-up interviews at approximately one month intervals

13. Data Recorded approximately 30 hours of interviews System was actively used Logged 19,500 Grey accesses for 29 users Active users averaged 12 accesses a week Five users accessed their office almost exclusively with Grey Users interacted with an average of 7.4 different doors during the study Study lasted a year 13

14. Outline • Introduction • Grey Overview & Deployment • Study 1: System Acceptance • Study 2: Policy Creation • Related Work • Conclusion

15. Research question • What are the usability challenges in building a smartphone-based access-control system?

16. Design issues • Analyzed interview data and identified five different design issues • Speed • Failures • Complex features • Non-Grey users • New uses

17. Issue 1: Perceived speed • Users quickly began to complain about speed and convenience of unlocking doors • We knew Grey and keys required similar amounts of time to open a door • Videotaped a highly trafficked door to better understand how doors are opened differently with Grey and keys

18. Issue 1: Videotaping • Videotaped participants accessing kitchenette door • Videotaped two hours daily after 6pm for two weeks • 18 users taped • 5 Grey participants • 13 additional participants were solicited as they passed through the door

19. 5.7 sec σ = 3.6 Issue 1: Average access times Keys Total 14.7 sec 3.6 sec 5.4 sec σ = 3.1 σ = 3.1 Door Closed Getting keys Door opened Stop in front of door σ = 5.6 Grey Total 15.1 sec 8.4 sec 2.9 sec 3.8 sec σ = 2.8 σ = 1.5 σ = 1.1 Door Closed Getting phone Door opened Stop in front of door σ = 3.9

20. Issue 2: Failure • Cost of failure is potentially high • Rebooting a phone or door was considered very inconvenient • Several users stopped using Grey actively after a single inopportune failure

21. Issue 2: Delays interpreted as failures • Delays can be interpreted as failures even when the system is functioning perfectly • Humans can be slow or unresponsive • Providing feedback on the status of the request is very important • Did it arrive? • Is a human currently responding?

22. Issue 3: Confusing features • Users would rather choose a suboptimal solution that they understand than one with an uncertain outcome • Initially tried for concise interface (top) • Adopted wizard solution (bottom)

23. Issue 4: Non-Grey users • Grey is a service that becomes more valuable as more people use it • Our participants were selected so that their work network included others with Grey • Still had many people who would have benefited if Grey participant could have given access

24. Issue 4: Alice’s colleagues Have Grey

25. Issue 5: Unanticipated uses • Unlocking door from inside the office without having to stand • Unlocking nearby door for someone else without leaving office

26. Study 1: summary • Perceived speed and convenience are critical to user acceptance • A single failure can strongly discourage adoption • Users won’t use features they don’t understand • Important to consider occasional users of the system • Unanticipated uses can improve acceptance

27. Outline • Introduction • Grey Overview & Deployment • Study 1: System Acceptance • Study 2: Policy Creation • Related Work • Conclusion

28. Research question • How well does a deployed smartphone-based access-control system match users’ needs? • Do users make more or less secure access-control decisions when using Grey than when using physical keys?

29. Policies • A policy is a collection of rules • A rule is a tuple containing a user, resource and condition (Bob, Alice’s office, true) Alice’s Office Bob True

30. Methodology overview • Examined access-control policies created by 8 resource owners • 8 offices • 1 machine room • Using interviews we created ideal, key and Grey policies for each of 9 resources • Compared ideal and implemented rules

31. Ideal policies Ideal Policy – Policy the user would enact if not restricted by technology Based on interview data Looked at not only what was enacted but endeavored to determine why 31

32. Policy synthesis Garry Frank Rick Larry Joan Mary . . . . . . Lab owner is notified Logged True Logged Logged False Charlie’s Lab 32

33. Ideal conditions True (can access anytime) Logged Owner notified Owner gives real-time approval Owner gives real-time approval and witness present Trusted person gives real time approval and is present False (no access) 33

34. Policy analysis We compared each of the 244 ideal access rules, with the key and Grey rules and marked them as: False Accept – User not required to fulfill all conditions required by the ideal policy False Reject – User must fulfill conditions not required by the ideal policy Faithfully Implemented – Matched the ideal policy 34

35. Policy analysis example Charlie’s Lab Faithfully implemented False Accept False Reject Alice Bob Sue 35

36. Keys vs. ideal Alice Bob User 29 Sue User 28 User 4 User 27 User 5 User 26 20 Faithful Implementations (Green) 4 False Accepts (Red) 5 False Rejects (Yellow) User 6 User 25 User 7 User 24 Charlie’s Lab User 23 User 8 User 22 User 9 User 21 User 10 User 20 User 11 User 19 User 12 User 18 User 13 User 17 User 14 User 16 User 15

37. Conditions True (can access anytime) Logged Owner notified Owner gives real-time approval Owner gives real-time approval and witness present Trusted person gives real time approval and is present False (no access) True (has a key) Ask trusted person with key access Know location of hidden key Ask owner who contacts witness False (no access) Ideal Keys ? 37

38. Key implementation accuracy Rules Ideal Conditions 38

39. Conditions True (can access anytime) Logged Owner notified Owner gives real-time approval Owner gives real-time approval and witness present Trusted person gives real time approval and is present False (no access) True (has a delegation) Ask trusted person with Grey access Ask owner via Grey Ask owner who contacts witness False (no access) Ideal Grey 39

40. Implementation accuracy Rules Ideal Conditions 40

41. Study 2: Contributions • Documented the collection of ideal policy data • Developed a metric and methodology for quantitatively comparing accuracy of implemented policies • Showed that a smarphone access-control system outperformed keys in overall security and effectiveness

42. Outline • Introduction • Grey Overview & Deployment • Study 1: System Acceptance • Study 2: Policy Creation • Related Work • Conclusion

43. Related work • Several Grey-like systems have been proposed but not implemented • Digital Key system [Beaufour and Bonnet] • The Master Key [Zhu, Mutka and Ni] • Access-control tokens are not very easy to use and those that are tend to be less secure [Braz and Robert; Piazzalunga et. al.]

44. Related work • Usability of access control for file systems • Manipulating access-control lists is difficult for users to do accurately [Cao and Iverson] • Users have difficulty understanding how rules interact to form the effective policy [Maxion and Reeder] • Studies of users’ access-control needs • Identified several different approaches to access control management [Ferraiolo et al.] • Users have dynamic access-control needs that very by task [Whalen et al.]

45. Summary • Study 1 • Users have low tolerance for failure and treat Grey like an appliance • Study 2 • Policies made using Grey were less permissive than key policies and better matched the ideal policies • Related work • Unlike previous work we study an actual working system and examine gathered empirical data

46. Future work • Explore the tasks policy authors engage in • Explore the use of a Grey like system in large organizations • Develop technologies that assist in the authoring of policies

47. CMUUsablePrivacy andSecurityLaboratoryhttp://cups.cs.cmu.edu/

48. Bibliography • X. Cao and L. Iverson. Intentional access management: Making access control usable for end-users. In Symposium On Usable Privacy and Security, 2006. • A. Beaufour and P. Bonnet. Personal servers as digital keys. In 2nd IEEE International Conference of Pervasive Computing and Communications, 2004. • C. Braz and J. Robert. Security and usability: The case of the user authentication methods. In IHM ’06, p 199-203, 2006. • D. F. Ferraiolo, D. M. Gilbert and N. Lynch. An examination of federal and commercial access control policy needs. In 16th National computer Security Conference, p 107-116, 1993.

49. Bibliography • R. A. Maxion and R. W. Reeder. Improving user-interface dependability through mitigation of human error. International Journal of Human-Computer Studies, 63(1-2), 2005. • U. Piazzalunga, P. Salveneschi, and P. Confetti. The usability of security devices. In L. F. Cranor and S. Garfinkel, editors, Security and Usability: Designing Secure Systems that People Can Use, p 221-241. O’Reilly, 2005. • T. Whalen, D. Smetters, and E. F. Churchill. User experiences with sharing and access control. In CHI ’06 extended abstracts on Human factors in computing systems, p 1517-1522, 2006. • F. Zhu, M. W. Mutka, and L. M. Ni. The master key: A private authentication approach for pervasive computing environments. In 4th IEEE Interantional Conference on Pervasive Computering and Communications, p 212-221, 2006.

50. Grey accesses per week Number of Accesses Week