1 / 33

Confused Johnny

Confused Johnny. Scott Ruoti, Nathan Kim, Ben Burgon, Tim van der Horst, Kent Seamons Internet Security Research Lab Computer Science Department Brigham Young University. When Automatic Encryption Leads to Confusion and Mistakes. Confused Johnny. E-mail encryption for the masses

denim
Download Presentation

Confused Johnny

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Confused Johnny Scott Ruoti, Nathan Kim, Ben Burgon, Tim van der Horst, Kent Seamons Internet Security Research Lab Computer Science Department Brigham Young University When Automatic Encryption Leads to Confusion and Mistakes

  2. Confused Johnny • E-mail encryption for the masses • We developed a system maximizing usability • Made everything transparent • Johnny became confused • Designed another system with manual encryption • This helped Johnny gain clarity

  3. Encrypted E-mail • Exists, but largely goes unused • S/MIME, PGP • Tools available • “Why Johnny can't encrypt: A usability evaluation of PGP 5.0” • Whitten and Tygar, 8th USENIX Security Symposium (1999) • Later research confirmed findings • What can be done?

  4. Usability Issues • Users resist change • Users are using webmail • If security is difficult users will forgo it • Key management is confusing • Hierarchical, web-of-trust • Recipient must already have key • Chicken and egg problem • Cryptography is complicated • Unclear which properties are provided • Unclear which properties are needed

  5. Private Webmail (Pwm) • Pronounced “Poem” • Adds end-to-end encryption to existing webmail systems • Gmail, Hotmail, Yahoo! Mail • Runs on all modern browsers • Designed to maximize usability • Provide good-enough security • Improvement for those already sending sensitive e-mail

  6. Users Resist Change • Security overlays • Integrates tightly with existing webmail systems • Users do not need to learn yet-another-system • Tightly integrates with existing systems • Replaces small portions of the interface • Displayed using iFrames • Functionally transparent • Low barrier to adoption • Visually distinctive • Easy to identify

  7. Usability Fixes • Users resist change • Focus on bootstrapping first-time users • Helpful instructions in e-mail • Bookmarklet-based installation • Key management is confusing • Key escrow based on IBE • Simple Authentication for the Web (EBIA) • No user interaction required • Cryptography is complicated • Encryption is automatically handled by Pwm • Users never interact with ciphertext

  8. Pwm: Walkthrough

  9. Pwm: Walkthrough

  10. Pwm: Walkthrough

  11. Pwm: Walkthrough

  12. Pwm: Walkthrough

  13. Pwm User Studies • Two studies • First study measured usability of Pwm • Also evaluated bookmarklets for use during installation • Second study compared Pwm to Voltage Secure Mail Cloud • Voltage Secure Mail Cloud is an existing depot-based secure email system • Pwm was run using a browser extension • Evaluation • Pre- and post-survey questionnaire • Monitored participants actions for unrecognized mistakes • Post-survey interviews

  14. System Usability Scale • Brook (1996) • Ten questions • Alternate negative and positive • Give a single number for usability • Bangor compared scores for hundreds of systems

  15. 76 71 63

  16. SUS Score Comparison

  17. Success? • Results are very promising • Very positive reception • Users indicated they wanted to begin using it • Not without problems • Small number sent e-mail without encryption • Participants were confused about security • Wanted to see more details • Unsure of who could read e-mails

  18. Where to go from here? • Simple solutions was to fix UI issues • One author (Nathan Kim) had a different idea • Manual encryption • Decoupled interface • Mocked up these ideas • Message Protector (MP) • Simple Interface • Direct handling of ciphertext • Implied key management

  19. MP: Walkthrough

  20. MP: Walkthrough

  21. MP: Walkthrough

  22. MP: Walkthrough

  23. First MP User Study • Evaluated MP using SUS • Compared against Encipher.it • Bookmarklet-based encryption system • Works in Gmail and Facebook • Evaluation • Pre- and post-survey questionnaire • Monitored participants actions for unrecognized mistakes • Post-survey interviews • The system usability scale • Evaluated comprehension • Survey included questions about comprehension • How to use the system • Who could read messages

  24. 61 72

  25. Second MP User Study • Surprising usability results • Participants had a positive reaction to seeing ciphertext • Similar SUS score to MP • Ran a second study comparing MP to Pwm • Modeled after the first MP study

  26. 76 74

  27. SUS Score Comparison

  28. Other results • MP improved users comprehension • Clearly understood how to use system • Clearly understood who could read messages • Usability scores nearly identical to Pwm • Participants preferred manual encryption of MP • Participants preferred tight integration of Pwm

  29. Study limitations • MP studies ignore bootstrapping new users • Studies assumed software pre-installed • Bootstrapping is a key component of Pwm’s design • Not fully representative of overall usability • Short-term studies • SUS question unclear • “I think that I would like to use this system frequently.” • Participants ranked low even when enthusiastic about the system • Relevant to security studies

  30. Related Works • “Johnny 2: a user test of key continuity management with s/mime and outlook express.” • Garfinkel and Miller, SOUPS 2005 • Applied automatic key management to e-mail • Allowed great success • “Helping Johnny 2.0 to encrypt his Facebook conversations.” • Fahl, et al., SOUPS 2012 • Encrypted Facebook communication • Explored manual vs. automatic encryption • Invisibility security not trusted by users

  31. Conclusion • Pwm was a success • Participants largely succeeded at using encrypted e-mail • Participants had high praise for Pwm • Succeeding in being easy for new users • Pwm wasn’t perfect • Security was too transparent • Caused users to be confused and make mistakes • Mocked up a system using manual encryption • Users enjoyed manual encryption • Wished it was tightly integrated with the browser • A combination of approaches is needed to solve the problem

  32. Conclusion: Future Work • Manual encryption in Pwm • Don’t automatically send encrypted email • “Encrypt” button which puts ciphertext in compose window • Sidebar • Browser sidebar allowing for manual encryption • Can be used on any site • Fallback for when Pwm has an error • Long-term studies • Larger populations • Real tasks

  33. Questions?

More Related