Why Johnny Can’t Encrypt A Usability Evaluation of GPG 5.0

1. Why Johnny Can’t EncryptA Usability Evaluation of GPG 5.0 Presented by Yin Shi

2. Overview • Introduction • Understanding the Problem • Cognitive Walkthrough • User Test • Conclusion

3. Introduction • Effective security requires a different usability standard • Security mechanisms are effective only when used correctly • Matt Bishop claimed that configuration errors are the cause of more than 90% of all computer security failures • Making security usable will require the development of domain-specific user interface design principles and techniques • Choose PGP 5.0 for our case study • Designed by general consumer software standards • “Significantly improved graphical user interface makes complex mathematical cryptograph accessible for novice computer users.”

4. Understanding the Problem • Defining Usability for Security • Definition: Security software is usable if the people who are expected to use it: • Are reliably made aware of the security tasks they need to perform • Are able to figure out how to successfully perform those tasks • Don’t make dangerous errors • Are sufficiently comfortable with the interface to continue using it

5. Understanding the Problem • Problematic Properties of Security • Five inherent properties of security • The unmotivated user property • The abstraction property • The lack of feedback property • The barn door property • The weakest link Property • A Usability Standard for PGP • Need for privacy and authentication • What needs to be done • How to do it and avoid dangerous errors

6. Evaluation Methods • Two Methods • An informal cognitive walkthrough • A user test performed in a laboratory

7. Cognitive Walkthrough • Visual Metaphors (keys) • PGP’s user interface relies on graphical depictions of keys and locks • Improvements • An extension of the metaphor to distinguish public keys for encryption and private keys for decryption • Different icons for public and private keys

8. Cognitive Walkthrough • Visual Metaphors (signatures) • The icon of the blue quill pen is used to indicate signing is problematic • Quill pen icon will not help user understand they need to use their private keys to generate signatures • Improvements • Keep quill pen to represent signing, but modify it to show a private key as the nib of the pen • Use some entirely different icon for signatures

9. Cognitive Walkthrough • Different Key Types • Originally, PGP used the RSA algorithm for encryption and signing • PGP 5.0 uses the Diffie-Hellman/DSS algorithm • PGP 5.0 can handle RSA keys, but other version PGP can’t handle DSS keys • Lack of forward compatibility • Recipients with RSA keys can’t decrypt it • Recipients with RSA keys can’t verify signatures • PGP 5.0 alerts its users to this compatibility issues in two ways

10. Cognitive Walkthrough • Different Key Types • Uses different icons to depict the different key types • When user attempt to encrypt documents using mixed key types, a warning message is showed • Improvement • Double-clicking on a key pops up a Key properties window

11. Cognitive Walkthrough • Metaphor of choosing people • Human icons obscure the key type information • Better to display multiple keys that person owns

12. Cognitive Walkthrough • Key Server • Are publicly accessible databases • PGP offers three key server operations under the Keys pull-down menu

13. Cognitive Walkthrough • Problems with the presentation of the Key Server • Users may not realize that it exists • No representation of it in the top level of PGPkeys display • PGPkeys keeps no records of key server access • PGP’s key revocation operation does not send the resulting revocation certificate to the key server

14. Cognitive Walkthrough • Key Management Policy • Two ratings for each public key • Validity – how sure the user is that the key is safe to encrypt with • Trust – how much faith the user has in the key • May not realize PGP can automatically sets the validity rating of a key based on whether it has been signed by a certain number of sufficiently trusted keys.

15. Cognitive Walkthrough • Irreversible Actions • Accidentally deleting the private key • Accidentally publicizing a key • Accidentally revoking a key • Forgetting the passphrase • Failing to back up the key rings • Consistency • encoding • Too Much Information • PGPkeys application presents the user with too much information to make sense of • Owner’s name, validity, trust level, creation date, and size • Nothing to help the user figure out which parts of the display are the most important to pay attention to

16. User Test • Test Design • Initial task is to send the secret message to the team members in a signed and encrypted email • Main steps • Generate a key pair, get the public keys • Make their own public key available to team members • Type the secret message into an emails • Sign the email using private key, encrypt the email using the team member’s public keys

17. User Test • One of the member had an RSA key • Participant would encounter mixed key types warning message • Each of the five campaign members was represented by a dummy email account and a key pair: • These were accessible to the test monitor through a network laptop • The test monitor could send email to the participant from the appropriate dummy account

18. User Test (Results) • Avoiding dangerous errors • Three of them accidentally emailed the secret without encryption • One forgot her passphrase • Figuring out how to encrypt with any key • One couldn’t figure out how to encrypt at all • A reconfiguration of PGP may required • Another one kept sending unencrypted test messages, and finally succeeded after being prompted to use the PGP plug in buttons

19. User Test (Results) • Figuring out the correct key to encrypt with • 11 participants figured out how to encrypt, but failed to understand the public key model • Another one so completely misunderstood the model that he generated key pairs for each team member rather than for himself • Decrypting an email message • Five participants received encrypted email • One can’t figure how to decrypt it • Two took a very hard time to figure it out • Other two were able to decrypt without any problem

20. User Test (Results) • Publishing the public key • Ten could make their public key available to the team members • Two never addressed key distribution • Those ten, five sent their keys to key server • Three emailed to the team members • Other two did both • Getting other people’s public keys • Eight successfully got the team members’ public keys • The others either never seemed aware they need other people’s public key, or they did know how to get it

21. User Test (Results) • Handing the mixed key types problem • Only four managed to send encrypted email correctly • One didn’t have mixed key types problem • The other three received a reply email for complaining that they couldn’t decrypt email • Signing an email message • Verifying a signature on an email message • Creating a backup revocation certificate • Only three participants managed to successfully send encrypted email and decrypt a reply • In response to direct prompting for backup • One didn’t send the key pair to the key server • One sent email to the campaign manager • One simply ignored the prompt

22. User Test (Results) • Deciding whether to trust keys from the key server • Of the eight participants, only three expressed some concern over if they should trust the keys • None of the three made use of the validity and trust labeling provided by PGPKeys

23. Conclusion/Questions • PGP 5.0’s user interface does not come even reasonably close to achieving our usability standard • It does not make public key encryption of electronic mail manageable for average computer users • Public work on usability evaluation in a security context would be extremely valuable • We expect to find better design strategies