Topic 6 usability evaluation of ia applications and mechanisms
This presentation is the property of its rightful owner.
Sponsored Links
1 / 34

Topic 6: Usability Evaluation of IA Applications and Mechanisms PowerPoint PPT Presentation


  • 101 Views
  • Uploaded on
  • Presentation posted in: General

Topic 6: Usability Evaluation of IA Applications and Mechanisms. Azene Zenebe, Ph.D. Bin Mai, Ph.D. Presentation Outline. Introduction Usability of IA applications and mechanisms - Reviewed Usability Evaluation: What, When and Why Usability Specification for Evaluation

Download Presentation

Topic 6: Usability Evaluation of IA Applications and Mechanisms

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Topic 6 usability evaluation of ia applications and mechanisms

Topic 6:Usability Evaluation of IA Applications and Mechanisms

Azene Zenebe, Ph.D.

Bin Mai, Ph.D.


Presentation outline

Presentation Outline

  • Introduction

  • Usability of IA applications and mechanisms - Reviewed

  • Usability Evaluation: What, When and Why

  • Usability Specification for Evaluation

  • Usability Evaluation Methods

    • Analytical methods

    • Empirical methods

  • Case Study

  • Summary


Learning objectives and outcomes

Learning Objectives and Outcomes

  • After completing this module, you should be able to:

    • Describe the factors that affect usability of security systems

    • Describe the importance of evaluation of usability security systems

    • Prepare usability specification for evaluation


Learning objectives and outcomes continued

Learning Objectives and Outcomes (Continued)

  • perform usability evaluation or testing of a security system using an analytical method such as expert inspection

  • Perform usability evaluation or testing of a security system using an empirical method such as a field study or lab testing

  • Report results of usability evaluation as well as describe how the results can be used to make improvement


Introduction

Introduction

  • Usability of IA application and mechanism - Reviewed

    • Usability refers to the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of user - ISO 9241-11


Multi dimensionality of usability

Multi-dimensionality of Usability

  • Ease of learning

  • Efficiency of use

  • Memorability

  • Effectiveness

  • Error frequency and severity

  • Subjective satisfaction


Usable security systems

“Usable” Security Systems

  • can easily and quickly learn a security system that they have never seen to accomplish basic tasks

  • can remember enough to use them later without major cost

  • are able to effectively perform and successfully complete security tasks supported by them

  • cannot make sever and frequent errors

  • are satisfied with the interface and functions of the systems


Framework for studying usability of security systems

Framework for studying usability of security systems

  • four principal components in a human-machine system

    • TOOL

    • USER

    • TASK

    • ENVIRONMENT


Four groups of people involved in security systems

Four groups of people involved in Security systems

  • Definers provide the policies, guidelines, and standards

  • Builders are the real techies, who create and install security solutions

  • Administrators operate and administer the security tools

  • End-users include home users and employees who are novice to CISS


Usability evaluation what when and why

Usability Evaluation: What, When and Why

  • Usability evaluation: whether a security system is usable for the users

  • Goal of usability evaluation: identify and correct flaws associated with ease of use of a security system

  • Performed during design and testing (or post-implementation) phases

  • Evaluation is iterative – an ongoing process


Usability specification for evaluation

Usability Specification for Evaluation

  • Usability specifications are statements of required usability characteristics that are precise and testable

  • Task analysis provides a more precise specification of what users are expected to do in order to accomplish a task successfully


A sample usability specification authenticity of a website

A sample usability specification - authenticity of a website


Usability evaluation methods

Usability Evaluation Methods

  • Analytical Methods - conduct analysis of a system’s features with the respect to their impacts for use

  • Empirical Methods – collect and use data from a system’s users. It is also referred as user-based testing


Analytical methods

Analytical Methods

  • Expert’s knowledge stated as a heuristic rules

    • Ten Usability Heuristics by Jakob Nielsen

    • Shneiderman’s 8 Golden Rules of Interface Design


Empirical methods

Empirical Methods

  • What usability evaluators want to know is what happens when users use the system

  • Different techniques are

    • Field studies

    • Usability Testing in a laboratory

    • Controlled Experiments


Quick quiz

Quick Quiz

  • What are the main advantages and disadvantages for analytical methods and empirical methods?

  • Come up with two sample scenarios in IA field where you think analytical methods should be preferred, and two other scenarios where you think empirical methods should be preferred


Steps for usability testing

Steps for usability testing

  • Identify and profile the representative users

  • Select the setting

  • Decide what tasks users should perform

  • Decide how and what types of data to collect

  • Perform necessary activities before test session

  • Perform necessary activities during test session

  • Perform necessary activities after test session


Usability testing in a laboratory

Usability Testing in a Laboratory

  • Validity concerns are associated with the following questions for lab based testing

    • Is the prototype system used in the testing missing any important features

    • Are test participants really the kind of users who will use the system

    • Will actual users do tasks like these participants

    • Will actual users be more distracted in their offices


Using the results of usability testing

Using the Results of Usability Testing

  • Results need to be looked at and actionable information regarding usability problems and issues should be made for design teams

  • Provide recommendations to address the identified problems


Automated usability testing tools

Automated Usability Testing Tools

  • A List of 24 Web Site Usability Testing Tools

    • http://www.usefulusability.com/24-usability-testing-tools/

  • UMD list of usability testing tools

    • http://otal.umd.edu/guse/testing.html#sect3a

  • Jay Forbes’ presentation about usability testing tools

    • http://www.gslis.utexas.edu/~l385t6rb/auto_tools.pdf


Quick quiz1

Quick Quiz

  • Suppose you are testing the usability of an IDS your company decided to implement.

    • What will be the setting of the testing?

    • Who will be the representative users?

    • What type of data should you collect? Justify your answers.


Quick quiz2

Quick Quiz

  • Among IT managers, business managers, usability specialists, or general public, who do you think are the main users for automated usability testing tools? Why?

  •  What aspects of a usability study do you believe can never be automated? Why?


Topic 6 usability evaluation of ia applications and mechanisms

Case

  • Perspectives: Usability Evaluation

    • Perspectives is a new approach to help clients securely identify Internet servers in order to avoid "man-in-the-middle" attacks

      • works with Firefox 3 extension

      • Demo

  • Mission of the Perspectives

    • detect whether a self-signed certificate is valid

    • detect the fake security certificate attack and will warn you


Usability evaluation design

Usability Evaluation Design

  • User Population

    • Potential Users: Novice, Intermediate and Expert in Security and IT

    • Targeted Users: Subset of the Potential Users

  • Context of Uses

    • Using the Internet

    • Home, free WiFi sites, and/or work

    • Quite or Not Quite environment

  • Tasks: Banking, Shopping, etc.


Usability evaluation design1

Usability Evaluation Design

  • Perspectives: evaluating the authenticity of a public key based on accompanying signatures and making use of a Browser’s built-in mechanisms for such evaluation

  • Requirements gathering

    • Develop usability specification

  • Usability Evaluation

    • Using Inspection

    • Using Empirical


Summary

Summary

  • From this module, reader should take away the following:

    • Usability is a combination of factors

    • Usability requires that users understands the organization policy and rules

    • There exist frameworks that guide the usability evaluation

    • For different stakeholders, the goals of usability differ


Summary continued

Summary (continued)

  • From this module, reader should take away the following:

    • Usability specification is required for usability evaluation

    • There are two categories of usability evaluation methods

    • There existing some tools that automate usability testing


Discussion topics

Discussion Topics

  • What are the advantages and disadvantages of Inspection method?

  • What are the advantages and disadvantages of Empirical method?

  • Compare and contrast the different methods of data collection. Describe the advantages and disadvantages of these methods.


Discussion topics1

Discussion Topics

  • How useful are these Heuristics for security systems? Which of the two is more relevant to security systems? Are these methods security systems dependent?

  • Is there a heuristics for security system interface design? Is there a methodology?


Discussion topics2

Discussion Topics

  • Describe and discuss scenarios where a system’s usability is important to one type of users, while not so important to another type

  • What are your opinions regarding the ideas that, as described by Jay Forbe, “automated usability testing is too good to be true”?


Project ideas

Project Ideas

  • Suppose your friend Joe opened an E-bay store online to sell his comic book collections, what data do you collect to evaluate his website’s usability?

  • Suppose a university Registrar Office hires you to evaluate the usability of its online registration system. What data would you collect?


Project ideas1

Project Ideas

  • Prepare a sample usability specification built to track usability of a scenario for setting a firewall in Windows XP.

  • Develop a usability evaluation desing to track usability of an IDS (Intrusion Detection System)

  • Design a usability evaluation study for the latest release of PGP.


References

References

1.Braz, C. and Robert, J.-M. Security and usability: the case of the user authentication methods. In Proceedings of the 18th International Conferenceof the Association Francophone d'Interaction Homme-Machine ACM, Montreal, Canada 2006 199-203

2.Garfinkel, S.L. Design Principles and Patterns for Computer Systems That Are Simultaneously Secure and Usable Department of Electrical Engineering and Computer Science, MASSACHUSETTS INSTITUTE OF TECHNOLOGY, Boston, 2005, 470.

3.Hoonakker, P., Bornoe, N. and Carayon, P., Password Authentication from a Human Factors Perspective: Results of a Survey among End-Users. In 3rd Annual Meeting of the Human Factors and Ergonomics Society, (San Antonio, TX, 2009).

4.Josang, A., Alfayyadh, B., Grandison, T., Alzomai, M. and Mcnamara, J., Security usability principles for vulnerability analysis and risk assessment. in Twenty-Third Annual In Computer Security Applications Conference, (Miami Beach, Florida, 2007), 269-278.

5.Lazar, J. Web Usability: A User-Centered Design Approach. Pearson, Addison Wesley, Boston, 2006.

6.Nielsen, J. Usability Engineering. Morgan Kaufmann, San Francisco, 1994.


References1

References

7. Rosson, M.B. and Carroll, J.M. Usability Engineering: Scenario-based development of human-computer interaction. Morgan Kaufmann, San Francisco, 2002.

8.Shackel, B. Usability - Context, Framework, Definition, Design and Evaluation. in Richardson, S. ed. Human Factors for Informatics Usability, Cambridge University Press, Cambridge, 1991.

9.Shneiderman, B. and Plaisant, C. Designing the User Interface. Addison-Wesley, Boston, 2005.

10.Weir, C.S., Douglasa, G., Carruthers, M. and Jacka, M. User perceptions of security, convenience and usability for ebanking authentication tokens. Computer & Security, 28 (1-2). 47-62.

11.Whitman, M.E. and Mattord, H.J. Management of Information Security. Course Technology, Thomson Learning, Inc., Canada, 2004.

12.Whitten, A. and Tygar, D., Why Johnny can't encrypt? In USENIX, (1999).


  • Login