Cognitive walkthroughs and heuristic evaluation
This presentation is the property of its rightful owner.
Sponsored Links
1 / 36

Cognitive Walkthroughs and heuristic evaluation PowerPoint PPT Presentation


  • 78 Views
  • Uploaded on
  • Presentation posted in: General

Cognitive Walkthroughs and heuristic evaluation. Evaluation - definitions. to assess the extent of the system’s functionality (i.e. does it map onto the user’s task requirements) to assess the effect (not affect, generally) of UI on the user to identify any specific problems with the system.

Download Presentation

Cognitive Walkthroughs and heuristic evaluation

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Cognitive walkthroughs and heuristic evaluation

Cognitive Walkthroughs and heuristic evaluation


Evaluation definitions

Evaluation - definitions

  • to assess the extent of the system’s functionality (i.e. does it map onto the user’s task requirements)

  • to assess the effect (not affect, generally) of UI on the user

  • to identify any specific problems with the system


Evaluation is

Evaluation is ...

  • Essential

  • Time consuming

  • Often difficult to organise

    • e.g. the availability of subjects / participants

  • But there is an easy(ish) alternative to working with end-users


Cognitive walkthroughs

Cognitive walkthroughs

  • Structured approach to usability analysis

  • Intended to supplement NOT replace empirical approaches (empirical = with users in this instance)

  • A number of different versions exist...

    • ‘Hand simulation of the cognitive activities of a user’

      Polson et al, 1992


Performing a walkthrough

Performing a walkthrough

  • Create a usage scenario

  • Identify a task within the scenario

  • Perform the walkthrough

  • Record the problems found


To begin

To begin...

  • Scenario:

    • e.g. someone writing a letter

  • Task:

    • To copy and paste some text

  • sub-tasks

    • Copy text into clipboard

    • Select point to enter (pasted) text

    • Issue paste command


Tasks for copy pasting text

Tasks for copy & pasting text

Note the order!


Examining the sub goals

Examining the sub-goals

  • Will the user try to achieve the right goal?

  • What knowledge is needed to achieve the right subgoal? Will the user have this knowledge?


Satisfying goals

Satisfying goals

  • Will the user notice that the correct action is available?

  • Will the user associate the correct action with the sub-goal they are trying to achieve?


Feedback

Feedback

  • Will the user perceive the feedback?

  • Will the user understand the feedback?

  • Will the user see that progress is being made towards the solution of their task?


Tasks for copy pasting text1

Tasks for copy & pasting text

1. Apply Q1& 2 to sub-goal B

2. Apply Q1 & 2 to C

3. Apply Q3-7


Recording problems

Recording problems

  • Any questions answered to the negative indicate potential usability problems

  • Describe the problem in as much detail as possible

    • When the problem occurred

    • Why the problem occurred

    • The consequences of the problem


Example setting column widths

Example - setting column widths


Now your turn

Now your turn...

  • Using your task analysis of requesting a loan conduct a cognitive walkthrough

  • Report the problems

    • consider how they might be fixed


Heuristic evaluation

heuristic evaluation

‘heuristic’ - ‘used of problem solving techniques that proceed by trial and error’ (related to Greek ‘eureka’)

Longman Concise English Dictionary, 1985

a method of usability evaluation where an analyst finds usability problems by checking the user interface against a set of supplied heuristics or principles

Lavery, Cockton and Atkinson, 1996


Who should do heuristic evaluation

who should do heuristic evaluation?

  • use more than one evaluator

  • ideally should not be the designer

  • ideally should be a usability specialist

    • technical authors also useful

  • each carries out independent inspection, then aggregate findings

  • evaluators may need help

    • unless ‘walk-up-and-use’ application

    • could provide typical usage scenario


Effectiveness of increasing number of evaluators after nielsen 1993

effectiveness of increasing number of evaluators (after Nielsen, 1993)


Evaluator performance

evaluator performance

Nielsen 1992:

  • same interface evaluated by 3 groups

  • novice (knowledge about computers only)

  • usability experts

  • usability and specialist domain experts


How to do heuristic evaluation

how to do heuristic evaluation

  • go through interface, compare against recognised usability principles

    • first pass flow of interaction and general scope

    • subsequently focus on specific elements

  • typically 1 - 2 hours in total

  • output: list of usability problems cross-referenced to usability guidelines


Usability metrics

learnability

memorability

errors

efficiency

subjective satisfaction

usability metrics


But how to set the target level

but how to set the target level...

  • skill and intuition...

  • better than last version

  • better than the competition

  • client targets

  • set a range of levels

    • unacceptable

    • minimum

    • target

    • ideal


A usability profile

a usability profile


Cognitive walkthroughs and heuristic evaluation

STOP

remember materials for next week


Usability testing planning

usability testing - planning

  • draw up a test plan

    • see separate handout ‘Checklist for usability test plans’

    • informal use by the test team

    • formal use for QA procedures

  • consider whether to use video/audio recording


Usability testing users

usability testing - users

  • test users should represent target users

    • remember sales staff as a special user group

    • may need to give basic training

  • getting hold of users

    • internal users should be easy

    • customers from user groups may help

    • paid volunteers : students, classified ads..

      • take account of older users if relevant


Pay off ratio for user testing after nielsen 1993

pay-off ratio for user testing(after Nielsen, 1993)


Usability testing designing test tasks

usability testing - designing test tasks

  • representative & provide reasonable coverage

  • do-able but not trivial

  • consider relating to a larger scenario

  • provide a written task description

  • present in increasing level of difficulty

  • decide whether to use verbal protocols


Relative effectiveness

relative effectiveness

  • Karat el (1992)

  • expert individual & group walkthroughs

    • used guidelines and tasks

  • usability testing

    • users identified and described problems

  • testing identified most problems

    • including some severe ones missed by experts


Effectiveness continued

effectiveness continued

  • walkthroughs useful when resources linited, or for early design

  • team walkthroughs better than individuals

  • techniques are complementary

  • cost effectiveness similar

  • also formal experimental trials


Usability testing procedure

usability testing - procedure

  • preparation

    • remember to switch off screen-savers, email, etc.

  • introduction

  • testing

  • debriefing

    • questionnaires if used

    • also ask about the testing process

  • write up quickly


  • Usability metrics1

    learnability

    time to reach specified level of proficiency, e.g. complete a specified, representative task

    note that learning is a continuum

    memorability

    test users on commands after trial session

    errors

    number of errors in completing specified task

    subjective satisfaction

    rating scales

    physiological measures

    efficiency

    times for experts to complete specified task(s)

    frequency of ‘non-productive’ actions

    ratio of used to unused commands

    usability metrics


  • Login