Cognitive walkthroughs and heuristic evaluation
1 / 36

Cognitive Walkthroughs and heuristic evaluation - PowerPoint PPT Presentation

  • Uploaded on

Cognitive Walkthroughs and heuristic evaluation. Evaluation - definitions. to assess the extent of the system’s functionality (i.e. does it map onto the user’s task requirements) to assess the effect (not affect, generally) of UI on the user to identify any specific problems with the system.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Cognitive Walkthroughs and heuristic evaluation ' - hilary-shannon

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Evaluation definitions
Evaluation - definitions

  • to assess the extent of the system’s functionality (i.e. does it map onto the user’s task requirements)

  • to assess the effect (not affect, generally) of UI on the user

  • to identify any specific problems with the system

Evaluation is
Evaluation is ...

  • Essential

  • Time consuming

  • Often difficult to organise

    • e.g. the availability of subjects / participants

  • But there is an easy(ish) alternative to working with end-users

Cognitive walkthroughs
Cognitive walkthroughs

  • Structured approach to usability analysis

  • Intended to supplement NOT replace empirical approaches (empirical = with users in this instance)

  • A number of different versions exist...

    • ‘Hand simulation of the cognitive activities of a user’

      Polson et al, 1992

Performing a walkthrough
Performing a walkthrough

  • Create a usage scenario

  • Identify a task within the scenario

  • Perform the walkthrough

  • Record the problems found

To begin
To begin...

  • Scenario:

    • e.g. someone writing a letter

  • Task:

    • To copy and paste some text

  • sub-tasks

    • Copy text into clipboard

    • Select point to enter (pasted) text

    • Issue paste command

Examining the sub goals
Examining the sub-goals

  • Will the user try to achieve the right goal?

  • What knowledge is needed to achieve the right subgoal? Will the user have this knowledge?

Satisfying goals
Satisfying goals

  • Will the user notice that the correct action is available?

  • Will the user associate the correct action with the sub-goal they are trying to achieve?


  • Will the user perceive the feedback?

  • Will the user understand the feedback?

  • Will the user see that progress is being made towards the solution of their task?

Tasks for copy pasting text1
Tasks for copy & pasting text

1. Apply Q1& 2 to sub-goal B

2. Apply Q1 & 2 to C

3. Apply Q3-7

Recording problems
Recording problems

  • Any questions answered to the negative indicate potential usability problems

  • Describe the problem in as much detail as possible

    • When the problem occurred

    • Why the problem occurred

    • The consequences of the problem

Now your turn
Now your turn...

  • Using your task analysis of requesting a loan conduct a cognitive walkthrough

  • Report the problems

    • consider how they might be fixed

Heuristic evaluation
heuristic evaluation

‘heuristic’ - ‘used of problem solving techniques that proceed by trial and error’ (related to Greek ‘eureka’)

Longman Concise English Dictionary, 1985

a method of usability evaluation where an analyst finds usability problems by checking the user interface against a set of supplied heuristics or principles

Lavery, Cockton and Atkinson, 1996

Who should do heuristic evaluation
who should do heuristic evaluation?

  • use more than one evaluator

  • ideally should not be the designer

  • ideally should be a usability specialist

    • technical authors also useful

  • each carries out independent inspection, then aggregate findings

  • evaluators may need help

    • unless ‘walk-up-and-use’ application

    • could provide typical usage scenario

Evaluator performance
evaluator performance

Nielsen 1992:

  • same interface evaluated by 3 groups

  • novice (knowledge about computers only)

  • usability experts

  • usability and specialist domain experts

How to do heuristic evaluation
how to do heuristic evaluation

  • go through interface, compare against recognised usability principles

    • first pass flow of interaction and general scope

    • subsequently focus on specific elements

  • typically 1 - 2 hours in total

  • output: list of usability problems cross-referenced to usability guidelines

Usability metrics





subjective satisfaction

usability metrics

But how to set the target level
but how to set the target level...

  • skill and intuition...

  • better than last version

  • better than the competition

  • client targets

  • set a range of levels

    • unacceptable

    • minimum

    • target

    • ideal


remember materials for next week

Usability testing planning
usability testing - planning

  • draw up a test plan

    • see separate handout ‘Checklist for usability test plans’

    • informal use by the test team

    • formal use for QA procedures

  • consider whether to use video/audio recording

Usability testing users
usability testing - users

  • test users should represent target users

    • remember sales staff as a special user group

    • may need to give basic training

  • getting hold of users

    • internal users should be easy

    • customers from user groups may help

    • paid volunteers : students, classified ads..

      • take account of older users if relevant

Pay off ratio for user testing after nielsen 1993
pay-off ratio for user testing(after Nielsen, 1993)

Usability testing designing test tasks
usability testing - designing test tasks

  • representative & provide reasonable coverage

  • do-able but not trivial

  • consider relating to a larger scenario

  • provide a written task description

  • present in increasing level of difficulty

  • decide whether to use verbal protocols

Relative effectiveness
relative effectiveness

  • Karat el (1992)

  • expert individual & group walkthroughs

    • used guidelines and tasks

  • usability testing

    • users identified and described problems

  • testing identified most problems

    • including some severe ones missed by experts

Effectiveness continued
effectiveness continued

  • walkthroughs useful when resources linited, or for early design

  • team walkthroughs better than individuals

  • techniques are complementary

  • cost effectiveness similar

  • also formal experimental trials

Usability testing procedure
usability testing - procedure

  • preparation

    • remember to switch off screen-savers, email, etc.

  • introduction

  • testing

  • debriefing

    • questionnaires if used

    • also ask about the testing process

  • write up quickly

  • Usability metrics1


    time to reach specified level of proficiency, e.g. complete a specified, representative task

    note that learning is a continuum


    test users on commands after trial session


    number of errors in completing specified task

    subjective satisfaction

    rating scales

    physiological measures


    times for experts to complete specified task(s)

    frequency of ‘non-productive’ actions

    ratio of used to unused commands

    usability metrics