1 / 36

Cognitive Walkthroughs and heuristic evaluation

Cognitive Walkthroughs and heuristic evaluation. Evaluation - definitions. to assess the extent of the system’s functionality (i.e. does it map onto the user’s task requirements) to assess the effect (not affect, generally) of UI on the user to identify any specific problems with the system.

Download Presentation

Cognitive Walkthroughs and heuristic evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Cognitive Walkthroughs and heuristic evaluation

  2. Evaluation - definitions • to assess the extent of the system’s functionality (i.e. does it map onto the user’s task requirements) • to assess the effect (not affect, generally) of UI on the user • to identify any specific problems with the system

  3. Evaluation is ... • Essential • Time consuming • Often difficult to organise • e.g. the availability of subjects / participants • But there is an easy(ish) alternative to working with end-users

  4. Cognitive walkthroughs • Structured approach to usability analysis • Intended to supplement NOT replace empirical approaches (empirical = with users in this instance) • A number of different versions exist... • ‘Hand simulation of the cognitive activities of a user’ Polson et al, 1992

  5. Performing a walkthrough • Create a usage scenario • Identify a task within the scenario • Perform the walkthrough • Record the problems found

  6. To begin... • Scenario: • e.g. someone writing a letter • Task: • To copy and paste some text • sub-tasks • Copy text into clipboard • Select point to enter (pasted) text • Issue paste command

  7. Tasks for copy & pasting text Note the order!

  8. Examining the sub-goals • Will the user try to achieve the right goal? • What knowledge is needed to achieve the right subgoal? Will the user have this knowledge?

  9. Satisfying goals • Will the user notice that the correct action is available? • Will the user associate the correct action with the sub-goal they are trying to achieve?

  10. Feedback • Will the user perceive the feedback? • Will the user understand the feedback? • Will the user see that progress is being made towards the solution of their task?

  11. Tasks for copy & pasting text 1. Apply Q1& 2 to sub-goal B 2. Apply Q1 & 2 to C 3. Apply Q3-7

  12. Recording problems • Any questions answered to the negative indicate potential usability problems • Describe the problem in as much detail as possible • When the problem occurred • Why the problem occurred • The consequences of the problem

  13. Example - setting column widths

  14. Now your turn... • Using your task analysis of requesting a loan conduct a cognitive walkthrough • Report the problems • consider how they might be fixed

  15. heuristic evaluation ‘heuristic’ - ‘used of problem solving techniques that proceed by trial and error’ (related to Greek ‘eureka’) Longman Concise English Dictionary, 1985 a method of usability evaluation where an analyst finds usability problems by checking the user interface against a set of supplied heuristics or principles Lavery, Cockton and Atkinson, 1996

  16. who should do heuristic evaluation? • use more than one evaluator • ideally should not be the designer • ideally should be a usability specialist • technical authors also useful • each carries out independent inspection, then aggregate findings • evaluators may need help • unless ‘walk-up-and-use’ application • could provide typical usage scenario

  17. effectiveness of increasing number of evaluators (after Nielsen, 1993)

  18. evaluator performance Nielsen 1992: • same interface evaluated by 3 groups • novice (knowledge about computers only) • usability experts • usability and specialist domain experts

  19. how to do heuristic evaluation • go through interface, compare against recognised usability principles • first pass flow of interaction and general scope • subsequently focus on specific elements • typically 1 - 2 hours in total • output: list of usability problems cross-referenced to usability guidelines

  20. learnability memorability errors efficiency subjective satisfaction usability metrics

  21. but how to set the target level... • skill and intuition... • better than last version • better than the competition • client targets • set a range of levels • unacceptable • minimum • target • ideal

  22. a usability profile

  23. STOP remember materials for next week

  24. usability testing - planning • draw up a test plan • see separate handout ‘Checklist for usability test plans’ • informal use by the test team • formal use for QA procedures • consider whether to use video/audio recording

  25. usability testing - users • test users should represent target users • remember sales staff as a special user group • may need to give basic training • getting hold of users • internal users should be easy • customers from user groups may help • paid volunteers : students, classified ads.. • take account of older users if relevant

  26. pay-off ratio for user testing(after Nielsen, 1993)

  27. usability testing - designing test tasks • representative & provide reasonable coverage • do-able but not trivial • consider relating to a larger scenario • provide a written task description • present in increasing level of difficulty • decide whether to use verbal protocols

  28. relative effectiveness • Karat el (1992) • expert individual & group walkthroughs • used guidelines and tasks • usability testing • users identified and described problems • testing identified most problems • including some severe ones missed by experts

  29. effectiveness continued • walkthroughs useful when resources linited, or for early design • team walkthroughs better than individuals • techniques are complementary • cost effectiveness similar • also formal experimental trials

  30. usability testing - procedure • preparation • remember to switch off screen-savers, email, etc. • introduction • testing • debriefing • questionnaires if used • also ask about the testing process • write up quickly

  31. learnability time to reach specified level of proficiency, e.g. complete a specified, representative task note that learning is a continuum memorability test users on commands after trial session errors number of errors in completing specified task subjective satisfaction rating scales physiological measures efficiency times for experts to complete specified task(s) frequency of ‘non-productive’ actions ratio of used to unused commands usability metrics

More Related