1 / 42

Heuristic Evaluation

Heuristic Evaluation. ☃. HCC 729, 2/13/14. We’ll follow up next time. Inspirations, reading feedback Your HTAs and personas. How to conduct a Heuristic Evaluation. Read this: http :// www.nngroup.com /articles/how-to-conduct-a-heuristic-evaluation/.

temira
Download Presentation

Heuristic Evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Heuristic Evaluation ☃ HCC 729, 2/13/14

  2. We’ll follow up next time • Inspirations, reading feedback • Your HTAs and personas

  3. How to conduct a Heuristic Evaluation Read this: http://www.nngroup.com/articles/how-to-conduct-a-heuristic-evaluation/ Adapted from slides by Karen Tang and Ryan Baker

  4. What is an evaluation? • Gather data about the usability of a product or design by a particular group of users for a particular activity or task within a particular environment or context • Evaluation goals: • Assess extent of system’s functionality • Assess effect of interface on user • Identify specific problems with system

  5. HE vs. user testing • When we can, we want to test with real users • HE is a “discount” usability technique • When it’s useful: • When real users are unavailable • Very early in the design • As a sanity check (but not a replacement for user testing)

  6. Why HE is great • Cheap • Doesn’t “spend” users • Fast • 1-2 days (instead of 1 week) • Good • Proven effective: the more careful you are, the better it gets • Easy to use • Relatively easy to learn, can be taught

  7. Heuristic Evaluation • A type of discount usability testing • A rational method – an expert applies “heuristics” • mentally apply a theory or rule to the design and see if that theory/rule’s advice is being followed • Key Idea: Multiple expert evaluators independently apply a set of heuristics to an interface, produce Usability Action Reports (UARs), combine & prioritize their findings

  8. What can you evaluate with a HE? • Any interface that has been “developed” • Pre-existing webpage • Sketch of a future interface (can be fully implemented or only exist as a sketch) • This method can be applied on your own interface, or a competitor’s • You will evaluate the interface according to a standard set of 10 heuristics

  9. How many evaluators are needed? • Nielsen recommends at least 3, but go for 5!

  10. Who should do the HE? • Anyone who knows the appropriate heuristics can do a HE • But, heuristic evaluation experts find almost twice as many problems as novices • Heuristic evaluation experts who are also domain experts find almost three times as many problems as novices

  11. Phases of Heuristic Evaluation 0) Pre-evaluation training (optional) Give evaluators needed domain knowledge & information on the scenario • Evaluate the interfaceto find usability problems • Record the problem • Aggregate problems • Assign severity rating 5) Find a solution complexity rating

  12. #1: evaluate the interface

  13. Which heuristics to use? • Many possible heuristic sets • Some standard sets (e.g. Nielsen’s usability heuristics) • You might create your own heuristics, e.g. for specific applications • We’ll focus on Nielsen’s, which cover a range of general usability issues

  14. Find which heuristic is violated • Simple & Natural Dialog • Speak User’s Language • Minimize User’s Memory Load • Consistency • Feedback • Clearly Marked Exits • Shortcuts • Good Error Messages • Prevent Errors • Help & Documentation Nielsen’s 10 Heuristics http://www.nngroup.com/articles/ten-usability-heuristics/

  15. Examples of applying the heuristics • http://www.slideshare.net/sacsprasath/ten-usability-heuristics-with-example

  16. #2: record the problem

  17. Record the problem • Each evaluator writes a Usability Action Report (UAR) describing each usability problem they encounter • HEs are typically used to report problems • However, UARs can be used to report both the goodand bad qualities of an interface in other usability evaluations… • I have posted a template UAR on the blog with the assignment

  18. Sample UAR • EVALUATOR: XXXXX • ID NUMBER: XXX • NAME: Descriptive name for the problem • EVIDENCE: Describe the violation, and why you wrote this report. what heuristic was violated, • EXPLANATION: Your interpretation: and why. • Severity: Write up at the end of the evaluation • Fixability: Write up at the end of the evaluation • Possible Fix: Write up at the end of the evaluation

  19. Keep looking for problems! • Usually takes a few hours • A shorter time may not find important problems • A longer time will exhaust the evaluator, and they may become less productive • For very large interfaces, it is good to break heuristic evaluation into several sessions

  20. What about multiple problems? • This happens a lot, record them separately. • This is not busywork…. • It may be possible to fix some of the problems, but not all of them • The problems might not always be linked to each other – one may show up in other situations too

  21. You are not done yet… • You still need to address the bottom half of the UAR: • Severity • Solution Complexity • Possible Fix • You may want to take a break before finishing these UARs…

  22. #3 aggregate the problems

  23. Aggregate Problems • Wait until all UARs are in • You are aggregating across all evaluators • Aggregating usability problems: • Combine problems by consensus • Gain a sense of relative importance after you’ve seen a few problems • At this point, decide which entries are and aren’t problems (but keep original version of report somewhere)

  24. #4: assign each problem a severity rating

  25. Assign Severity Rating to UARs • Severity Ratings help project leads determine what problems should be given more developer time • Not all problems can be fixed • Some problems will have more severe consequences • Each evaluator should assign severity separately

  26. Assign Severity Rating to UARs Based on a combination of: • Frequency • How common or rare is the problem? • Impact • How easy is it to overcome the problem? • How disastrous might the problem be? • Persistence • How repeatedly will users experience the problem? • Are workarounds learnable?

  27. Assign an Overall Severity Rating • It is helpful to developers in allocating resources to have one severity rating for the problem. • Therefore, evaluators need to combine their opinion of a problem’s Frequency, Impact, & Persistence ratings into one Severity evaluation

  28. Nielsen’s Severity Ratings • Usability Blemish. Mild annoyance or cosmetic problem. Easily avoidable. • Minor usability problem. Annoying ,misleading, unclear, confusing. Can be avoided or easily learned. May occur only once. • Major usability problem. Prevents users from completing tasks. Highly confusing or unclear. Difficult to avoid. Likely to occur more than once. • Critical usability problem. Users won’t be able to accomplish their goals, and may quit using system.

  29. False positives • There’s no virtue in finding 6,233 problems, if very few of them actually cause problems for a user • Every problem reported in a heuristic evaluation takes time for the developers to consider • Some interface aspects that seem like problems at first might not be problems at all

  30. #5 Assign each solution a complexity rating

  31. 5: Solution Complexity Rating • Some problems take more time to fix than others, so it’s important to allocate developers’ time well • Ideally this could be made by either a developer, or someone who is familiar with development in the target platform

  32. Solution Complexity Rating • Trivial to fix. Textual changes and cosmetic changes. Minor code tweaking. • Easy to fix. Minimal redesign and straightforward code changes. Solution known and understood. • Difficult to fix. Redesign and re-engineering required. Significant code changes. Solution identifiable but details not fully understood. • Nearly impossible to fix. Requires massive re- engineering or use of new technology. Solution not known or understood at all.

  33. Record Possible Fixes • While evaluating solution complexity evaluator may have thought about how the problem could be fixed • Record these possible fixes as suggestions to developers • Don’t focus on feasibility of solutions (that is their job) • Your suggestions may be thought-provoking

  34. Phases of Heuristic Evaluation 0) Pre-evaluation training (optional) Give evaluators needed domain knowledge & information on the scenario • Evaluate the interfaceto find usability problems • Record the problem • Aggregate problems • Severity rating 5) Solution complexity rating

  35. Why HE? • They find a reasonably large set of problems • They are one of the easiest, quickest, and cheapest methods available

  36. HE vs. User Testing • User tests are more effective at revealing when a system’s manifest model or metaphor is confusing • User tests are less effective at finding obscure problems • User tests are also much more expensive • Advice: use HE first, to find the obvious problems, then user test.

  37. For next week Assignment Readings

  38. In-class assignment Perform HE on UMBC class search with PeopleSoft Use template form from the blog

  39. HW: Perform HE on your sites • Go through the 5 stages of HE for your website • If on a team, each go through HE individually, then combine later • Turn in 1 completed form for each incident • Come up with at least 8 UARs for each webpage • Aggregate, finish filling out the template! • Everyone writes 100-200 words describing what they learned

  40. HW Extra credit • Perform a HE on another student’s web site (same as previous slide)

  41. Reading • Required • Usability Engineering Chapter 6 (focus on 6.1-6.6 and 6.8) • Optional • Think aloud method

More Related