Some usability engineering methods
Download
1 / 39

Some Usability Engineering Methods - PowerPoint PPT Presentation


  • 104 Views
  • Uploaded on

Some Usability Engineering Methods. Randolph Bias Week 9. Remember. Our approach, within usability engineering of web sites and other user interfaces, is: Empirical Iterative User-Centered Design. The Methods of Usability Engineering.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Some Usability Engineering Methods' - latifah-warren


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Remember
Remember . . .

  • Our approach, within usability engineering of web sites and other user interfaces, is:

  • Empirical

  • Iterative

  • User-Centered Design


The methods of usability engineering
The Methods of Usability Engineering . . .

  • Are employed in order to enable you to bring user data (empiricism) to bear on the emerging product design.

  • You (the usability engineer) become an advocate for the user, in the product development process.


Three methods
Three Methods

  • End-user testing (lab testing)

  • Heuristic Evaluations

  • Usability Walkthroughs

    Today let’s talk about WHY and WHEN we employ one method or another, and HOW to carry them out.


End user testing
End-user Testing

  • Also called “lab testing”

  • Can be done on paper-and-pencil design, prototype, early code, existing product


Eut benefits
EUT - Benefits

  • Gather performance and satisfaction data

    • Performance data: time on task, error rates, # calls to the help desk, # references to the documentation, . . . .

    • Satisfaction data: End-of-test questionnaire

  • Collect benchmark data

    • Let’s talk about find-and-fix vs. benchmarking

  • Ensure coverage of certain parts of the UI -- good control


Eut limitations
EUT - Limitations

  • Artificial situation

  • Successful test doesn’t “prove” the product works

  • Need representative users!

  • Ease of learning vs. ease of use

  • Hard to test longitudinally


Eut what to test
EUT -- What to test?

  • Can rarely cover all the UI.

  • I like to test:

    • critical tasks

    • frequent tasks

    • nettlesome tasks


Rubin s 4 types
Rubin’s 4 Types

  • Exploratory (working on the “skeleton”)

  • Assessment test (working on the “meat and flesh” of the UI)

  • Validation test (does it meet the objectives?)

  • Comparison test (compare two competing designs)

    • Rubin, J. Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. Wiley: New York, NY, 1994.


Set up
Set up

  • Create environment (ambient setting, HW, SW)

  • Identify participants

  • Establish test roles (test monitor/administrator, data logger, timer, video operator, product experts, other observers)

  • Create test plan

  • Prepare observers

  • Prepare test materials


What materials
What materials?

  • Instructions

  • NDA

  • Permission to videotape

  • Test scenarios

  • Questionnaire(s)


Conduct the test
Conduct the Test

  • Welcome the test participant

  • Communicate that this is not a test of THEM, and they can leave any time

  • The scenarios should match the real world setting

  • Ask the test participants to “think aloud” to better understand intent

  • Offer post-test questionnaire, and debrief


After the test
After the Test

  • Quick data to product team

  • Assign severities and build recommendations

  • Build archival report

  • Serve as change agents!


What is a heuristic evaluation

What is a Heuristic Evaluation?

Evaluators systematically inspect the application interface to check for compliance with recognized usability guidelines (heuristics). (Thus, an INSPECTION method.)

Identifies major and minor usability problems

- Conducted by three to five experienced usability engineers

- Problems are reported along with the heuristic it violates


Problems identified

Problems Identified

The probability of one evaluator finding . . .

A major usability problem - 42% *

A minor usability problem - 32%

More evaluators = more problems identified

*from www.useit.com


Strengths of an he

Strengths of an HE

Done by people experienced in usability not just “dumb” users

Can identify both major and minor usability problems

Can be done relatively quickly and inexpensively

UNlike EUT, can sometimes cover every corner of a UI or web site


Weaknesses of an he

Weaknesses of an HE

Since done at end of design, designers may be resistant to changes

Some designers/developers may be unmoved by “just opinions”

Experienced usability evaluators may miss content problems that actual users would find

Can HELP address this issue by using SMEs


Typical methodology

Typical Methodology

Interface is exercised

-1st pass to develop the big picture

-2nd pass to accomplish typical tasks

Each problem is reported along with the heuristic it violates

Comments are consolidated

Severity levels – Critical, Major, Minor


Heuristics used

Heuristics Used

- Architecture

- Task Design

- Navigation and User Control

- Consistency and Standards

- Information is Meaningful

- Flexibility and Efficiency of Use

- User Support

- Recognition rather than Recall

- Aesthetic and Minimalist Design


Severity levels used

Severity Levels Used

  • Critical Usability Issue:

  • Loss of user data

  • System shutdown

  • Abandoned task

  • Major Usability Issue –

  • Completed task but considerable frustration or extra steps

Minor Usability Issue –

- Moderate work around or multiple attempts

* Usability Suggestion


Sample summary of results

TYPE

CRITICAL

MAJOR

MINOR

Installation

0

0

3

Project Creation

0

0

5

Visual Layout

0

3

16

Code Editor

0

1

1

Run Project

0

0

1

Debug

0

0

1

Main IDE

0

0

7

Icons

0

0

2

Help

0

0

2

Totals

0

4

38

Sample Summary of Results


Nielsen s usability heuristics
Nielsen’s Usability Heuristics

  • Visibility of system status

    • The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.

  • Match between system and the real world

    • The system should speak the users' language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.

  • User control and freedom

    • Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.

  • Consistency and standards

    • Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.

  • Error prevention

    • Even better than good error messages is a careful design which prevents a problem from occurring in the first place.


Nielsen s heuristics cont d
Nielsen’s Heuristics (cont’d.)

  • Recognition rather than recall

    • Make objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.

  • Flexibility and efficiency of use

    • Accelerators -- unseen by the novice user -- may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.

  • Aesthetic and minimalist design

    • Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.

  • Help users recognize, diagnose, and recover from errors

    • Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.

  • Help and documentation

    • Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large.


How to
How to . . .

  • http://www.useit.com/papers/heuristic/heuristic_evaluation.html


Other heuristics
Other Heuristics

  • http://www.stcsig.org/usability/topics/articles/he-checklist.html

  • One view: Non-categorized reporting. Instead of organizing our findings in terms of a heuristic category, we prefer to rate how much a specific feature or design element hinders the users' tasks or the company’s objectives.

    • We mention the feature, the objective impacted (and why), the severity of the impact, and any recommended changes that might correct the problem. All this comes after we have done some initial business, user, and task analyses. In the report we will typically group by the feature set or objectives of the product or site, which makes it easy for the client to relate to since that is typically the only organization they are already familiar with. We usability types know what the heuristic categories mean, but our clients typically don't. So organizing by these categories is not very meaningful for them, if not just down right confusing.


Usability walkthroughs
Usability Walkthroughs

Purpose

  • To collect user data -- from multiple users at one time -- to help drive the design of a user interface


Context
Context

  • This is just one step in a series of “User-Centered Design” methods we’re employing:

    • Heuristic evaluation (professional judgment)

    • Usability walkthrough (YOU ARE HERE)

    • End-user testing in the lab


Key characteristics
Key Characteristics

  • Three types of people in the room

  • We’re going to go through some task flows, as a group.

  • You’ll have hard-copy packets, where you will write the actions you would take, if you were online, carrying out a certain, prescribed task.

  • No discussion until all have written a response.

  • We’ll announce the “correct” action (according to the current design).

  • We’ll discuss the page -- representative users first.


Please note
Please note . . .

  • Design

  • is

  • HARD

  • SW development teams budget time to debug the code; we’re debugging the design!


This is a test of the design
This is a test of the DESIGN!

  • Not a test of YOU.

  • If you have trouble finding the “right” answers, then WE have a problem with the UI.

  • The [Product] team is being very bold, to expose their design to users early like this -- but they’re doing it because they realize the benefit, and they want to get it right!


And so
And so . . .

  • Given the input we hear today, the [Product] team (assisted by a usability professional) will redesign the interface.


The flow of the day
The Flow of the Day

  • We’ll hand you a piece of paper with a scenario description on it.

  • We’ll hand you a packet of screen shots, in order. DO NOT LOOK AHEAD, please.

  • We’ll ask you to write down on the top of the page what action you’d take, to accomplish the task in the scenario.


The flow cont d
The Flow (cont’d.)

  • We’ll announce the “right” answer.

  • I’ll ask you to indicate if you got it right. When you did not, I’ll welcome discussion.

  • I’ll welcome the designers and developers to jump in, as discussion winds down.

  • I’ll let SOME redesign go on, real-time.

  • Write more comments under the screen.


Yet more flow
Yet more flow . . .

  • Then you’ll be asked to turn the page.

  • Now, you’ll have to “reset,” and assume you got to this new page (somehow).

  • Then, what action would you take on THIS page, while still trying to accomplish the task?

  • After a scenario, we’ll give you a questionnaire to complete.


Benefits
Benefits

  • Lot of data early in the design cycle.

  • Usability of individual screens, terminology, SOME task flow.

  • Collaborative redesign on the fly.


Limitations
Limitations

  • Can’t get some data (e.g., time on task).

  • You can’t browse, as you might online -- tendency to “lose your place.”

  • Feel free to turn BACK in your packet, but not ahead.


Any questions
Any questions?

  • We’ll try to finish up by noon or so.

  • Then have some more informal time to discuss the interface.

  • We may get more informal at points.

  • THANK YOU very much for being here.

  • Absolutely NO ONE should be embarrassed -- not the users, not the developers.



Next week
Next Week

  • Meet here for panel session of usability professionals.

  • Turn in test plan.


ad