The Usability Test Process: Steps, tips, and more!

1. The Usability Test Process:Steps, tips, and more! Dr. Jennifer L. Bowie For Digital Rhetoric

2. The Design Continuum System-Centered User-Friendly User-Centered • Users dumb • Users same as us • Will use regardless • Bells and whistles • Do what they can, not what they should • Consider the audiences • Users will like this • Often draw on stereotypes • Reasoning not necessarily supported • Based on untested profiles and assumptions • Users valuable • Users part of the design process • Early focus on users • Iterative • Involves research of/with users • Includes participatory design, contextual inquiry, ethnography, and usability testing

3. What is Usability? • “A function of particular users performing particular tasks in a particular environment” (Smith et al. 68) • The “people who use the product can do so quickly and easily to accomplish their own tasks” (Dumas and Redish 4) • User-centered design, not “user-friendly”

4. What is Usability Testing? • An empirical study of a product’s usability by observing actual users do real tasks with the product • Involves: • Real users • Real tasks • Specific usability goals/concerns • Observing and recording the testing • Data analysis

5. Step 1: User Analysis & Profiles • Who are your actual users? You may need to break your users into typical user categories. Consider: • Demographics: age, sex, race, education level, cultural background, socioeconomic status,… • Experience level with the product, with products of the same genre, with required technology,... • Other things: • motivation • learning style • subject matter knowledge • location of use • physical characteristics • people with disabilities or impairments (from color blindness and learning disabilities to more severe disabilities)

6. Step 1: User Analysis & ProfilesCon. • Create user profiles: • Break users into clear subgroups • Profile/Define the characteristics of each subgroup • Choose user profiles to test: • Ideally users from all major profiles will be tested • If limited testing: Choose profiles based on highest number of users in that profile or profiles that you think may have the greatest usability issues

7. Step 2: Decide what to Test • Choose an overall purpose • Example: How useable is our new website? • Determine objectives or what you are testing for. Examples: • Does our search engine provide usable results in the first 5 links returned? • Are search results clear to the users? • Choose type of test: • Performance: Can they do it? • Understandability: Can they understand it? • Read-and-locate: Can they find it?

8. Step 2: Decide what to Test con. • Select tasks: • Consider tasks with a high chance of user failure (complex tasks, one-of-a-kind tasks, highly abstract or technical tasks) • Consider tasks with a high cost of user failure (tasks that require support, like help or support calls, to complete; tasks where data could be damaged or lost) • Consider: • First impressions (look and feel) • First tasks • Tasks most performed • Critical tasks • Specific problem areas • New task for the product • Select performance objectives (should be individualized for each task) • Time: How long to complete tasks, to find things, to performance procedures • Error/Success: user errors, attempts to do/find something, numbers of times section re-read, if the task was completed successfully

9. Step 3: Preparing for the Testing • Choose order of tasks: start easy, go sequential, or be random • Create written test materials: • Task list for users • Written welcome speech/ Intro to be read to user • Consent forms • Observation forms • Pre-task and post task questionnaires & interview questions • Other materials • Recruit participants & determine “payment” • Define team member’s roles: • Facilitator/Briefer (necessary): Often only team member to interact with users • Observation recorder/note taker (necessary) • Camera operator (optional) • Help desk operator (optional) • Test administer (optional) • Create written test plan • Practice: conduct walkthroughs of the testing and if possible pilot test (the pilot test users could even be a team member) • Prepare test environment (day of test)

10. Step 4: Conducting the test • Greet & Brief participant: • Read/say welcome • Emphasize that you are not testing them, but the product and that they should act as natural as possible • Explain think-aloud protocol (if using) • Emphasize how user tells you she has completed a task • Stress that the testing is “anonymous” • Be unbiased (especially the Facilitator/Briefer) • Intervene carefully (avoid as much as possible) • Observe and record data • Debrief user

11. Step 5: Analyzing the Data • Collate data into findings: • Choose an approach: • Top-down approach: predetermine categories of findings (like navigation, design, terminology) and go through data looking for “hits” • Bottom-up approach: put each observation on a sticky note/note card, sort into categories and label categories • Determine time and errors/success • Examine findings for each user, user profile, and task • Use analysis techniques such as statistics (even averages help)

12. Step 5: Analyzing the Datacon. • Analyze data: • Determine cause of problems • Determine scope/severity of problems • Make recommendations/changes • Report Findings

13. Good Luck & Have Fun! • Where to find out more: • Barnum Usability Testing and Research • Barker Writing Software Documentation, Chapter 6 “Conducting Usability Tests” • Hom “General Concepts of Usability Testing” http://jthom.best.vwh.net/usability/general.htm