HCI 510 : HCI Methods I

HCI 510 : HCI Methods I
Usability

Usability Testing Introduction Attributes Process Numerical Web Pages Corporate Mistakes

Usability : Introduction

Usability : Introduction Usability is mostly measured on a simple, two-value scale: it’s terrible or it’s okay. Your early design efforts are usually terrible; you keep improving them until they are okay, then you ship. Most usability engineering is qualitative in nature: you observe how users use your product what they have difficulty doing, what they like, and what they hate-then you refine your design accordingly. Given the severe time pressure in most development projects, it’s enough to know that something is good or bad. Rarely is it necessary or possible to take the time to find out how good or bad, since if it’s good you want to leave it alone and if it’s bad you want to change it (if you have time, that is). There are some cases, though, where you do want numbers.

Usability : Introduction Generally, quantitative data makes when you are trying to decide between two or more alternatives. Individual users making a purchase might rely on gut feelings about product quality, but a corporate IS department that is trying to pick a standard Web browser for tens or hundreds of thousands of users should be more thorough. Let’s say there are two competing products with a marked difference in price. If the IS group decides the cheap ,one is the best, the choice is clear. If, however, they think the expensive better, they must then decide if the increased quality is worth the price.

Usability : Introduction Metrics are also important for organizations that want to increase their level of usability engineering maturity. Typical issues to consider are how good your own products are, how they compare to the competition, and what the trend curves are for your own and competitors’ products. If, for example, your product is currently 30 percent more usable than the closest competition, you may feel safe, but if your cempetitors typically improve usability by 20 percent per year and your improvement rate is 5 percent, they will surpass you in two years. Obviously, you’re better off estimating these annual changes while you still have time to enhance your usability efforts.

Usability : Introduction

Usability : Introduction “Although measuring usability can cost four times as much as conducting qualitative studies (which often generate better insight), metrics are sometimes worth the expense. Among other things, metrics can help managers track design progress and support decisions about when to release a product.” Jakob Neilson

Usability : Introduction Usability can be measured, but it rarely is. The reason? Metrics are expensive and are a poor use of typically scarce usability resources. Most companies still under-invest in usability. With a small budget, you're far better off passing on quantitative measures and reaching for the low-hanging fruit of qualitative methods, which provide a much better return on investment. Generally, to improve a design, insight is better than numbers. generated during the year.

Usability : Introduction However, as organizations increase their usability investments, collecting actual measurements is a natural next step and does provide benefits. In general, usability metrics let you: Track progress between releases. You cannot fine-tune your methodology unless you know how well you're doing. Assess your competitive position. Are you better or worse than other companies? Where are you better or worse? Make a Stop/Go decision before launch. Is the design good enough to release to an unsuspecting world?

Usability : Attributes Unfortunately, it is somewhat simplistic to assume that usability can be measured by a single number. There is no such thing as a “user friendly” product. It is only meaningful to speak of usability relative to certain types of users performing certain categories of tasks. For example, a graphics program might be rated highly usable if children want to use it to draw postcards, but highly unusable if engineers want to use it to draw airplane engine parts. Another program might get the opposite score.

Usability : Attributes So what are the attributes of a usable system ?

Usability : Attributes So what are the attributes of a usable system ? Usability is normally measured as five product attributes: Learnability: The system should be easy to learn so that the user can begin working quickly. Efficiency: The system should be efficient, so that once the user has learned it, high productivity is possible. Memorability: The system should be easy to use and remember, so that the casual user can return to it after not using it for a time and still know how it works. Errors: The system should have a low error rate, so users will encounter few errors; those they do encounter should be easy to recover from. Further, catastrophic errors must not occur. Satisfaction: The system should be pleasant to use.

Usability : Attributes To Measure Learnability ?

Usability : Attributes To Measure Learnability ? You can measure these attributes by timing 10 typical users as they perform a range of tasks on the system. You choose novice users to measure learnability If, for example, novice users could perform a set of test tasks in 50 minutes on average, you assign the system a learnability score of 5O, meaning that another group of novices would have to perform the same tasks in 40 minutes for the system to have improved its learnability score by 20 percent.

Usability : Attributes To Measure Efficiency ?

Usability : Attributes To Measure Efficiency ? You can measure these attributes by timing 10 typical users as they perform a range of tasks on the system. You choose experienced users to measure efficiency. If, for example, experienced users could perform a set of test tasks in 50 minutes on average, you assign the system a efficiency score of 5O, meaning that another group of experts would have to perform the same tasks in 40 minutes for the system to have improved its efficiency score by 20 percent.

Usability : Attributes To Measure Memorability ?

Usability : Attributes To Measure Memorability ? You can measure these attributes by timing 10 typical users as they perform a range of tasks on the system. To measure memorability, you can bring back novice users after a few days and test them again. If, for example, novice users could perform a set of test tasks in 50 minutes on average, you assign the system a learnability score of 5O. If those novices come back a few days and perform the same tasks in 40 minutes then the users have improved by remembering the system. The system memorability caused an speed increase of 20 percent.

Usability : Attributes To Measure Error Rate?

Usability : Attributes To Measure Error Rate? Error rates are usually measured by counting user errors, either over a fixed period or during the performance of a given set of tasks. You can also measure error impact. For example, you could measure the time users need to recover from their errors or even how much money the business lost because of the errors.

Usability : Attributes To Measure Subject Satisfaction?

Usability : Attributes To Measure Subject Satisfaction? Subjective satisfaction is typically measured by having users rate their satisfaction with the system on a 1-to-5 scale. You can also ask them to rate various aspects of product quality, such as how pleasant the system was to use, how much control they felt over the dialog, how much they liked the graphics and sound effects, and so on.

Usability : Attributes Relative Measures The importance of each usability attribute will vary according to your product. Sometimes learnability is of utmost importance because users will use the interface only a few times. EXAMPLE Other times, efficiency is crucial because users will work on the system day in and day out for years. EXAMPLE

Usability : Attributes Worthwhile ? Usability metrics are not recommended for all projects. Qualitative methods are usually sufficient to ensure substantial usability improvements. Of course, you won’t know bow much better your new design is, but as long as it’s not worse than the old one, going with the new one is clearly the better decision. However, organizations that do extensive software development and care about their engineering maturity should collect metrics for at least some of their projects.

Usability

Usability : Process The process that Jeff Rubin and Dana Chisnell present in the Handbook of Usability Testing could be used for a formal usability test, but it could also be used for less formal tests that can help to explore ideas and form concepts and designs. The steps are basically the same for either kind of test: * Develop a test plan * Choose a testing environment * Find and select participants * Prepare test materials * Conduct the sessions * Debrief with participants and observers * Analyze data and observations * Create findings and recommendations

Usability : Process Develop a test plan Sit down with the team and agree on a test objective (something besides “determine whether users can use it”), the questions you’ll use, and characteristics of the people who will be trying out the design. The plan also usually includes the methods and measures you’ll use to learn the answers to your research questions. It’s entirely possible to complete this discussion in under an hour. Write everything down and pick someone from the team to moderate the test sessions. In the classic process, a usability test plan can be several pages long. Teams in the swing of doing testing all the time can work with a minimalist structure with one or two lines on the elements of the plan.

Usability : Process Choose a testing environment You need to find a location where you can run the test sessions. Contrary to popular belief, you don’t always need recording equipment or data-logging software. In fact, to run casual tests with a small group of users, all you need is a system to test (this could be a Web or paper prototype), a desk, two chairs and a participant. More formal or larger-scope testing should be accompanied by video or audio recording equipment for analysis. Conduct formal tests in an environment that simulates normal use as much as possible. Data-logging software, is nice to have and can be extremely useful for both the analysis and presentation of results.

Usability : Process Find and select participants The results are only as good as the people you evaluate. When you have built a project to scratch your own itch, your intended audience will be people just like you. Test subjects in this case should be easy to find – friends, co-workers etc. This is not always the case; your users may not be like you at all. When they are not, it’s all the more important to run usability tests. Testing on friends, family and co-workers is better than not doing usability tests at all, but can’t be compared to testing on actual samples of your intended audience. People who would use the system will provide more genuine feedback and deeper insight..

Usability : Process Find and select participants The results are only as good as the people you evaluate. Never let your test subjects put themselves in the shoes of your ‘actual’ users. For example, you should discourage comments like “Well, I would do this BUT if I was a bus driver I’d do that”. Users are not qualified to put themselves in the position of others. Inaccurate data is often worse than no data. Aim for five or six test subjects: any more and you probably won’t learn anything new; any less and you’re likely to be overwhelmed by issues stemming from people’s individual personalities.

Usability : Process Find and select participants Web Site Example Imagine that you’re creating a site that sells images. Your customers are people who want to buy images—a huge group of people. Narrow your focus to a short and concise user profile, a picture of your ideal test participants. This profile should be based on your primary user (customer) segment and contain characteristics that those users share.

Usability : Process Find and select participants Web Site Example In this scenario, our participants are graphic designers or other people who use graphic design software and purchase images online. Create and order a list of these users’ characteristics. While you’re creating the user profile, you may realize that you have two or more equally important subgroups—people who buy images for business use and people who buy images for home use. This is fine as long as you can justify the relevance of each subgroup to the features that you’ll be testing.

Usability : Process Prepare test materials (tasks) Choosing tasks Set tasks that are essential to the new system’s success, For example for a web site these may be: * Buying products * Paying bills * Contacting the client The system was built for a reason - can your target audience do what you need them to do? It's also a good idea to ask the user to suggest tasks. While this gives another indication of their expectations and requirements, it may suggest new functionality or priorities.

Usability : Process Prepare test materials (tasks) How to word tasks People tend to perform more naturally if you provide them with scenarios rather than instructions. When giving them tasks, you should use phrases like ‘Scenario A has occurred, and you need to ring the company urgently - find the telephone number’. This is far better than ‘find the contact us section of the site’.

Usability : Process Prepare test materials (tasks) Presenting tasks Only give participants one task at a time. More than this may intimidate them, or alter their approach to the test. If the user is required to use inputs from outside the test (e.g. an email giving them a password to the site), give them these inputs in the form they will be presented. This will provide useful feedback on all elements of the process, rather than simply the system.

Usability : Process Prepare test materials (script) The Script is a single side of A4 (or letter) paper, consisting of questions for your testers and reminders for yourself. Have a balance of task-based questions and expectation analysis. This helps maintain consistency across tests. Expectation analysis is more important for testing designs and new applications: “Where would you find X?”, “What would you expect to happen if you clicked on Y?”. In an established system users will probably know where these things are though it can still be illuminating to ask these questions though phrased slightly differently.

Usability : Process Prepare test materials (script) Task-based questions involve providing a task for the user to complete. If you are testing an established system it is a good idea to ask users to bring in tasks that they would normally perform. This is because the user will be more invested in the outcome of the task and will behave in a more realistic fashion. When designing tasks for new systems and designs ensure you only provide loose task details for the same reason. Avoid introducing bias with the way questions are phrased.

Usability : Process Prepare test materials (script) It’s a good idea to ask for users’ first impressions at the beginning of the test, especially when testing design mockups. “What are the main elements on the page?” or “What strikes you first?”. You script should run for a maximum of 45 minutes. 30-35 minutes is better. Your script should be flexible. It is possible that during the test a trend will come to light that opens up whole new avenues of possible questioning. For example, during one initial test of an established system I noticed that the test subject had been printing off items from the application and placing them in a folder in date order (the system ordered alphabetically). I changed the script to ask future participants in that run, if they ever used external tools to help them with tasks within the system. This revealed a number of interesting issues that otherwise would not have been found.

Usability : Process Prepare test materials (legalities) Tests may need to be accompanied by a legal form that addresses three important issues. The first is nondisclosure, a confidentiality agreement regarding a site or service that is under development. Participants should be instructed not to talk about the site or their opinions of it to anyone. Next is the waiver, which gives consent to use any recording made during the test for the purpose of evaluating the site. Finally, the legal form should clearly state the participant’s rights, which informs participants about their right to withdraw, take a break, have privacy and understand the purpose of the test.

Usability : Process Conduct the sessions (when) In an ideal world usability tests would be carried out frequently from an early stage of the project. Time and budget constraints lead this to be impractical; usability is often the first thing to get dropped from the project plan. If you can only test at one stage in the project, whatever the size, the most valuable time is before your first public beta — leaving long enough to fix issues and not so late that you can’t rethink your scope. There are three main categories of usability test: * Testing design mockups * Testing a new working application * Testing established applications

Usability : Process Conduct the sessions (before) As with everything in life, first impressions are vital. Each participant must be put at ease. Remember, the usability testing session is often an extremely artificial environment and, for the most beneficial and informative results, we want them to behave as if they were using the site at home or work. Ensure any forms or instructions are written in plain English, and are as short as possible. The last thing any nervous usability testing subject wants is to be given a contract that looks like they're signing their soul away. All you want is for them to be reassured that the tests are completely confidential, and for permission to use the data generated during the test as part of our results.

Usability : Process Conduct the sessions (beginning) Before diving into key tasks, get the user familiar with the environment. Tell them about the system, and ask them for initial feedback on what they would expect from the system or what they would like the system to do. Make note of any terms or phrases they use - this not only demonstrates you are taking their feedback seriously, but may provide useful tips as to possible labels for key functionality or navigation. Next, let them look at the system they are testing. Gauge their first impressions before allowing them to familiarise themselves with the system. These few simple tasks will help convince the participant that the usability testing will not be difficult and, perhaps most importantly, that they're not the ones being tested.

Usability : Process Conduct the sessions (during) It's essential that you remember that it's the system that is being tested, not you or the subject. Any feedback you get is valuable - make sure the participant knows this. If they can't do something, make sure they know it's not their fault. You must stay quiet and out of sight during the test. You must not alter the test results by providing clues, suggesting directions or by reacting to things they say or do. All feedback you give must be neutral. Do not start shaking your head or huffing, however tempting it might be! The only time you should speak is to help the participant give an opinion, or to clarify a response. If in doubt, keep quiet.

Usability : Process Conduct the sessions (during) Moderating the Test Keep the participants on the task. People will tend to wander off, but it’s important to keep them focused on the task. Take shorthand notes or (even better) get someone else to take notes, focusing on really important/interesting behaviors. Let the participant make mistakes. This will reveal aspects of the interface that may need improvement. Answer questions with questions, forcing the user to give more feedback. Don’t tell the participant what to do, but you can suggest and discuss design solutions that might solve a usability issue. Don’t explain the interface. Doing so prevents you from getting their unbiased perception of the site.

Usability : Process Conduct the sessions (during) “Treat your test subjects like hedgehogs. Depending on your target audience they probably feel a little nervous and perhaps even scared of you. So make them a little nest out of straw, stroke their prickles and give them some cat food.” Reassure them that you are testing the system and that they can’t give a wrong answer. Try to ensure the test environment is relaxed and quiet, but also as close as possible to the situation where they would actually use the system. Have your subjects talk out loud is very important as you can’t read their minds, but it is a very unnatural process. “To loosen up your subjects and get them talking in the way you want them to, try the Stapler Trick. Give them a stapler or similar item and ask them to open it, take the staples out, replace them, shut the stapler and staple some paper – talking all the time about what they see, what they expect to happen, what actually happens and how that matches up.”

Usability : Process Debrief with participants and observers Advanced Test Moderation Probe expectations. Participants usually have expectations about what will happen before they click on something. Well-timed questions asking them what they expect can reveal a lot about their perception and understanding of the site. Investigate mistakes. It’s always a good idea to follow up mistakes with probing questions, especially when the participant doesn’t realize they’ve made a mistake. Ask “Why?” a lot. Why did they click on that link? Why wouldn’t they use a product like this? Why are they so upset?

Usability : Process Analyze data and observations Primarily, you should look out for incidents where the user stops concentrating on their tasks and starts thinking about the tool and how she is going to use it. For example, when you are hammering in a nail you don’t think about how to use a hammer; good software should be the same. Words like ‘it’ and ‘the system’ are good indications that the test subject has stopped thinking about the task in hand. Note questioning words, especially where testers question their own judgement, “why can’t I find …”, “I expected to see …” etc. as this indicates that the work flow for the task may have broken down.

Usability : Process Analyze data and observations Also keep an eye on occasions where the user completely fails to do a task. They may need some prompting to unstick them, but you should be careful not to bias the test. These should be the highest priority issues for you to fix. If users recover from getting stuck, make a note of how they recovered. Prolonged periods of silence from the test subject may also require prompting as they should be talking all the time. Ask them what they are thinking or looking for but avoid words like ‘try’ (e.g. ‘what are you trying to do?’) as this implies that they are currently failing.

Usability : Process Create findings and recommendations Even if you are the only developer it’s important to summarise the key issues that emerged during testing: your notes won’t make much sense to you a week or so after the test. If you are writing for other people, include a summary no longer than two pages; this can consist of a list or table of the issues including recommendations and their priorities. Remember to anonymise the users in the report. In team situations, you may be surprised at how many people are interested in the results of the usability test even if it doesn’t relate directly to something that they can fix.

Usability

Usability : Web Pages Web Page Metrics Any idea why visitors view so few pages? Why do they click on that one link, but never on the other one? And how satisfied are they with your website? Only these figures don't say why users do what they do, and what they think about it. If you limit yourself to these figures, you may draw the wrong conclusions.

Usability : Web Pages Web Page Metrics Figures on the user experience. In addition to the figures on use you'll want to know what influences a visitor's experience. Positive and negative. And from the first impression to intensive use. Usability metrics can give a detailed and reliable picture of both the use and the appreciation of a website. Look at how visitors navigate through the site and ask questions about their expectations and experiences before, during and after this.

Usability : Web Pages Web Page Metrics Every real estate site or blog’s success can be measured in the usability of the site. Many sites today still concentrate on being pretty with special effects like Flash or AJAX, sound or video. While cool looking in most cases it’s not the most important factor that decides whether your site is going to fail or to succeed, usability is.

Usability : Web Pages Web Page Metrics Bounce rate The bounce rate is one of the most important usability metrics and thanks to Google Analytics it is easy to follow. 1000 visitors with a bounce rate of 95% means that in fact only 50 actually visited your site. So a site with a much lower visitor number AND bounce rate can be much more successful than a “stupid traffic” site with huge traffic numbers. Targeted quality traffic is the main key to a successful site.

Usability : Web Pages Web Page Metrics Returning visitors This should be obvious, only returning visitors really like your site. So the more come back the better, the more successful you are. One time search visitors and casual social media visitors are not the backbone of your site. The subscribers and returning visitors will be.

Usability : Web Pages Web Page Metrics Pageviews per visit Measuring pageviews is over hyped. If they land on the right page to start with then you have succeeded. Often bad sites require users to navigate through the site several layers to accomplish your goal not theirs. Time on page The time spent on a page can be read in many ways but you can deduce from it whether people just skim your content or read your whole article among others. Time on site It’s not always the longer the better but 5 minutes is in most cases better than 30 seconds, especially for a MLS search page or blog. Next pages To encourage people visit more than one page on a site we use internal links. Some of the links are links that we really want the people to follow such as “free CMA” or “list of foreclosures”. Understanding the “next pages” from a particular landing page we can determine whether the readers followed our advice or not and adjust our strategy accordingly.

Usability : Corporate Stages Stage 1: Hostility Toward Usability Stage 2: Developer-Centered Usability Stage 3: Skunkworks Usability Stage 4: Dedicated Usability Budget Stages 5: Next Column

Usability : Mistakes Mistakes Mistake #1: Do You Know Why You're Testing? Mistake #2: Not Bringing the Team Together Mistake #3: Not Recruiting the Right Participants Mistake #4: Not Designing the Right Tasks Mistake #5: Not Facilitating the Test Effectively Mistake #6: Not Planning How You'll Disseminate the Results Mistake #7: Not Iterating to Test Potential Solutions

HCI 510 : HCI Methods I