Usability Testing for Accessible UX

1. Usability Testing for Accessible UX Kate Walser Whitney Quesenbery

2. Hi Kate Interaction design, usability testing and slipping accessibility under the radar. Whitney Users’ stories, plain language, discovered accessibility as a right in elections What about you?

3. What do you want to learn today? What do you think usability testing is? Does anyone in your organization do any sort of usability testing? What stands in the way of doing usability?

4. Here’s what we planned Introduction to usability testing Demo of a usability test Planning a usability test First practice test Debrief and moderating skills

5. Schedule

6. What is usability (testing)? 1

7. Usability is about people Different types ofpeople…doing differenttasks… have differentusability needsdepending on their goals and tasks For this person efficiency and effectiveness are the most important dimensions. For this person, easy to learn and error tolerant are the most important dimensions.

8. Helpful models for thinking about usability A’s Ability, aptitude, attitude (what users bring to the site) E’s Efficient, effective, engaging, error-tolerant, easy to learn (dimensions of usability) Use Useful, usable (operable), desirable (how we define success)

9. Usability testing lets you see behavior Observe • People like the audience using the site rather than asking them about it • How easily they meet their goals • What causes confusion or problems Inform design • Lets you confirm or challenge assumptions about the design. • Gives you the opportunity to fix problems. Informal usability testing of voting materials at the Farmer's Market in Olympia, Washington

10. How easily can we learn from users? We learn from users by watching them use the site or other materials. • Don’t explain or demo. • Watch what they do. • Listen to their comments. • Take their problems seriously. Dona Vitale conducting a usability test in Chicago, 2005

11. A usability test lab A formal lab includes a room where the participants work, and a separate room for observers to watch. Sessions are often recorded on video Michigan State University Usability & Accessibility Centerhttp://usability.msu.edu/default.asp

12. Usability testing vs. accessibility audits How will a person with disabilities use the product? • Assistive technology use and settings • Other resources they use How usable is the product by people with disabilities? • Content, forms without surrounding visual context, proximity • Navigation • Truly accessible vs. just meets the standards How engaging and effective is your content strategy? • Images only vs. engaging context + images • Search engine readiness (headings, link labels) • Great content flow

13. Demo 2

14. Your goal You want to take a group from AccessUto go fishing at a local park. Find a park that has a wheelchair accessible fishing pier. What are the admission fees?

15. Planning a usability test 3

16. Begin with your goals Your site visitors' goals (and their reality) Your organization'sgoals • Answer people's questions about your topic • Have people complete a transaction correctly without calling • Get more people to fill out a form correctly without calling Success − for youand your users

17. The purpose of the test influences the plan Location and context: formal informal Recruiting: defined opportunistic Activities: instructed tasks free tasks Questions: structured unstructured Data collection: observation task/data only Results quantitative qualitative

18. Think about what senses are most involved in the UX

19. What you need for a usability test What • The material you want to test • Scenarios, if it’s not just an exploratory “try it out” type test Where • A quiet room (Maybe. We know of very successful usability testing in an open marketplace, a hangar at an air show, the chemotherapy center at the NIHClinical Center.) Who • Moderator • Observer/note-taker • Users: 3 − 6 people, one at a time

20. The moderator runs the session • Impartial, unbiased, observing • No teaching! • Listen and watch • Open-ended questions: Why?How? What were you doing? Moderator roles: Flight attendant: Ensuring safety and comfort Scientist: Planning, maintaining objectivity, managing data

21. Observers and note-takers Watch (and listen) quietly for whether the user • Is confused by anything, or has to re-read information • Has to correct mistakes, or backtrack to correct naviation Take good notes • Write down what you see and hear. Don’t “translate” or put words in the user’s mouth • Be specific. Not "he's confused." But "he said he doesn't know what APR means.” • Don't infer the user's reasons for doing something. • Don't solve problems while taking notes. That's for later. It will take all your concentration to note what is happening.

22. Briefing the user • Thanks for trying out this […]. Your doing this will help improve this material. • Note: If you never use the words "test" or "evaluate" with the user,you will not have to say "We are not testing you." • You can stop anytime. • Your involvement will be confidential. • If you get stuck or confused, say so. • Please let me know what you are thinking as you use this …

23. Different ways of having people try out your site Ask participants for their own scenarios. • Have you used this site? What for? Please show me how you did that with this site. or Would you be likely to use this site? What for? Please show me how you would do that with this site. Give participants tasks you have written. • Write scenarios that will have people use parts of the site you are worried about. Customize scenarios for the participants. • Write general scenarios, but adapt them to what you know about the participant, letting them fill in details to make the task more relevant. Use both of these with their own scenario first.

24. Use think aloud during − or after the task Consider asking people to “think aloud” as they work • What they’re doing • Why they’re doing it Assistive technology may make think aloud harder. If so, you can go back over the material (“retrospective”) • Ask the participant to walk you through what they did and why. • Use the material as a guide for the discussion. • Ask if anything was confusing or frustrating.

25. Use simple techniques to help users keep going If the user says, “hmmm” or “oops” or “I wonder…” • Say, “What questions do you have right now?” If you are doing "think aloud" and the user is silent for 10 or 20 seconds (count!) • Say, “What are you thinking?” If users stop because they think they're done or they are stuck (and you think there’s a problem) • Summarize what you saw the user do. • Ask "What would you do now?"

26. First practice test25 minutes 4

27. Plan your practice test Start with the task: what do you want people to do? Review the script for briefing the participant. Write the task instructions. Starting point (desktop, a URL?) Phrase it in their language What do you want to watch for? Remember to thank the participant at the end

28. Ready? Three rounds Get into a group of three. Take turns, so everyone tries each role. We’ll call time.

29. How did that go? What felt awkward? What worked well? Anything surprise you?

30. Moderating skills 5

31. From earlier: The moderator runs the session • Impartial, unbiased, observing • No teaching! • Listen and watch • Open-ended questions: Why?How? What were you doing? Moderator roles Flight attendant Ensuring safety and comfort ScientistPlanning, maintaining objectivity, managing data

32. Moderator Responsibilities Often, but not always, create the test plan and testing scenarios Welcome and support the participant • Informative but not revealing Avoid accidentally influencing results • Explain the objectives succinctly • Helpful but not too helpful • Try not to answer questions that affect outcomes Observe quietly and avoid distracting participants • Thoughtful about taking notes • Ask questions only when they realize participant is at a stopping point or is ready to talk

33. Managing the customer Often, moderators also manage the customer team whose products will be used in the sessions.

34. Prepare the customer team Define and prioritize objectives • Address these in scenarios, probe questions, questionnaire • Determine which are “must haves” and which are optional Describe to the customer the process, logistics • Where they will sit (if they will attend) • What you expect from them (no interruptions, other room, etc.) Explain how you will handle “edge cases” • Late or no-show participants • Technology woes • Questions that arise during the sessions (and how to get moderator’s attention to ask)

35. A good moderator uses a test plan Creates a test plan to use during the session • Manage the usability testing like a mini-project • Provide clear schedule with tasks and deadlines – moderator, customer, etc. • Follow up to make sure everything’s set Creating a test plan and schedule for usability testing helps avoid mistakes and oversights.

36. A good moderator Prepares • Create a test script • Practice on real participants • Verify that everything is ready and there are plan B’s Arrive early – “on time” is “late” Listen and watch with an open mind • If you wonder, ask without bias – don’t jump to conclusions • Story – boy with the black crayons Respect the participant • Watch for “trouble” indicators • Stop as needed due to time, tech issues, or other factors

37. Expect most participants to be… Hesitant or nervous • Are thier skills being tested? • What if they don’t give the “right” answer? Ready to throw their friends and colleagues under a bus before themselves! • Give them an easy way to provide their feedback without worrying about hurting feelings • Ask what their friends or colleagues will think Photo credit: “accuse,” Grey Worldhttp://www.flickr.com/photos/greyworld/ Smiling man points to friend who is covering his face.

38. Great moderators make participants feel valued and important If possible, confirm the session yourself • Call or email the participant to confirm, introduce themselves • Make sure participant knows where to go • Give participant a friendly face to expect Memorize the script Learn negotiation tactics • Maintain an open mind • Manage challenges that arise (Handle surprises without disruption A little girl sitting in dentist’s chair is smiling from ear to ear after learning she has no cavities. Photo credit: “No Cavities!” daveparkerhttp://www.flickr.com/photos/daveparker/

39. Have a backup plan Photo credit: “Usability Testing our Paper Prototypes” Jason de Runahttp://www.flickr.com/photos/jderuna/ Person uses paper printout of screens to describe interaction and path

40. Summary: Moderators Prepare and anticipate problems Observe without influencing outcomes Make participants feel valued

41. How easily can we learn from users? Usability testingdoes not have to be formal, lengthy, or expensive. You don’t need • a formal laboratory • 100s of participants • special equipment (except for your voting system) • special recording systems Poster from Washington State

42. Take any opportunity for a quick usability observation The next time someone asks you a question about your site, ask them to use it instead. Watch as they try to find the information.

43. Ready for more? After lunch: Usability testing with people with disabilities: practice running sessions with people using assistive technology

44. Learn more about usability testing Handbook of Usability Testing by Jeff Rubin and Dana Chisnell usabilitytestinghowto.blogspot.com Rocket Surgery Made Easy by Steve Krug www.sensible.com www.peachpit.com/promotions/promotion.aspx?promo=137602 Rocket Surgery demonstration video www.usability.govA guide to developing usable and useful web sites (from US Dept of HHS) www.howto.gov/web-content/usability/first-fridays First Fridays Product Testing Program

45. Whitney QuesenberyCenter for Civic Designwhitney@centerforcivicdesign.org908-617-1122 Kate Walser CX Insights kwalser@cxinsights.com571-281-2626