Usability testing for usable accessibility
Download
1 / 63

Usability Testing for Usable Accessibility - PowerPoint PPT Presentation


  • 144 Views
  • Uploaded on

Usability Testing for Usable Accessibility. U. Kate Walser Whitney Quesenbery. Hi . Kate Interaction design, usability testing and slipping accessibility under the radar. Whitney Users’ stories, plain language, discovered accessibility as a right in elections What about you?.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Usability Testing for Usable Accessibility' - elam


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Usability testing for usable accessibility
Usability Testing for Usable Accessibility

U

Kate Walser

Whitney Quesenbery


Hi

Kate

Interaction design, usability testing and slipping accessibility under the radar.

Whitney

Users’ stories, plain language, discovered accessibility as a right in elections

What about you?


What do you want to learn today
What do you want to learn today?

What do you think usability testing is?

Does anyone in your organization do any sort of usability testing?

What stands in the way of doing usability?


Here s what we planned
Here’s what we planned

Introduction to usability testing

Demo of a usability test

Planning a usability test

Take a deep breath

First practice test

Debrief and moderating skills

Break

Testing with people with disabilities

Practice tests

Debrief and applying what you learned





Different types ofpeople…doing differenttasks… have differentusability needsdepending on their goals and tasks

For this person efficiency and effectiveness are the most important dimensions.

For this person, easy to learn and error tolerant are the most important dimensions.


Helpful models for thinking about usability
Helpful models for thinking about usability

A’s Ability, aptitude, attitude

(what users bring to the site)

E’s Efficient, effective, engaging, error-tolerant, easy to learn

(dimensions of usability)

Use Useful, usable (operable), desirable

(how we define success)


Usability testing lets you see behavior
Usability testing lets you see behavior

Observe

  • People like the audience using the site rather than asking them about it

  • How easily they meet their goals

  • What causes confusion or problems

    Inform design

  • Lets you confirm or challenge assumptions about the design.

  • Gives you the opportunity to fix problems.

Informal usability testing of voting materials at the Farmer's Market in Olympia, Washington


How easily can we learn from users
How easily can we learn from users?

We learn from users by watching them use the site or other materials.

  • Don’t explain or demo.

  • Watch what they do.

  • Listen to their comments.

  • Take their problems seriously.

Dona Vitale conducting a usability test in Chicago, 2005


A usability test lab
A usability test lab

A formal lab includes a room where the participants work, and a separate room for observers to watch.

Sessions are often recorded on video

Michigan State University Usability & Accessibility Centerhttp://usability.msu.edu/default.asp


Usability testing with people with disabilities
Usability testing with people with disabilities

How will a person with disabilities use the product?

  • Assistive technology use and settings

  • Other resources they use

    How usable is the product by people with disabilities?

  • Content, forms without surrounding visual context, proximity

  • Navigation

  • Truly accessible vs. just meets the standards

    How engaging and effective is your content strategy?

  • Images only vs. engaging context + images

  • Search engine readiness (headings, link labels)

  • Great content flow


Demo

2


Your goal
Your goal

You want to a group from Access to go fishing at a local park.

Find a park that has a wheelchair accessible fishing pier.

What are the admission fees?



Begin with your goals
Begin with your goals

Your site visitors' goals (and their reality)

Your organization'sgoals

  • Answer people's questions about your topic

  • Have people complete a transaction correctly without calling

  • Get more people to fill out a form correctly without calling

Success − for youand your users


The purpose of the test influences the plan
The purpose of the test influences the plan

Location and context: formal informal

Recruiting: defined opportunistic

Activities: instructed tasks free tasks

Questions: structured unstructured

Data collection: observation task/data only

Results quantitative qualitative



What you need for a usability test
What you need for a usability test

What

  • The material you want to test

  • Scenarios, if it’s not just an exploratory “try it out” type test

    Where

  • A quiet room (Maybe. We know of very successful usability testing in an open marketplace, a hangar at an air show, the chemotherapy center at the NIHClinical Center.)

    Who

  • Moderator

  • Observer/note-taker

  • Users: 3 − 6 people, one at a time


The moderator runs the session
The moderator runs the session

  • Impartial, unbiased, observing

  • No teaching!

  • Listen and watch

  • Open-ended questions: Why?How? What were you doing?

Moderator roles:

Flight attendant: Ensuring safety and comfort

Scientist: Planning, maintaining objectivity, managing data


Observers and note takers
Observers and note-takers

Watch (and listen) quietly for whether the user

  • Is confused by anything, or has to re-read information

  • Has to correct mistakes, or backtrack to correct naviation

    Take good notes

  • Write down what you see and hear. Don’t “translate” or put words in the user’s mouth

  • Be specific. Not "he's confused." But "he said he doesn't know what APR means.”

  • Don't infer the user's reasons for doing something.

  • Don't solve problems while taking notes. That's for later. It will take all your concentration to note what is happening.


Briefing the user
Briefing the user

  • Thanks for trying out this […]. Your doing this will help improve this material.

    • Note: If you never use the words "test" or "evaluate" with the user,you will not have to say "We are not testing you."

  • You can stop anytime.

  • Your involvement will be confidential.

  • If you get stuck or confused, say so.

  • Please let me know what you are thinking as you use this …


Different ways of having people try out your site
Different ways of having people try out your site

Ask participants for their own scenarios.

  • Have you used this site? What for? Please show me how you did that with this site. or Would you be likely to use this site? What for? Please show me how you would do that with this site.

    Give participants tasks you have written.

  • Write scenarios that will have people use parts of the site you are worried about.

    Customize scenarios for the participants.

  • Write general scenarios, but adapt them to what you know about the participant, letting them fill in details to make the task more relevant.

    Use both of these with their own scenario first.


Use think aloud during or after the task
Use think aloud during − or after the task

Consider asking people to “think aloud” as they work

  • What they’re doing

  • Why they’re doing it

    Assistive technology may make think aloud harder. If so, you can go back over the material (“retrospective”)

  • Ask the participant to walk you through what they did and why.

  • Use the material as a guide for the discussion.

  • Ask if anything was confusing or frustrating.


Use simple techniques to help users keep going
Use simple techniques to help users keep going

If the user says, “hmmm” or “oops” or “I wonder…”

  • Say, “What questions do you have right now?”

    If you are doing "think aloud" and the user is silent for 10 or 20 seconds (count!)

  • Say, “What are you thinking?”

    If users stop because they think they're done or they are stuck (and you think there’s a problem)

  • Summarize what you saw the user do.

  • Ask "What would you do now?"


How easily can we learn from users1
How easily can we learn from users?

Usability testingdoes not have to be formal, lengthy, or expensive.

You don’t need

  • a formal laboratory

  • 100s of participants

  • special equipment (except for your voting system)

  • special recording systems

Poster from Washington State


Summary when to do usability tests
Summary: When to do usability tests

Early

Understand users and their goals

Middle

Evaluate the site throughout the development process

End

Evaluate the finished site to measure its success

Ongoing

Watch site usage and technical support for evidence of problems

Project Initiation

Design and Development

Product Release

In Use


Take any opportunity for a quick usability observation
Take any opportunity for a quick usability observation

The next time someone asks you a question about your site, ask them to use it instead.

Watch as they try to find the information.


First practice test 25 minutes
First practice test25 minutes

4


Plan your practice test
Plan your practice test

Pick something you want people to do.

Review the script for briefing the participant.

Write the task instructions.

Starting point (desktop, a URL?)

Phrase it in their language

What do you want to watch for?

Remember to thank the participant at the end


Ready three rounds
Ready? Three rounds

Get into a group of three.

Take turns, so everyone tries each role.

We’ll call time.


How did that go
How did that go?

What felt awkward?

What worked well?

Anything surprise you?



From earlier the moderator runs the session
From earlier: The moderator runs the session

  • Impartial, unbiased, observing

  • No teaching!

  • Listen and watch

  • Open-ended questions: Why?How? What were you doing?

Moderator roles

Flight attendant Ensuring safety and comfort

ScientistPlanning, maintaining objectivity, managing data


Moderator responsibilities
Moderator Responsibilities

Often, but not always, create the test plan and testing scenarios

Welcome and support the participant

  • Informative but not revealing

    Avoid accidentally influencing results

  • Explain the objectives succinctly

  • Helpful but not too helpful

  • Try not to answer questions that affect outcomes

    Observe quietly and avoid distracting participants

  • Thoughtful about taking notes

  • Ask questions only when they realize participant is at a stopping point or is ready to talk


Managing the customer
Managing the customer

Often, moderators also manage the customer team whose products will be used in the sessions.


Prepare the customer team
Prepare the customer team

Define and prioritize objectives

  • Address these in scenarios, probe questions, questionnaire

  • Determine which are “must haves” and which are optional

    Describe to the customer the process, logistics

  • Where they will sit (if they will attend)

  • What you expect from them (no interruptions, other room, etc.)

    Explain how you will handle “edge cases”

  • Late or no-show participants

  • Technology woes

  • Questions that arise during the sessions (and how to get moderator’s attention to ask)


A good moderator uses a test plan
A good moderator uses a test plan

Creates a test plan to use during the session

  • Manage the usability testing like a mini-project

  • Provide clear schedule with tasks and deadlines – moderator, customer, etc.

  • Follow up to make sure everything’s set

Creating a test plan and schedule for usability testing helps avoid mistakes and oversights.


A good moderator
A good moderator

Prepares

  • Create a test script

  • Practice on real participants

  • Verify that everything is ready and there are plan B’s

    Arrive early – “on time” is “late”

    Listen and watch with an open mind

  • If you wonder, ask without bias – don’t jump to conclusions

  • Story – boy with the black crayons

    Respect the participant

  • Watch for “trouble” indicators

  • Stop as needed due to time, tech issues, or other factors


Expect most participants to be
Expect most participants to be…

Hesitant or nervous

  • Are thier skills being tested?

  • What if they don’t give the “right” answer?

    Ready to throw their friends and colleagues under a bus before themselves!

  • Give them an easy way to provide their feedback without worrying about hurting feelings

  • Ask what their friends or colleagues will think

Photo credit: “accuse,” Grey Worldhttp://www.flickr.com/photos/greyworld/

Smiling man points to friend who is covering his face.


Great moderators make participants feel valued and important
Great moderators make participants feel valued and important

If possible, confirm the session yourself

  • Call or email the participant to confirm, introduce themselves

  • Make sure participant knows where to go

  • Give participant a friendly face to expect

    Memorize the script

    Learn negotiation tactics

  • Maintain an open mind

  • Manage challenges that arise (Handle surprises without disruption

A little girl sitting in dentist’s chair is smiling from ear to ear after learning she has no cavities.

Photo credit: “No Cavities!” daveparkerhttp://www.flickr.com/photos/daveparker/


Have a backup plan
Have a backup plan

Photo credit: “Usability Testing our Paper Prototypes” Jason de Runahttp://www.flickr.com/photos/jderuna/

Person uses paper printout of screens to describe interaction and path


Summary moderators
Summary: Moderators

Prepare and anticipate problems

Observe without influencing outcomes

Make participants feel valued


Break 15 minutes
Break15 minutes

*


Practice tests 45minutes
Practice tests45minutes

6


Moderating and observing
Moderating and observing

  • Conduct sessions the same way.

  • Ask the participant to “pause” the AT or lower the volume as they may have thoughts they want to share.

  • Remember that the AT becomes a large component of the UX

  • Avoid interrupting the user as they use the AT

  • Remember to watch the cursor, not the mouse.


Get into teams
Get into teams

If you are the moderator, use your practice test.

If you are an observer, think about

What you will watch for

Where to sit so you can see, but won’t distract the participant.

Challenges observing this person’s AT or accessibility setup.

Remember: focus on what happens, not analysis or recommendations



How did that go1
How did that go?

What felt awkward?

What worked well?

Anything surprise you?


Consider how different at and disabilities affect usability test logistics
Consider how different AT and disabilities affect usability test logistics

  • Recruiting

  • Participants

  • Scenarios and handouts

  • Incentives

  • A place to tests


Recruiting
Recruiting test logistics

Allow more time to find participants

  • Screener + disabilities

    Look for community groups and ask

  • Meetup, Facebook, Twitter

    Use specialized recruiting services

  • KnowbilityAccessWorks portal

    Work with outreach groups

  • Advocacy groups

  • Employment non-profit groups

  • Colleges and universities


Outreach groups
Outreach Groups test logistics


Knowbility test logisticsAccessWorks Portalhttp://www.knowbility.org/v/service-detail/AccessWorks-Usability-Accessibility-Testing-Portal/3k/

Knowbility’sAccessWorks portal gives users with disabilities a way to sign up to participate in online tests and explains to usability and marketing professionals that they can use Loop11’s online testing and recruit users with disabilities.


Decide where when to conduct testing
Decide where, when to conduct testing test logistics

In Person

Conduct in a physical location

Traditional lab

Some other place

Moderator sits with participant

Be prepared to provide the assistive technologies or ask users to bring theirs (including any personalized style sheets for web pages)

Remote

Conduct via online tools

Accessible screen sharing software (or) accessible online usability testing

Moderator may facilitate or may just review results later

User uses their own, native, familiar assistive technology and setup


In person environment setup
In-person: Environment & Setup test logistics

Location becomes even more critical

  • Is it reachable by public transportation?

  • Is the building itself accessible?

  • Are the signs to find the room accessible?

    Room setup matters

  • Is there space to navigate with wheelchairs or walking devices? Is there space for a service animal to lie down near its human?

  • Are tables or desks wheelchair accessible?

  • If visually impaired, can the person sit close enough to the monitor to view the screen? Can you mirror the screen on another monitor for the moderator to view?

  • How close are other workstations if screen readers will be used?


For in person tests explain setup and ask about accommodations
For in-person tests, test logisticsexplain setup and ask about accommodations

Explain what will happen and ask about specifics

  • “We usually provide handouts for users to read that describe the scenario. What’s the best way to provide that to you?”

    Describe the setup, and include accessibility specifics in your confirmation emails

  • “We’re on the 3rd floor. There’s a ramp to the right of the front doors. The doors automatically open. There’s an elevator bay in the lobby to get to our floor.”

    Ask if they will bring anything

  • Service animal (would they like a rest or water stop for the animal?)

    If refreshments will be served, ask about dietary restrictions


Remote online setup use
Remote / Online: Setup & Use test logistics

Ensure that the testing and/or screen sharing application are truly accessible

  • Can the user download / access them ahead of time to install any plugins and test with their chosen assistive technologies?

  • Can they turn on / off screen sharing?

  • Can they access and use their assistive technology while using the screen sharing capabilities?

  • Can the remote moderator hear and see what they’re doing?

    Ask about the best way to interact with the participant

  • Explain what will happen and ask about specifics

  • “We usually provide the scenarios via the tool. What’s the best way to provide that to you?”


Schedule1
Schedule test logistics

Add time between sessions for

  • Setup time

  • Moderator break

    Dry runs are critical, preferrably with a representative user or else, with someone trying to reflect the different considerations

  • Online: Try downloading / opening the tool, going to the scenario, talking with the moderator with AT on

  • In-person: Try going from closest transportation spot to location, into building to designated location, taking / using handouts


Tips for moderating and observing
Tips for Moderating and Observing test logistics

Act like you normally would

  • Treat them like you would any other colleague or friend

  • Ask first before assuming they need help

    Learn the AT

  • Practice listening to JAWS or other assistive technology while someone is talkingAsk first before assuming they need help

  • Know when the AT will affect how you interact with the participant

    Look for general usability problems as well as those made worse by poor accessibility or interaction with the AT

  • Missing information, problems navigating can reveal problems for everyone

  • Look for issues caused by the presentation


Summary testing with people with disabilities
Summary: Testing with People with Disabilities test logistics

Consider logistics, location, and setup

Allow more time for recruiting and between sessions

Do a dry run and practice observing


Learn more about usability testing
Learn more about usability testing test logistics

Handbook of Usability Testing by Jeff Rubin and Dana Chisnell

usabilitytestinghowto.blogspot.com

Rocket Surgery Made Easy by Steve Krug

www.sensible.com

www.peachpit.com/promotions/promotion.aspx?promo=137602

Rocket Surgery demonstration video

www.usability.govA guide to developing usable and useful web sites (from US Dept of HHS)

www.howto.gov/web-content/usability/first-fridays

First Fridays Product Testing Program


Whitney quesenbery wqusability com whitney@wqusability com 908 617 1122
Whitney test [email protected]1122

Kate Walser

CX Insights

[email protected]


ad