Deciding How to Measure Usability
Download
1 / 36

Deciding How to Measure Usability How to conduct successful user requirements activity? - PowerPoint PPT Presentation


  • 50 Views
  • Uploaded on

Deciding How to Measure Usability How to conduct successful user requirements activity?. Deciding How to Measure Usability. Understanding what you can measure Matching Measures to your Goals and concerns Matching Measures to the Product's Stage of Development

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Deciding How to Measure Usability How to conduct successful user requirements activity?' - kelli


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript


Deciding how to measure usability
Deciding How to Measure Usability

  • Understanding what you can measure

  • Matching Measures to your Goals and concerns

  • Matching Measures to the Product's Stage of Development

  • Setting Quantitative Criteria for Each measure and each Task


Understanding what you can measure
Understanding what you can measure

In a usability test you measure both Performance measures and Subjective measures


Understanding what you can measure1
Understanding what you can measure

  • Performance Measures: count of behaviors or actions you can see.

    Quantitative

    e.g. how many errors people make and how many times they repeat the same error


Understanding what you can measure2
Understanding what you can measure

  • Subjective Measures: people's perceptions, opinions and judgments.

    either quantitative or qualitative

    e.g. give a people 5 or 7 point scale and ask them to rate difficulty in using a product.


Deciding how to measure usability1
Deciding How to Measure Usability

  • Understanding what you can measure

  • Matching Measures to your Goals and concerns

  • Matching Measures to the Product's Stage of Development

  • Setting Quantitative Criteria for Each measure and each Task


Deciding how to measure usability2
Deciding How to Measure Usability

  • Understanding what you can measure

  • Matching Measures to your Goals and concerns

  • Matching Measures to the Product's Stage of Development

  • Setting Quantitative Criteria for Each measure and each Task


Matching measures to your goals and concerns
Matching Measures to your Goals and Concerns

Performance measures chosen should be directly related to quantitative usability goals and concerns behind the usability test

How to collect performance data?

Data Logging Software


S... Stop the test B Take a Break

T... Stop the task A Assist

M Menu Error H Help desk

S Select from list error F Frustration

E Other Error N Observations

Event: O 06:21:33

Comment: Looking for online help for To: field


Points to be considered in building data logging program
Points to be considered in building Data Logging Program

  • Include preset codes for events that happen in every test.

  • Code should be small

  • Short description with code

  • Timestamp every event

  • Allow easy backup and movement of data into a file

    What if you don't have a data logging software?

    Should you measure positive behavior?


Deciding how to measure usability3
Deciding How to Measure Usability

  • Understanding what you can measure

  • Matching Measures to your Goals and concerns

  • Matching Measures to the Product's Stage of Development

  • Setting Quantitative Criteria for Each measure and each Task


Matching measures to product s stage of development
Matching Measures to Product's stage of development

Very important to consider where product is in the development cycle while planning performance measures.

  • Testing prototype for a manual (without index)


Deciding how to measure usability4
Deciding How to Measure Usability

  • Understanding what you can measure

  • Matching Measures to your Goals and concerns

  • Matching Measures to the Product's Stage of Development

  • Setting Quantitative Criteria for Each measure and each Task

    Choose the performance measures you want to count


Selecting performance measures
Selecting Performance Measures

General Concerns:

  • Ease of users who have never used email

  • Ease of users who have used other email

  • Will the online help be useful

  • Will new user be able to select items from screens quickly and easily


Selecting performance measures1
Selecting Performance Measures

Specific Concerns:

  • Will the user be able to read a specific piece of mail and skip over mail they don't want to read

  • Will new users be able to find and select people's addresses to send them mail

  • Will users be able to find the right menu path to read/write/send a message


Setting quantitative measure for each measure and task
Setting Quantitative measure for Each Measure and Task

How to select criteria for performance measures?

Typical criteria for Performance measures

  • Excellent

  • Acceptable or OK

  • Unacceptable

    Base your criteria on users



Are the measures the same for all tasks in a given test?

No

Are the criteria of performance same for all tasks?

No


Do you take the test situation into account in selecting criteria?

Yes

Should you count system response time in setting the criteria?

Yes


What we have discussed till now
What we have discussed till now criteria?

Tasks that participants will do during tests and how to measure participant's performance with products.



How to conduct successful user requirements activity
How to conduct successful user requirements activity? criteria?

  • Welcoming your participants

  • Dealing with late and absent participants

  • Warm up exercises

  • Inviting Observers

  • Introducing your think aloud protocol

  • Moderating your activity

  • Recording and note taking

  • Dealing with awkward situations

  • Conclusion


Welcoming your participants
Welcoming your participants criteria?

  • Ask participants to come about 15 minutes earlier.

  • Introduction.

  • Welcome Signs.

  • Playing CDs.

    Don't leave participants alone


Dealing with late and absent participants
Dealing with late and absent participants criteria?

Despite your best efforts some participants can be late.

  • The late participant

  • You can't wait any longer

  • Including late participants

  • The No-Show


Warm up exercises
Warm-up Exercises criteria?

  • Start with light conversation.

  • Introduce yourself.

  • Provide nametags

    Don't spend much time


Inviting observers
Inviting Observers criteria?

  • Developer observers get to know user requirements better.

  • Tell observers to come early and remain quiet.

    Don't allow managers to observe


Introducing your think aloud protocol
Introducing your think aloud Protocol criteria?

Make participants speak

  • Their thought process while doing task .

  • Steps in the task.

  • Expectations and evaluation statements.

    Provide some examples to participants

    e.g. Stapler

    Observe loudness of participants


Moderating your activity
Moderating your activity criteria?

  • Have personality

  • Ask questions

  • Stay focused

  • You are not a participant

  • Keep activity moving

  • No critiquing

  • Everyone should participate

  • No-one should dominate

  • Practice makes perfect


Take notes
Take Notes criteria?

  • You get data immediately and can start analysis

  • Participants feel they are saying important things

    Problems:

  • You can get wrapped being a stenographer

  • If discussion pace is fast all information cannot be captured


Video and audio recording
Video and Audio Recording criteria?

  • Capture nuances, body language of participants (cant have in notes)

  • Listen to recording and take notes

  • Video better than audio recording

    Problems:

  • Can make participants uncomfortable


What is the best approach? criteria?

Combine Video/Audio Recording and Notes


Dealing with awkward situations
Dealing with awkward Situations criteria?

Some Uncomfortable situations and ways to deal with them :

  • Participant Issues

  • Observers Issues


Participants issues
Participants Issues criteria?

  • Participant is called in middle of test

  • Participant's cell phone rings continuously

  • Wrong participant recruited

  • Participant thinks he is on a job interview

  • Participant refuses to be videotaped and wants to leave

    Contd..


Participants issues1
Participants Issues criteria?

  • Participant is confrontational with other participants

  • Participant dominates the group

  • Participant is not truthful about his identity

  • Participant refuses to sign consent form

  • Fire Alarm sounds in middle of test and participant still wants to continue


Product team observers issues
Product Team/Observers Issues criteria?

  • Team changes the product mid-test

  • An observer turns on light in control room

  • Observers talk loudly during an activity


Conclusion
Conclusion criteria?

To conduct any of your user requirements effectively

  • Deal with participant arrivals.

  • Make them think creatively.

  • Moderating any activity

  • Instruct your participants to think aloud.

  • Deal with any awkward situation in testing that may arise.


ad