Deciding How to Measure Usability How to conduct successful user requirements activity?. Deciding How to Measure Usability. Understanding what you can measure Matching Measures to your Goals and concerns Matching Measures to the Product's Stage of Development
In a usability test you measure both Performance measures and Subjective measures
e.g. how many errors people make and how many times they repeat the same error
either quantitative or qualitative
e.g. give a people 5 or 7 point scale and ask them to rate difficulty in using a product.
Performance measures chosen should be directly related to quantitative usability goals and concerns behind the usability test
How to collect performance data?
Data Logging Software
S... Stop the testB Take a Break
T... Stop the taskA Assist
M Menu ErrorH Help desk
S Select from list error F Frustration
E Other ErrorN Observations
Event: O 06:21:33
Comment: Looking for online help for To: field
What if you don't have a data logging software?
Should you measure positive behavior?
Very important to consider where product is in the development cycle while planning performance measures.
Choose the performance measures you want to count
How to select criteria for performance measures?
Typical criteria for Performance measures
Base your criteria on users
Are the measures the same for all tasks in a given test?
Are the criteria of performance same for all tasks?
Do you take the test situation into account in selecting criteria?
Should you count system response time in setting the criteria?
Tasks that participants will do during tests and how to measure participant's performance with products.
Don't leave participants alone
Despite your best efforts some participants can be late.
Don't spend much time
Don't allow managers to observe
Make participants speak
Provide some examples to participants
Observe loudness of participants
What is the best approach?
Combine Video/Audio Recording and Notes
Some Uncomfortable situations and ways to deal with them :
To conduct any of your user requirements effectively