What sells usability?

1. What sells usability? Nigel Bevan nigel.bevan@serco.com Serco Usability Services and University of York UK

2. Why is usability important? • The individual components of usability make business sense: • Effective: success in achieving goals • Efficient: productivity, staffing, waiting time in line • Satisfied: willingness to use the system • Risk: reduced business risk

3. Development costs can be reduced by: Producing a product that has only relevant functionality Detecting and fixing usability problems early in the development process Reducing the cost of future redesign or radical change of the architecture to make future versions of the product more usable Minimizing or eliminating the need for documentation Redesigning web sites to increase revenue, not just to change the image Reducing the risk of product failure E-commerce sales can be improved by increasing the number of web site customers who will: Be able to find products that they want Find supplementary information easily (e.g. delivery, return and warranty information) Be satisfied with the web site and make repeat purchases Trust the web site (with personal information and to operate correctly) Not require any support, or use the web site for support rather than calling the support center Recommend the site to others Support and increase sales by other channels Product sales can be increased as a result of the usability of the product: Improving the competitive edge by marketing the product or service as easy to use Increasing the number of customers satisfied with the product who will make repeat purchases and recommend the product to others Obtaining higher ratings for usability in product reviews Employers can benefit from easier to use systems in the following ways: Faster learning and better retention of information Reducing task time and increased productivity Reducing employee errors that have to be corrected later Reducing employee errors that impact on the quality of service Reducing staff turnover as a result of higher satisfaction and motivation Reducing time spent by other staff providing assistance when users encounter difficulties E. Suppliers and/or employers can benefit from reduced support and maintenance costs in the following ways: Reducing support and help line costs Reducing costs of training Reducing maintenance costs Potential cost benefits

4. So why is usability often ignored? • Believed to add to project risk • Additional time and cost • Outside area of experience • No established process for usability • Not specified in requirements • Not my responsibility • Customers don’t ask for it • Intangible • Every user wants something different

5. Barriers to uptake • What motivates an organisation (or project) to invest in usability? • The barriers, perceived: • Increased cost • Increased development time • Therefore increased risk?

6. Main motivator for investing in usability • The requirements/RFP mentions ease of use • What is the least we can do to address this? • This project has a problem with usability • Usability can help solve it: ie risk reduction • Senior usability convert • Usability is important to our business success • We realise that usability is important for success • Usually for consumer products and web sites

7. Selling the concept? • Usability can • improve your productivity • Improve your sales • Improve the quality of your product • Reduce the risk of product failure • Improve your process • Drive innovation (IBM) • Improve affecive/emotional design (Arnie) • Usability helps design products to match user needs (Arnie) • Usability is acknowledged good practice

8. The importance of user requirements • Standish Group found that 15% of projects failed 51% were partially successful • Main causes were poor user requirements: 13.1% Incomplete requirements 12.4% Lack of user involvement 10.6% Inadequate resources 9.9% Unrealistic user expectations 9.3% Lack of management support 8.7% Requirements keep changing 8.1% Inadequate planning 7.5% System no longer needed

9. One number for usability • Jeff Sauro has shown that it can be done • Sauro and Kindlund, UPA 2005 Making Sense of Usability Metrics: Usability and Six Sigma • Satisfaction questionnaire can provide a single number • But the weighting is context-dependent? • The individual components of usability make business sense: • Effective: success in achieving goals • Efficient: productivity, staffing, waiting time in line • Satisfied: willingness to use the system • Risk: reduced business risk

10. But, to quote Allen Cooper • If you were a plastic surgeon, would you go down the road knocking on doors, and telling the person who comes to the door: • “I can see you are ugly: but I am a plastic surgeon and I can help you!”

11. Tailor the message for the audience • No one size fits all? • Adapt the message to the audience? • But how do you know what the audience wants to hear? • Need to analyse the business environment and the motivators and constraints for the gatekeeper • Tailor the message to the context of use. What matters to the gate keeper? • Increased revenue? • Decreased costs? • Reduced risk of failure of the product? • None of these: no sell!

13. Essential prerequisites for usability

14. Human centred design process for interactive systems: ISO 13407 (1999) 1. Plan the human centred process Meets requirements 2. Specify the context of use 5. Evaluate designs against user requirements 3. Specify user and organisational requirements 4. Produce design solutions

15. 1.Stake-holder meeting 2. Context of use 3.Scenarios Essential activities to achieve usability PlanProcess SpecifyContext of Use SpecifyRequirements DesignSolutions Evaluate against Requirements System lifecycle feasibility requirements design implement release 4. Usability requirements 5. Evaluate existing system 10. Collect feedback 6. Prototyping 7. Style guide 8.Evaluation 9. Usability testing www.usabilitynet.org/trump/ucdmethods

16. ISO TR 18529 Human-centred lifecycle processes descriptions HCD.1 Ensure HCD content in system strategy HCD.2 Plan and manage the HCD process HCD.3 Specify the user and organisational requirements HCD.4 Understand and specify the context of use HCD.5 Produce design solutions HCD.6 Evaluate designs against requirements HCD.7 Introduce and operate the system Can be used to assess organisational capability and identify needs for process improvement

17. ISO TR: 18529 Human-centred lifecycle processes descriptions Understand and Specify User and Organizational Requirement and Context of Use 2.1 - Clarify and document system goals 2.2 - Analyze stakeholders 2.3 - Assess risk to stakeholders 2.4 - Identify, document and analyze the context of use 2.5 - Define the use of the system 2.6 - Generate the stakeholder, user and organizational requirements 2.7 - Set usability objectives

18. What is usability?

19. ISO/IEC 9126 Software quality characteristics 1991 • Functionality • Efficiency • Reliability • Usability • Maintainability • Portability

20. ISO/IEC 9126-1 Software Product Quality Model (1991) functionality reliability accuracy suitability interoperability security maturity fault tolerance recoverability availability efficiency usability understandability learnability operability attractiveness time behaviour resource utilisation maintainability portability adaptability installability co-existence replaceability analysability changeability stability testability

21. ISO 9241-11 Guidance on Usability (1998) Usability The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use Effectiveness : The accuracy and completeness with which users achieve specified goals. Efficiency : The resources expended in relation to the accuracy and completeness with which users achieve goals. Satisfaction : The comfort and acceptability of use

22. Usability model personalenvironment goals user user satisfaction interactiontasks physical environment output: effectiveness efficiency social and organisationalenvironment product technical environment

23. q u a l i t y i n u s e ISO/IEC 9126-1 Software Product Quality Model functionality reliability accuracy suitability interoperability security maturity fault tolerance recoverability availability efficiency usability understandability learnability operability attractiveness time behaviour resource utilisation maintainability portability adaptability installability co-existence replaceability analysability changeability stability testability

24. Quality in use • The extent to which a product used by specific users meets their needs to achieve specific goals with effectiveness, productivity, safety and satisfaction in specific contexts of use • Effectiveness • The extent to which users are able to achieve specified goals with accuracy and completeness in a specified context of use • Productivity • The extent to which users are able to expend appropriate amounts of resources in relation to the effectiveness achieved in a specified context of use • Satisfaction • The extent to which users are satisfied in a specified context of use • Safety/Risk • The extent to which risk of harm to people, business, software, property or the environment is acceptable in a specified context of use

25. How do you know a consumer product is usable? • Can’t tell from appearance in the shop • Difficult to tell from a demonstration • By the time you find out it is too late … • So no incentive for suppliers

26. Plan for Usability Assurance Statements 1990 • Planned that ISO 9241-11 would provide Usability Assurance Statements • Usability test results giving effectiveness, efficiency and satisfaction in a particular context of use

27. But not everyone wants their usability assured …

28. US computer company opposed usability assurance statements • The results could be misinterpreted • They did not wish to be obliged to produce such statements • US computer company around the world would lobby for vote no votes • So usability assurance statements were dropped

29. Common Industry Format for usability test reports • NIST initiative in 1998 • National Institute of Standards and Technology • Suppliers provide standard test reports to purchasers • Suppliers included: • IBM, Microsoft, HP, Sun, Oracle, Compaq • Purchasers included: • Boeing, Northwest Mutual Life, State Farm Insurance, Fidelity, Kodak • Reports provided in confidence • Could permit comparisons

30. Common Industry Format usability tests Objectives • Raise the profile of usability in procurement • Encourage suppliers to work more closely with purchasers to understand user needs • A common format for sharing usability data • Reduce uncontrolled overhead costs of usability problems • Enable purchasers to compare usability

31. CIF motivation • Boeing “We traditionally have had little visibility of how usable a product will be or how much training and support users will need. This has made it difficult to compare products, to plan for support, or estimate total cost of ownership.” • US WEST “US WEST has been actively participating in [the CIF] initiative and will clearly benefit from the results of this effort such as a standard testing process for usability, a standard specification for reporting usability tests, and other techniques to enable us to partner more effectively with our vendors.” • State Farm Insurance “We have found it difficult to identify software products that meet our needs without contributing to excessive overhead, increasing support costs, or negatively impacting employee productivity or morale. If successful the [CIF] initiative should result in the development of better, more usable software for all of industry.”

32. How it should work • Purchaser requests supplier to provide a CIF report • initially as part of agreed trials • The supplier may already be planning a usability test • may have to adapt the test plan to be suitable for CIF report or • The supplier asks the purchaser to carry out the test or • The supplier asks a third party to carry out the test • The test results help the purchaser decide • whether to purchase • whether to request changes

33. Report format - technical • Test objectives • Participants • Total number and user groups • Key characteristics and capabilities expected • How selected: with the key characteristics and capabilities? • Context of evaluation • Task scenarios and task data used • Any performance criteria used • Physical and computer environment

34. Report format - results • Design • Independent and control variables • Measures used • Any training and assistance given • Task instructions • Effectiveness • % completely and correctly achieving their goals • Separate data where assistance was given • Efficiency • Mean time taken to complete the task • Standard deviation (and standard error) • Satisfaction • Optional: own scales and/or standard questionnaire

35. Common Industry Format Standard • ANSI/NCITS 354 • ISO 25062

36. Why are so many consumer products difficult to use? • Difficult to assess usability when purchasing • Corporate purchasers want usability reported in the Common Industry Format • Consumers need a label or statement

37. ISO 20282: Ease of operation of everyday products Part 1: Context of use and user characteristics • Context of use • Are the main goals identified? • Are actions on other equipment influential? • Are environmental factors relevant? • Is privacy or the social environment relevant?

38. User characteristics • Psychological and social characteristics • Do cognitive abilities influence ease of operation? • Do knowledge, experience and habit influence ease of operation? • Do cultural differences influence ease of operation? • Does literacy influence ease of operation? • Does language influence ease of operation? • Physical and sensory • Are body dimensions relevant for ease of operation? • Are strength and biomechanical abilities relevant? • Are visual abilities relevant for ease of operation? • Are auditory abilities relevant for the ease of operation? • Does handedness influence the ease of operation? • Categories • Does age influence the ease of operation? • Does gender influence ease of operation?

40. ISO CD 20282-2: Test method • A summative test method for measuring usability based on • ISO 9241-11 (Guidance on usability) • Common Industry Format for usability test reports • Identify intended context of use and user groups • Measures • Success rate (ease of operation) • Task time (efficiency) • Satisfaction scale

41. ISO TS 20282-2: Test method for usability of everyday products • Measures • Success rate (effectiveness) • Task time (efficiency) • Satisfaction scale • Criteria • 80% of the intended users can successfully complete the main tasks without assistance

42. A manufacturer can use ISO 20282-2 to provide evidence of the ease of operation of a product for marketing purposes A manufacturer wants to demonstrate that a videorecorder is very easy to program without instructions. A stratified sample of 50 people who have purchased a videorecorder with the intention of programming it to record programmes are recruited who are representative in age, education and ownership of brands of videorecorder owners. Each session is expected to take a total of 5-10 minutes, and 2 videorecorders are set up with TVs for programming. Each person is given asked to program the videorecorder to record a specified programme in a TV magazine. It is expected that most people will use the Videoplus code, but direct programming on the channel and time is also acceptable. The test result is given as the measured success rate together with a confidence interval. To be 95% confident that 80% of the user population can program the videorecorder, 45 of the 50 people tested will have to be successful.

43. Not everyone wants their usability assured (again) …

44. German computer company opposes the standard • “The results could be misinterpreted” • “We do not wish to be obliged to carry out such tests” • “Remove all reference to consumer products” • Lobbied for for industry association to vote no • Tried to raise procedural obstacles to prevent the standard progressing

45. We nearly forgot about requirements!

46. CIF-R Usability requirements Common Industry Format Supplier Consumer Usability requirements Usability test report CIF Purchase decision

47. CIF Requirements • Context of use • Users • Key characteristics and capabilities of each user group for which requirements are provided • Goals • Main goals for each group with scenarios of use • Equipment: technical environment • Computer hardware and software • Physical and social environments • Scenarios of use for most important goals • Training scenarios • Usability measures • Effectiveness: unassisted completion rate • Efficiency: mean time to achieve goals • Satisfaction: mean score on a satisfaction scale • Relative user efficiency (optional)

48. CIF Requirements • Context of use • Users • Key characteristics and capabilities of each user group for which requirements are provided • Goals • Main goals for each group with scenarios of use • Equipment: technical environment • Computer hardware and software • Physical and social environments • Scenarios of use for most important goals • Training scenarios • Usability measures • Effectiveness: unassisted completion rate • Efficiency: mean time to achieve goals • Satisfaction: mean score on a satisfaction scale • Relative user efficiency (optional)

49. Example: Usability measures • Task 1: Download SITE guard software • Effectiveness: 90% unassisted task completion rate. • Efficiency: 10 minutes of user time. • Satisfaction: Total SUMI score of 50. • Task 2: Burn SITE guard software to DVD and launch installer • Effectiveness: 90% unassisted task completion rate. • Efficiency: 15 minutes of user time. • Satisfaction: Total SUMI score of 50. • Task 3: Prepare SITE guard software for staged install and launch installer • Effectiveness: 70% unassisted task completion rate. • Efficiency: 25 minutes of user time. • Satisfaction: Total SUS or SUMI score of 50.

50. What can YOU do about it?Usability Professionals Association • World Usability Day • 14 November 2006 • Usability Body of Knowledge