Chapter 15: Design and Evaluation in the Real World: Communicators and Advisory Systems

1. Chapter 15: Design and Evaluation in the Real World: Communicators and Advisory Systems Group 4: Tony Masi, Sam Esswein, Chris Troisi, Brian Rood

2. Introduction Text books make design and usability testing processes sound straightforward and step-by-step. This is never the case in real world applications. It is only when you become involved in the actual design project that the pressure, trade-offs, and demands influence the way the design project is carried out. In the real world, design and evaluation are VERY closely integrated. You don’t do one without the other.

3. Key Concepts • Show how design and evaluation are brought together in development of interactive products • Show how different combinations of design and evaluation methods are used in practice • Describe the various design trade-offs and decisions made in real-world situations

4. Key Issues • The number of iterations through the design-evaluate cycle depend on the requirements of the project • Many practical issues and unexpected events must be dealt with by the design team • No two projects are ever exactly the same; each will face a different set of constraints, demands, and crises

5. Real World Usability Cases http://www.asktog.com/columns/042ButterflyBallot.html - Florida Butterfly Ballot http://www.digitwireless.com/ - FasTap Keypad http://www.consult-me.co.uk/csc-case-studies.htm - Some other usability case studies

6. Designing Mobile Communicators • Nokia • Communicator for adults • Philips • Communicator for children

7. Mobile communicators are devices that commonly push the limits of technology. They combine many functions such as those listed to the right into one small device. Therefore, a key challenge is how to make these devices useable and affordable to most people. Send and receive telephone calls, email, faxes, and other messages Keep contact info, journal entries, calendars, and other notes Watch stock and other finance reports Background

8. Background Other considerations: • Device should be small and lightweight • Made of light materials • Small enough to fit in pocket or small bag • Software must work with limited screen size and memory • What will user be doing while using the device? • Device should work in all conditions • Various noise levels • Various lightings • Device should take little effort to operate so user can concentrate on other things at the same time • like driving? • Hands free mode? • Simple operations should be one-handed • Answer call • Browse internet

9. Background Other considerations: • Device should account for distractions that may occur • Interaction sequences should be easy to return to and continue after interruption • Internet trade off: How long should the device remain connected? • Tasks tend to be time-critical, triggered by other people or events, relatively brief, low in terms of attention to be applied to the task, and very personal • Flow among tasks should be smooth • Easy flow between related functions such as contact list and telephone function • Must be simple and not involve much training (if any) • Needs to be robust and reliable

10. Designing Nokia’s Mobile Communicator • What kind of lifecycle? • Iterative user-centered approach • Top level design concept cycle (Fig 15.2) • Which methods to use? • Ethnographic research • Scenarios and task models (Fig 15.3, 15.4) • Confidential product issues: • First in the market is key • Evaluation must be very limited • No real users

11. Designing Nokia’s Mobile Communicator • Physical aspects: • Screen size • Number of buttons versus functionality • Soft keys with changing functions or values • Hard coded keys that always return same value • Consistency issues • Internal consistency (within mobile software) • External consistency (with desktop software) • User testing • None before release (confidentiality) • Summative testing & questionnaires after

12. Designing Philips’ communicator for children • design cycle:iterative and evolutionary • which methods:low-fidelity prototyping participatory designinterface metaphors • physical aspects:color, shape, size, robustnesspen input bags to protect screen

13. Designing Philips’ communicator for children • user involvement:children involved throughoutprototypes evaluated constantlyinvaluable insights for the designers • lessons learned:agree on assumptions in requirementsthink of follow-on projects early onusers are not designersact quick and dirty if necessary

14. Case study 2: A telephone response information system (TRIS) • Interactive voice response systems are common in government offices and large companies. Do you know of examples that you have used? • Why are these systems often so frustrating to use?Forming a mental model is difficult because there is no visual feedback and the user must remember the menu structure • Many menus and deep menus are particularly difficult

15. Why was TRIS difficult to use? • Having to remember the menu structure. • The programmers traded computational elegance for usability, e.g., the system asked for social security number and employee identification number, confusing users who did not have both. • TRIS was comprised of different systems each with its own interaction style. Users were not told this but when they moved between the systems they experienced sudden, unexplained changes.

16. How was TRIS evaluated? • A combination of techniques were used:- a review of the literature provided information about problems with interactive voice response systems- expert reviews - GOMS analysis of the proposed redesign • The redesign was implemented- usability tests confirmed that the redesigned system offered better usability than the original design

17. Why was using different methods valuable? • The evaluators were able to build-up a broad picture of usability problems. • Using GOMS and heuristic evaluation they could explore the potential benefits of the redesigned system. • User testing enabled them to confirm that the redesigned system offered better usability. • User satisfaction questionnaires confirmed that users preferred the redesigned system.

18. Key points • Design involves trade-offs • Design space for making changes when upgrading a product is limited • Cycles of rapid prototyping and evaluation allow designers to examine alternatives • Simulations are useful when evaluating systems used by large numbers of people • Piecing together evidence from a variety of sources can be valuable