Design and Evaluation: A case study

1. Design and Evaluation: A case study HCI1 lecture 11

2. Aims of Today’s Lecture • To see interactive systems design being done • To reflect on the process of design • To explore some of the issues that arise during design • To tell the story of the design of a Home Information Centre (HIC) • European funded research project • 1997 - 1999 (which seems old, but all the issues are still current)

3. Design • David Kelly (IDEO) • Design is messy; designers try to understand this mess. • They observe how their products will be used; design is about users and use. • They visualize which is the act of deciding what it is’.

4. Our view of design - The Star Method Requirements - finding out people need from the system Evaluation - checking that you’ve got it right Conceptual design - creating the overall idea Physical design - filling in the details Prototyping and Envisionment - bringing ideas to life

5. The Home Information Centre (HIC) • There are two typical situations at home • The ‘lean back’ situation of TV, video and DVD • The ‘lean forward’ situation of work and the PC • Aim of the HIC • To provide a move around situation for ‘infotainment’ • Project was to look at design concepts for internationally famous producer of high quality video, TV and CD players • One question concerns convergence or divergence? • Where is technology heading? • Infomercial, edutainment - converging services • Technology convergence? - but packing too much in one device

6. Early on in the project a hardware prototype was produced The prototype HIC

7. Abstract representation of the information contents of information sources Speech, pen, touch, keyboard as input Sound, images, text and animation as output Speech recognition Natural language queries An ‘intuitive’ user interface HIC - Key features

8. Arnold is busy in the kitchen, washing up the dishes from the party he gave yesterday evening. He listens to the morning news in his loudspeakers, connected to his HIC in the kitchen. He is going to be at work within an hour, and the speaker was saying something about a roadblock on the road between his home and his work. He turns his attention to the HIC saying, “Turn on”. He could have used touch or the keyboard as well, but his is occupied with his hands right now. A Scenario for the HIC

9. Can a computer recognize speech? Can a computer wreck a nice beach? Tricky Speech recognition systems are still very poor and unacceptable from a user perspective Speaker-dependent systems are OK but require training Speaker-independent systems can be as poor as 40% accurate Dealing with background noise - is a real issue, hence phone systems are the most successful You have to get the computer’s attention (except in particular situations, e.g. Star Trek and the Turbo-lift) Issue 1 - Speech recognition

10. Scenario continued • The screen now gives Arnold some possibilities. Arnold has earlier on defined his personal profile in the HIC, including details of the traffic situation between his home and his work, so one of the options given is service number 5 “traffic”. He could say 5, but he chooses to say “traffic situation” • He is now given some more possibilities, and says “local” number 2. The screen now shows the block on the main road, and he realises that he has to leave 10 minutes earlier, because he has to take another road.

11. Issues 2 • Personal profile • How is this defined? • “… screen shows block on the main road” • How? • Where does data come from? • What form is it in? • Who provides this data?

12. State-of-the-art in personalizing design - My Yahoo

15. HIC Scenario continued • Jane is woken by the sound of his car, and when she comes down to the kitchen, she sees a flashing note indicator on the HIC. She says, “turn on” and she sees the message from Arnold attached with the local traffic map, …. • Finished cleaning up and walking the dog, she goes to the HIC, rolls out the keyboard and writes: “When will the road be cleared for traffic?” The HIC, intelligent and flexible as it is, automatically will know that she is referring to the roadblock on the local road, due to Arnold's request early on in the morning, displays and says “13:30”. • Can we do this? What is involved?

16. HIC - Summary • We have a conceptual design for the HIC as a device that is suitable for a ‘move around situation for infotainment’ • We have a physical prototype; the ‘lectern’ • We have some requirements • Speech recognition • Personal profiling • Natural language queries • How to approach the design from here?

17. Design approach for HIC - Highly iterative, lots of evaluation. • Prototype P0 should be delivered by the end of month 6, and was to be used for the very first ‘discount evaluation’ experiments with the HIC system to aid in the design specification. P0 would not contain software, but presents a first design of appearance and visualisation. • Prototype P1 is delivered by the end of month 13, … An operator at the PC (hidden to the user) will start computer actions requested orally by the user, but not yet handled by the software (the so-called ‘wizard-of-Oz’ approach). • Prototype P2 is delivered at the end of month 19 and will be used for full laboratory experiments at first and then in real user homes. • Prototype P3 is the final official version of the HIC, usable for demonstrations and for the possible basis of further exploitation.

18. PACT analysis for HIC • People. We identified a number of personas representing different groups; wealthy middle class couple in their thirties; older person, child, etc. • Activities. We identified three main activity types • Information • Communication • Entertainment • Contexts were home based; variety of locations (e.g. rural versus city) • Technologies - as prescribed by client in the early requirements, but Content became a key issue.

19. Content • Devices have to have appropriate content to be useful. • Where does content come from? • Download it from the web. • But how do you know about the quality of the content? • How can you search and find what you want when there is so much rubbish? How do you know what content there is? • Think about magazines. Which ones do you buy? Why? • You choose a magazine because it has the content that you enjoy. You trust the editors to know what is going on and to provide you with the most relevant. • We expect a HIC to attract content providers who need to make money about of providing good quality content. • Think of service and content suppliers e.g. iTunes.

20. Break Be back in ten minutes

21. Our Approach to Interactive Systems Design • The Star Method - evaluation as central…. Highly iterative, develop and evaluate prototypes, being human-centred. • PACT analysis to scope the issues • Developing personas to represent the archetypal people • Developing scenarios that describe the activities that the people are doing, the contexts in which they are working and the technologies they are using • Scenarios can be used throughout the design process.

22. Scenarios throughout design

23. General scenarios for HIC • Scenario 1 - ‘what shall we do now?’ • A group (not necessarily co-located) of people are trying to decide what to do with some spare time, they want to find out what options for activities are open to them, how much they would cost, whether they can get there in time, etc. • A number of versions of this scenario can be developed which stay within this broad framework – • e.g. we want to go skiing this weekend, • is there anything on at a local pub tonight, • what’s on at that children’s festival and what kind of reviews have the various events had, etc?

24. More scenarios for HIC • Scenario 2 - ‘what shall we have for dinner?’ • Someone in the kitchen trying to decide what to eat for dinner. Variations might include looking for a recipe and then trying to find out where the ingredients can be purchased, or finding a recipe with a given list of ingredients (i.e. what’s in the fridge at that moment). • Scenario 3 - ‘entertain me’ • A more casual kind of HIC use, someone who has a few minutes to kill so perhaps wants to play a game like cards, or a kid who has decided they want to know how many castles there are in the country and when they were built, or someone who wants the answer to a quiz question.

25. Brainstorming. One method is the ‘future workshop’ • Stage 1, Critique – a group brainstorming session that tries to different approaches to the activities/problems being addressed by the proposed system. A set of themes arising out of this stage is used in stage two, Fantasy. • Stage 2, Fantasy – the emphasis here is on creative solutions/ideas, no matter how unrealistic. From these a number of fantasy themes are generated and these are then used to drive stage 3. • Stage 3, Implementation – the groups work out what would be necessary in order to produce a real solution to some of the fantasy themes generated (e.g. by producing a mock-up or a storyboard).

26. Developing ideas through group work and lots of stimuli

27. Developing paper prototypes; sketches of ideas

28. Developing 3D prototypes using modeling clay

29. A 3D model of a remote control for the HIC

30. Main findings from the Future Workshop

31. Concrete scenarios • Concrete scenarios are developed from the conceptual ones by specifying the particular people, technologies and contexts of the interaction - and include real content. • Use concrete scenarios for evaluation of designs • Organize the concrete scenarios • Identification - who wrote it, when, revisions, title, etc. • Provide summary PACT analysis for scenario • Number paragraphs for ease of reference • Make footnotes where design decisions need to be made • Develop scenarios that cover the ‘dimensions’ of the domain • Different media, contexts, etc. • Include example data where possible.

32. ‘Discount’ evaluation • Discount Evaluation is a ‘quick and dirty’ method of evaluation for when you do not have much time. • We used a discount evaluation on some early physical prototypes of the HIC • Develop visual versions of the concrete scenarios “envisionment” • Get people to walk through scenarios • Get them to think aloud about the design • Expert watches them and criticizes design • Using usability guidelines - “heuristics”. • It is good cheap method of evaluation early on in design process • Picks up major usability issues • Helps to formulate general design features

33. Design Heuristics i.e. Guidelines for evaluation • Learnability • Can people guess what system is likely to do? Does system conform to guidelines of visibility, consistency, affordance and familiarity? • Effectiveness • Can people correct things when they go wrong? Can they do what they want easily, effectively and safely? Does system conform to guidelines of navigation, control, feedback, recovery and constraints? • Accommodating • Is the system accommodating of people, their differences and preferences? Does system conform to guidelines of flexibility, style and conviviality?

34. HIC Prototype 1- Entertain me scenario • Learnability - looks like a media player, but what does ‘define’ mean? • Accommodating - flexibility is not good, no obvious signs of personalization ability. • Effectiveness - what about ‘undo’ options • Conclusions - does not appear to deal with large scale complexity of the HIC

35. HIC Prototype 2 - What shall we do now? • Learnability - easy to follow; had rollover explanations. History bar at top is good • Accommodating - more than one way of completing task, access previous levels = OK. Near/far button interesting • Effectiveness - seems good in terms of control and feedback • Conclusions - generated a lot of ideas. To be investigated further

36. HIC Prototype 3 - what shall we have for dinner? • Learnability - simple to use, initial 4 categories are clear, hierarchical navigation clear • Accommodating - flexibility of search, but search and history not joined up • Effectiveness - does not deal with back, undo etc. • Conclusions - history/navigation deserved a further look. Colour co-ordination is good.

37. Evaluation outcome

38. Further developments

39. HIC - developing the design • Colour coded category bar at top • Generic actions down left hand side • Information providers down right hand side • Main objects at bottom • Navigation ‘compass’ at bottom

40. History of searches shown as miniature icons Category bar at top spins to reveal sub-categories

41. Information categories Content providers HIC Prototype 2 Actions Search results

42. Conclusions • The Design of the Home Information Centre (HIC) provides a good case study of: • Our design approach in action • Some issues of new technologies • Some issues of interactive systems design • Evaluation is central to human-centred interactive systems design • Evaluation based on developing scenarios and envisioning them using expert critique is effective - particularly in early stages of design • Expert inspects a design a critiques it using heuristics • Learnability • Accommodation • Effectiveness • Prototypes can be used to incrementally develop ideas