Olympic Message System

1. Olympic Message System John Kelleher Material Source: THE 1984 OLYMPIC MESSAGE SYSTEM: A TEST OF BEHAVIORAL PRINCIPLES OF SYSTEM DESIGN GOULD, STEPHEN J. BOIES, STEPHEN LEVY, JOHN T. RICHARDS, and JIM SCHOONARD

2. Overview • 1984 Olympic voice-mail system • 15 behavioural methologies employed • Three principles • Early focus on users • Empirical measurement • Iterative design

3. J. Gould Principles • Early focus on users and tasks This means first understanding who the users will be by directly studying their cognitive, behavioural, anthropomorphic, and attitudinal characteristics. This required observing users doing their normal tasks, studying the nature of those tasks, and then involving users in the design process • Empirical measurement Early in development, the reactions and performance of intended users to printed scenarios, manuals, etc. is observed and measured. Later on, uses interact with simulations and prototypes and their performance and reactions are observed, recorded and analysed. • Iterative Design When problems are found in user testing, they are fixed and then more tests and observations are carried out to see the effects of the fixes. This means that design and development is iterative, with cycles of ‘design, test, measure, and redesign’ being repeated as often as necessary.

4. Reactions to 3 principles • “They’re obvious – everybody says that” • Gould study1: 26% mentioned none; 35% mentioned only one • “Everybody does these things.” • should be process-oriented not mile-stone oriented • “Human factors is just fine-tuning.” • Gloss does not fix design defects – not peanut butter • “In real life, you can’t follow them.” • “You can’t measure usability” 1 Gould. J.D.. and Lewis. C. Designing for usability: Key principles and what designers think. Commun ACM 28, 3 (Mar. 1985). 300-311.

5. Background • High risk system • Short project schedule • 10,000 Olympic athletes and officials • across two campuses • extended family and friends (postcards provided) • 6 networked mainframes • 25 kiosks • 12 different languages supported (a first) • 24/7 system

6. Use of system • Kiosk housing: • CRT of Olympians with new messages • Push-button telephone • Entertaining videodisk of a mime giving instructions on use (4 min.) • Copies of OMS guide • People from around the world send messages to athletes (>18,000 sent) • Olympians send messages to each other • Olympians pick up their own messages (43,000 sign-ons)

7. How principles were followed • Printed scenarios • first definition of system function • tangible, identifiable usage of system available at a time when comments could have greatest impact • several features dropped (E.g. call verification) • later coding saved

8. How principles were followed • Early iterative tests of user guides • guides written before coding began • Olympians’ guide – 200 iterations • Families guide – 50 iterations • needed to minimise long distance call times • option to change language heard E.g. Canadians. • Discouraged late changes • proposals had to be proved to be better empirically

9. How principles were followed • Early simulations • Wizard of Oz prototyping on ‘simpler-to-program’ IBM VM System • Provided early feedback from users • Lab personnel and visitors proved poor judges of usability • Feedback helped form ‘prompt’ messages • Need for consistent ‘escapes’ identified • E.g. USA Olympian accidentally enters USSR

10. How principles were followed • Early Demonstrations • Prompted dropping of several functions to reduce audio prompting. • Olympians on the design team • Ghana ex-Olympian • Tours of Olympic Villages • Difficulty of access • Classroom training for Olympians rejected due to campus layout, climate and availability of large rooms

11. How principles were followed • Interviews with Olympians • Insight into their routine and time schedules • Overseas Tests of Family/Friends interface • Added an example to OMS guide • Hallway and Storefront Methodology • Refined dimensions of kiosk and documentation • E.g. Deutsch rather than German • Asked people to find their name on the list

12. How principles were followed • Yorktown Prototype test • 100 participants • Try-to-Destroy-it Tests • Pre-Olympic Field Test • Competitors from 65 countries • 57 usability issues arose • Embarrassed at exclusion of competitors from Oman, Columbia, Pakistan, Japan and Korea • Yorktown Final Prototype Test • 2800 participants

13. Sample Dialog User: (Dial 8540.) OMS: Olympic Message System. Please keypress your three-letter Olympic country code. Systeme de Message Olympique Tapez le numer de votre pays, s’il vous plait. User: USA OMS: United States. Les Etats-Unis. Please keypress your last name. User: goul OMS: John Gould. Please keypress your password. User: 319 OMS: Welcome to Olympic Message System. New Message from Stephen Boies.

14. Usability Findings 1/3 • Three-letter Olympic country code • Interruptible initial prompt not obvious • “Time-out” help messages tuned only for English. • Name problems • Middle/Far Eastern uses unsure of whether first or last name required • Solved by referring to Olympic ID badge • Indicated by change in all documentation

15. Usability Findings 2/3 • Name issues (contd.) • Alphabet translations often inaccurate for 5 languages • Referred to badge for accuracy • Spelling names of other Olympians • Added ‘reply’ option for messages received • Sign-on required only enough chars to distinguish from other Olympians from same country • ‘smart sponge’ programming • 80-90% of Olympians spoke 1 of 4 major languages • Testing indicated need for 12 languages to be supported

16. Usability Findings 3/3 • Passwords • Often did not know password • Solved by initially assigning last 3 digits of badge number as password • Some non-English-speaking users had difficulty answering some prompts by chars and others by digits • ‘We have to read the telephone keys differently’ • Typography changed to emphasise distinction • Upper case used and letters separated