Insert the title of your presentation here

1. Exploring measures of usability for in-vehicle technology Insert the title of your presentation here Presented by Name HereJob Title - Date Presented by Alistair WeareHF Researcher at TRL

2. Agenda 1 Introduction: What is usability? 2 Design Guidelines 3 Methods related to usability measurement 4 Conclusions 5 Questions / Feedback Page 2

3. Introduction: What is usability? Page 3 • Usability is an accepted concept, but not rigidly refined • Nielsen (1993) – 5 key usability attributes: Learnability, Efficiency, Memorability, Errors, Satisfaction • ISO 9241: ‘The extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use’ • Jordan (1998) – 5 higher-order components of usability: Guessability, Learnability, Experienced User Potential (EUP), System potential, Re-usability (or memorability) • ISO 17287 defined a new and related concept - ‘suitability’ - as: ‘The degree to which a [system] is appropriate in the context of the driving environment based on compatibility with the primary driving task’

4. Introduction: What is usability? • It must be useful to the driver within the higher driving task, efficient such that it presents a minimal distraction, and its ease of use must be compatible with any competing demands on the driver at the time of use (which may or may not be when driving) Page 4 In determining the usability of an IVIS (whether designed specifically for use in a vehicle or not), there must be an understanding of how the system fits into the larger vehicle-driver-road system

5. Design Guidance Sources Regulations and Standards International standards • Not legally binding, but form a framework for a common design philosophy • Standards attempt to define best practice and so may form basis for mandatory regulations • Play an important role, but only if kept up to date United States regulations • laws about in-vehicle distraction generally fall under the jurisdiction of individual states but with some at the national (federal) level • As an example of national provision, in October 2009 President Obama issued an Executive Order prohibiting Federal employees from texting while driving European regulations • There is currently little in the way of European legislation specifically related to the HMI of IVIS • European regulations may be a consideration in the future Page 5

6. Design Guidance Sources Design Guidelines Europe: European statement of principles • High-level HMI design guidance with status as recommended code of practice for use in Europe • 34 design principles, grouped into: Overall Design Principles, Installation Principles, Information Principles, Interactions with Controls and Displays Principles, System Behaviour Principles and Information about the System Principles United States: Alliance and NHTSA • The US motor vehicle manufacturers have developed ‘Alliance Guidelines’ that cover similar, high-level, design principles to the ESoP • NHTSA guidelines are based on the ESoP/Alliance guidelines and introduce some specific assessment procedures • Provides a more definitive assessment but does rely on the testing of participants to determine levels of distraction (sampling issues) Japan: JAMA • 25 specific requirements that apply to driver interfaces to ensure safe operation whilst driving • Requirements are grouped into: Installation of Display Systems, Functions of Display Systems, Display System Operation While Vehicle in Motion, and Presentation of Information to Users Page 6

7. Design Guidance Sources Warning Guidelines Page 7 • Some international work to standardise warning signals • One technical specification on prioritization of warning signals: • ISO/TS 16951: Road Vehicles – Ergonomic aspects of transport information and control systems – Procedures for determining priority of on-board messages presented to drivers • Two technical reports containing guidance information: • ISO/PDTR 16352: Road Vehicles – Ergonomic aspects of transport information and control systems – MMI of warning systems in vehicles • ISO/PDTR 12204: Road Vehicles – Ergonomic aspects of transport information and control systems – Introduction to integrating safety critical and time critical warning signals

8. Design Guidelines Driver Assistance Systems Guidelines Page 8 • Key issue relating to ADAS is driver controllability, determined by: • by the possibility and driver’s capability to perceive the criticality of a situation • the driver’s capability to decide on appropriate countermeasures (such as overriding or switching off the system) • the driver’s ability to perform any chosen countermeasures (taking account of the driver’s reaction time, sensory-motor speed and accuracy) • Controllability must exist at all levels of interaction: • during normal use within system limits • at and beyond system limits • during and after system failures

9. Design Guidelines Driver Assistance Systems Guidelines Page 9 • Two key European projects relevant to controllability: • RESPONSE: • Developed a Code of Practice for defining, designing and validating ADAS • The Code describes current procedures used by the vehicle industry to develop safe ADAS with particular emphasis on the human factors requirements for ‘controllability’ • ADVISORS: • Attempted to integrate the RESPONSE Code within a wider framework of user-centreddesign taking account of the usability of information, warning and assistance systems • (IHRA-ITS) • There is also activity by the International Harmonized Research Activities – Intelligent Transport Systems Working Group to develop a set of high-level principles for the design of driver assistance systems

10. Methods related to usability measurement Methods for assessing IVIS usability can take two different approaches: Evaluate actual driver performance when using the product/system in a realistic context of use Depends on: Which environment the method is used in (road, test track, simulator, laboratory etc.) Which task manipulations occur (multiple task, single task loading, no tasks given etc.) Which dependent variables (operationalised as metrics) are of interest. Evaluate how well a product/system meets the design principles in the relevant guidelines Page 10

11. Methods related to usability measurement Available methods for evaluating actual driver performance: Road trials Simulator trials Occlusion • Participants drive a real vehicle • Wide range of metrics • Usually a complex set of obderved behaviours • Carefully controlled simulated environment • Specific objective measures • Key benefit is experimental repeatability • Focus on visual demand of IVS • Metrics include: time taken to complete task, number of glances and number of errors Peripheral detection Lane-change task Keystroke Level Model • Often used as part of sim study • Participants respond to changes in periphery • Speed and accuracy measured • Standardised method using sim environment • Participant manoeuvres compared to a normative model • Desk-based • Component operations of a task are timed • Enables predictions of overall task times Page 11

12. Methods related to usability measurement Methods for assessing IVIS usability can take two different approaches: Evaluate actual driver performance when using the product/system in a realistic context of use Evaluate how well a product/system meets the design principles in the relevant guidelines • Possible to develop checklists based on the design principles laid down in the guidelines (e.g. ESoP, NHTSA and JAMA guidelines). • To this end a functional IVIS usability checklist has been developed, based on an existing checklist produced for the UK Government in the late 1990’s, and incorporates requirements taken from the ESoP. Page 12

13. Excel Spreadsheet for PC, tablet etc. Supportive Information for each question is available via a ‘help’ icon Assessment Summary Sheet is automatically populated based on the data entered Methods related to usability measurement IVIS usability checklist Page 13

14. Methods related to usability measurement IVIS usability checklist: Issues / considerations Page 14 • Questions need to be sufficiently ‘elemental’ • Three Checklist questions require further work on measurements and evaluation criteria: • Is the IVIS securely fitted? • Is the IVIS visual display positioned close to the driver’s normal line of sight? • Are presented messages visually simple? • Potential to include a rating system, whereby differing systems can be compared in terms of their overall usability • However, five key difficult issues hindering this:

15. Methods related to usability measurement IVIS usability checklist: Issues / considerations Page 15 • Elements- Which elements are included within the rating? (E.g. all Checklist questions) • Scoring* - How are the individual elements scored? (E.g. +3/0/-3 or 1-10) • *Also, is it “better” to have a feature, even if poorly designed, than for that feature to be absent; and how should the scoring reflect this? • Weighting- How are the individual elements weighted? (E.g. all even, high and low weights, individual weights) • Combining- How are the scores and weights combined? • Rating- How is the final number converted into the consumer rating?

16. Conclusions Page 16 Guidelines and checklists are helpful design aids Usability is really important and this means usability within the context of use HMI technology is always developing Specific quantitative requirements are more likely to stifle innovation than promote safety Safety is only partly about design – it’s mostly about driver behaviour

17. Alistair WeareHuman Factors Researcher Tel: (+44)1344 770901Email: aweare@trl.co.uk Page 17