1 / 20

Jeffrey Nichols Carnegie Mellon University October 15, 2002

Requirements for Automatically Generating Multi-Modal Interfaces for Complex Appliances The Fourth International Conference on Multi-Modal Interfaces (ICMI) 2002 Pittsburgh, PA. Jeffrey Nichols Carnegie Mellon University October 15, 2002. The Problem. Appliances are too complex.

gordon
Download Presentation

Jeffrey Nichols Carnegie Mellon University October 15, 2002

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Requirements for Automatically Generating Multi-Modal Interfaces for Complex AppliancesThe Fourth International Conference on Multi-Modal Interfaces (ICMI) 2002 Pittsburgh, PA Jeffrey Nichols Carnegie Mellon University October 15, 2002

  2. The Problem Appliances are too complex

  3. The Problem, cont. Each complex appliances has its own idiosyncratic interface! • Stereo systems • Telephones • VCRs • Hotel Alarm Clocks • … Increasingly computerized Low Usability Inaccessible to disabled users

  4. Feedback Specifications Control Our Solution Separate the interface from the appliance! Handheld becomes personal universal controller (PUC) Key Features • Interface-independent appliance specification • Automatic generation of GUI and speech interfaces

  5. Important Work By Others • INCITS V2 Standardization Effort Alternative Interface Access Protocol (AIAP) [Zimmermann, CHI 2002] • User Interface Modeling Language (UIML) http://www.uiml.org/ • Xweb [Olsen Jr., UIST 2000] • Stanford iRoom, iCrafter [Ponnekanti, Ubicomp 2001]

  6. Automatic Generation of UIs Benefits • All interfaces consistent for the user • With conventions of handheld • Even from multiple manufacturers Addresses hotel alarm clock problem! • Multiple modalities (GUI + Speech UI) • Can take into account user preferences • Will work on special purpose devices (for disabled) A Hard Problem • Previous systems have not generated high quality interfaces

  7. Our Approach • Create graphical and speech interfaces by hand • Perform user studies to ensure high quality With our interfaces, half the time and half the errors compared to manufacturers’ interfaces

  8. Approach, cont. • Examine the hand-designed interfaces to understand what information is needed for their creation • Focus on functional appliance information • We have developed 8 requirements for systems that automatically generate interfaces

  9. 1. Two-Way Communication Controller and appliance must have two-way communication. • Download of Specification • Sending Control Signals • Synchronization of State Information • Disabling of inactive components • State change feedback • Support for verbal querying of state in speech

  10. 2. Simultaneous Multiple Controllers Multiple users will want to control common appliances at the same time • e.g. Television, VCR Allows construction of multi-modal control interfaces • One graphical controller and one speech controller used simultaneously

  11. 3. No Specific Layout Information What if this was allowed? • Specifications would become much longer • Specifications might lose forward-compatibility with devices of the future • Some advantages of automatic generation are lost • e.g. Consistency of similar appliance interfaces UIML allows specific information in description.

  12. Good organization is a requirement of good UI A tree is a nice structure for specifying organization 4. Hierarchical Grouping

  13. 5. State Variables and Commands • The functions of an appliance are represented as state variables and commands • Most functions can be inferred from state variables (e.g. tuning from radio station) • But some cannot (e.g. seek) and some functions are stateless (e.g. flash on telephone), so we need commands.

  14. 6. Dependency Information • Most GUIs have “disabled control” concept • Important for usability • When a control will be active can be described in terms of other states • Also useful for layout! • Controls that are never available at the same time might overlap each other on the screen (e.g. tabs) PUC and V2 specify dependency information.

  15. 7. Sufficient Labels • Labels are important for creating good UIs • A problem for the physical interfaces in our study • Even more important for speech • Need more information • Multiple strings • Graphics • Pronunciation Information • Recordings for text-to-speech Most systems provide only single string labels

  16. 8. High-Level Semantic Knowledge Need knowledge of design conventions to build good interfaces • Country-specific date formatting • 3x4 button configuration of a phone dial pad Solution • Add high-level elements where conventional knowledge would help, but fall back to variables and commands if controller does not understand

  17. Our Current Status Created specification language and built interface generator Built adaptors to existing appliances • Sony Camcorder • Audiophase Shelf Stereo • Mitsubishi VCR • X10 Collaborating with V2 on standard • V2 description language changing in response

  18. PUC Project Members Brad A. Myers Michael Higgins Joseph Hughes Thomas K. Harris Roni Rosenfeld Stefanie Shriver Peter Lucas Mathilde Pignol Kevin Litwack Funding National Science Foundation Pittsburgh Digital Greenhouse Microsoft Equipment Grants Mitsubishi (MERL) VividLogic Symbol Technologies Hewlett-Packard Lucent Acknowledgements

  19. Thanks! http://www.cs.cmu.edu/~pebbles/puc/ http://www.cs.cmu.edu/~jeffreyn/ October 15, 2002 Come see our demo! Tonight at 7pm at Carnegie Mellon

More Related