Generating remote control interfaces for complex appliances
This presentation is the property of its rightful owner.
Sponsored Links
1 / 27

Generating Remote Control Interfaces for Complex Appliances PowerPoint PPT Presentation


  • 91 Views
  • Uploaded on
  • Presentation posted in: General

Generating Remote Control Interfaces for Complex Appliances. Jeffrey Nichols and Brad A. Myers Carnegie Mellon University October 30, 2002. The Fifteenth Annual Symposium on User Interface Software & Technology (UIST) 2002 Paris, France. The Problem. Appliances are too complex.

Download Presentation

Generating Remote Control Interfaces for Complex Appliances

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Generating remote control interfaces for complex appliances

Generating Remote Control Interfaces for Complex Appliances

Jeffrey Nichols and Brad A. Myers

Carnegie Mellon University

October 30, 2002

The Fifteenth Annual Symposium on User Interface Software & Technology (UIST) 2002 Paris, France


The problem

The Problem

Appliances are too complex


The problem cont

The Problem, cont.

Each complex appliances has its own idiosyncratic interface!

  • Stereo systems

  • Telephones

  • VCRs

  • Alarm Clocks

  • Increasingly Computerized

    Low Usability


Our solution

Feedback

Specifications

Control

Our Solution

Separate the interface from the appliance!

Handheld becomes personal universal controller (PUC)

Key Features

  • Interface-independent appliance specification

  • Automatic generation of GUI and speech interfaces


Automatic generation of uis

Automatic Generation of UIs

Benefits

  • All interfaces consistent for the user

    • With conventions of handheld

    • Even from multiple manufacturers

      Addresses hotel alarm clock problem!

  • Multiple modalities (GUI + Speech UI)

  • Can take into account user preferences

  • Will work on special purpose devices (for disabled)


Important work by others

Important Work By Others

  • INCITS V2 Standardization Effort

    Alternative Interface Access Protocol (AIAP)

    [Zimmermann, CHI 2002]

  • User Interface Modeling Language (UIML)

    http:/www.uiml.org/

  • Xweb (now ICE)

    [Olsen Jr., UIST 2000]

  • Stanford iRoom, iCrafter

    [Ponnekanti, Ubicomp 2001]

  • Speakeasy

    [Newman, UIST 2002]


Architecture

Architecture

- Comm. Protocol

- Interface Generators

- Specification Lang.

- Appliance Adaptors

XML-based


Language design

Language Design

Approach

  • Create reference interfaces

    • AIWA Shelf Stereo

    • AT&T Telephone/Answering Machine

  • Test interfaces with subjects

    • Users twice as fast and made half the errors with reference interfaces as compared to manufacturers’ interfaces

  • Analyze interfaces for functional information


Language elements

Language Elements

State Variables and Commands

  • Represent functions of appliance

  • State variables have types

    • Boolean, Enumeration, Integer, String, etc.

  • Variables sufficient for most functions but not all

    • “seek” button on a Radio

      Label Information

  • One label not suitable everywhere

    • The optimal label length changes with screen size

    • Speech interfaces may benefit from pronunciation and text-to-speech information


Language elements cont

Language Elements, cont.

Group Tree

  • Specify organization of functions

  • We use n-ary tree with variables or commands at leaves


Language elements cont1

Language Elements, cont.

Dependency Information

  • Formulas that specify when a variable or command is active in terms of other state variables

    • Equals, Greater Than, Less Than

    • Linked with logical operators (AND, OR)

  • For example,

    <and> <equals state=“PowerState”>true</equals> <equals state=“RadioBand”>AM</equals></and>


Interface generators

Interface Generators

Generators for Two Modalities

  • Graphical

    • Implemented for PocketPC in Java 1.1

    • Uses dependency information to generate panel structure of interface

  • Speech

    • Implemented using Universal Speech Interface (USI) techniques [Rosenfeld 2001]

    • Uses dependency information to disambiguate shortcut words (e.g. “play”) and resolve pre-conditions for a requested function (e.g. “play CD”)


Graphical interface generator

Graphical Interface Generator

Focuses on panel structure of user interface

  • Small groups of controls have basic layouts

  • Complexity comes from structure of groups

  • Structure can be inferred from dependency info!


Inferring structure

Inferring Structure

Find sets of variables that are “mutually exclusive”

  • Every variable in a set will never be active at the same time as a variable in another set

    Create structure with sets, using overlapping panels


Choosing panel types

Choosing Panel Types

a)

b)

c)

partial screen

full screen

tabbed


Making the interface concrete

Making the Interface Concrete

Finish conceptual layout

  • Choose controls (decision tree)

  • Choose row layouts

    (one column, two column, etc.)

    Allocate space

  • Examine panel contents and choose sizes

    Instantiate and place controls


Generating speech interfaces

Generating Speech Interfaces

Automatically build USI tree from dependencies

  • Allows verbal navigation of functional groups

    Automatically generate grammar for parser

  • Phrases for query and control

    “What is playmode?”

    “Set playmode to play”

    “play”

    Automatically generate language model and pronunciation for recognizer


Controlling appliances

Controlling Appliances

We have built adaptors for many actual appliances

  • Sony Digital Camcorder

  • X10 Lighting

  • Audiophase Shelf Stereo

  • AudioReQuest MP3 player

  • Software Media Players (WinAmp, WMP)

    Written specifications for others

  • Elevator

  • Telephone/Answering Machine


Demonstration

Demonstration


Future work

Future Work

  • Improve quality of generated user interfaces

    • Back-tracking, etc.

  • Specification and generation of list interfaces

    • Necessary for many appliances

  • Adaptive interface generation

    • New interfaces modeled on past interfaces

  • General purpose appliance adaptor for HAVi and UPnP protocols


Acknowledgements

PUC Project Members

Brad A. Myers

Michael Higgins

Joseph Hughes

Thomas K. Harris

Roni Rosenfeld

Mathilde Pignol

Stefanie Shriver

Peter Lucas

Kevin Litwack

Funding

National Science Foundation

Pittsburgh Digital Greenhouse

Microsoft

Equipment Grants

Mitsubishi (MERL)

VividLogic

Symbol Technologies

Hewlett-Packard

Lucent

Acknowledgements


Thanks

Thanks!

User Interface Software & Technology (UIST)

October 30, 2002

http://www.cs.cmu.edu/~pebbles/puc/

http://www.cs.cmu.edu/~jeffreyn/


Determining panel structure

Determining Panel Structure

  • Find states that others depend upon

  • Search for the group that contains this state

  • Determine mutually exclusive sets within group children

  • Assign sets to panels

    • Based upon types, tree properties

FMStation

Mode == Any

FMPresets

Power

AMStation

Mode

AMPresets

Band

Radio

Mode == Radio

PlayState

CD

CDTrack

Mode == CD


Choosing panel types1

Choosing Panel Types

Rules for Choosing Panels

  • If…

    • state has boolean type

    • all controls are one set

  • Create…

    • Two full-screen overlapping panels.


Choosing panel types cont

Choosing Panel Types, cont.

Rules for Choosing Panels

  • If…

    • state has enumerated type

    • there is a set for every value of the state

  • Create…

    • a tabbed panel


Choosing panel types cont1

Choosing Panel Types, cont.

Rules for Choosing Panels

  • If…

    • there are mutually exclusive sets of controls

  • Create…

    • overlapping panels

    • a control for the state not on the overlapping panel


  • Login