Huddle and information triggers helping users automate complex and temporally distant tasks
Download
1 / 72

Huddle and Information Triggers: Helping Users Automate Complex and Temporally-Distant Tasks - PowerPoint PPT Presentation


  • 279 Views
  • Uploaded on

Huddle and Information Triggers: Helping Users Automate Complex and Temporally-Distant Tasks. Jeffrey Nichols IBM Almaden Research Center UC Berkeley June 14, 2007. Today – Two Mini Talks. Huddle Automatic generation of interfaces for systems of connected appliances

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Huddle and Information Triggers: Helping Users Automate Complex and Temporally-Distant Tasks' - ryanadan


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Huddle and information triggers helping users automate complex and temporally distant tasks l.jpg

Huddle and Information Triggers:Helping Users Automate Complex and Temporally-Distant Tasks

Jeffrey Nichols

IBM Almaden Research Center

UC Berkeley

June 14, 2007


Today two mini talks l.jpg
Today – Two Mini Talks

  • Huddle

    • Automatic generation of interfaces for systems of connected appliances

    • Part of my dissertation work on the Personal Universal Controller

  • Information Triggers

    • Helps users recall important information or tasks at the moment they are important

    • New project at IBM Almaden Research Center

    • In collaboration with Jake Wobbrock at the University of Washington



The multi appliance problem cont l.jpg
The Multi-Appliance Problem, cont.

How can we provide improved interfaces for systems of connected appliances?


Control of multiple appliances l.jpg
Control of Multiple Appliances

Universal Remote Controls

Built-in Interfaces

Custom Installations


Huddle l.jpg
Huddle

  • Improved multi-appliance interfaces based on content flow

Flow-Based Interface

Aggregate User Interfaces


Personal universal controller puc l.jpg
Personal Universal Controller (PUC)

  • Automatically generates interfaces for remotely controlling appliances [Nichols, UIST 2002]

  • Specification language for describing appliance functionality

    Dependency information that describes the availability of functions based on appliance state

  • Automatically generate new interfaces to be consistent with previous interfaces [Nichols, CHI 2006]

    Knowledge base that identifies and stores similarities between appliances


Related work l.jpg
Related Work

  • Roadie [Lieberman, IUI 2006]

    Goal-oriented natural language interface for appliances

    Uses planner to configure appliances based on common sense database (EventNet)

  • ICrafter [Ponnekanti, Ubicomp 2002]

    Builds aggregate user interfaces based on “service interfaces”

    Hand-coded interface aggregators

    Limited automatic generation capabilities

  • Omojokun, et al. [Omojokun 2006]

    Examined how remote control interfaces might be generated from recordings of actual usage

    Recorded remote control usage with IR receiver

  • General Work on Interface Generation

    • Task-based work: TERESA [Mori 2004], XIML [Puerta, IUI 2002], others…

    • SUPPLE[Gajos, IUI 2004]


Huddle overview l.jpg
Huddle Overview

  • Introduction

  • Content Flow Modeling

  • Flow-Based Interface

  • Aggregate User Interfaces

  • Discussion and Future Work


Content flow model l.jpg
Content Flow Model

  • Home Theater

DVD

Receiver

BroadcastTelevision

TV

VCR #1

VCR #2


Modeling tasks l.jpg
Modeling Tasks

  • Home Theater: Watching Television

DVD

Receiver

BroadcastTelevision

TV

VCR #1

VCR #2


Modeling tasks12 l.jpg
Modeling Tasks

  • Home Theater: Watching Television

DVD

Receiver

BroadcastTelevision

TV

VCR #1

VCR #2


Modeling tasks13 l.jpg
Modeling Tasks

  • Home Theater: Watching Television

DVD

Receiver

BroadcastTelevision

TV

VCR #1

VCR #2


Modeling tasks14 l.jpg
Modeling Tasks

  • Home Theater: Watch DVD

DVD

Receiver

BroadcastTelevision

TV

VCR #1

VCR #2


Modeling tasks15 l.jpg
Modeling Tasks

  • Home Theater: Watch DVD + Copy Tape

DVD

Receiver

BroadcastTelevision

TV

VCR #1

VCR #2


Modeling tasks16 l.jpg
Modeling Tasks

  • Presentation Room

VCR

PowerPoint

Projector

Laptop Task Manager

External Video Control

Media Player


Modeling tasks17 l.jpg
Modeling Tasks

  • Presentation Room: Presenting PowerPoint

VCR

PowerPoint

Projector

Laptop Task Manager

External Video Control

Media Player


Where does the model come from l.jpg
Where does the model come from?

DVD

Receiver

BroadcastTelevision

TV

VCR #1

VCR #2


Where does the model come from19 l.jpg
Where does the model come from?

DVD

Receiver

TV

VCR #1

VCR #2


Where does the model come from20 l.jpg
Where does the model come from?

DVD

Receiver

TV

VCR #1

VCR #2


Flow based interface fbi l.jpg
Flow-Based Interface (FBI)

Flow-Based Interface

  • Allows user to manage high-level tasks

  • Quick specification of flows

  • AI Planner automatically configures appliances

  • Question/Answer interface to address conflicts











Aggregate user interfaces auis l.jpg
Aggregate User Interfaces (AUIs)

Aggregate Controls for “Watch DVD”

  • Allows user to manage low-level details

  • Control or setup of active flows

  • General setup of all appliances

  • Merged functions


Aggregate user interfaces l.jpg
Aggregate User Interfaces

  • Three kinds

    • Control of Active Flows

DVD

Receiver

BroadcastTelevision

TV

VCR #2

VCR #1


Aggregate user interfaces33 l.jpg
Aggregate User Interfaces

  • Three kinds

    • Control of Active Flows

DVD

Receiver

BroadcastTelevision

TV

VCR #2

VCR #1


Aggregate user interfaces34 l.jpg
Aggregate User Interfaces

  • Three kinds

    • Control of Active Flows

DVD

Receiver

BroadcastTelevision

TV

VCR #2

VCR #1


Aggregate user interfaces35 l.jpg
Aggregate User Interfaces

  • Three kinds

    • Control of Active Flows

DVD

Receiver

BroadcastTelevision

TV

VCR #2

VCR #1


Aggregate user interfaces36 l.jpg
Aggregate User Interfaces

  • Three kinds

    • Control of Active Flows

DVD

Receiver

BroadcastTelevision

TV

VCR #2

VCR #1


Aggregate user interfaces37 l.jpg
Aggregate User Interfaces

Volume for TV

Volume for Receiver

  • Three kinds

    • Control of Active Flows

“Copy Tape”

“Watch TV using TV speakers”

“Watch TV using Receiver speakers”


Aggregate user interfaces38 l.jpg
Aggregate User Interfaces

  • Three kinds

    • Control of Active Flows

    • Setup of Active Flows

Two screens of “Watch DVD” setup


Aggregate user interfaces39 l.jpg
Aggregate User Interfaces

  • Three kinds

    • Control of Active Flows

    • Setup of Active Flows

    • General Setup of all appliances

Two screens of General Setup for a home theater config.


Aggregate user interfaces40 l.jpg
Aggregate User Interfaces

  • Three kinds

    • Control of Active Flows

    • Setup of Active Flows

    • General Setup of all appliances

  • Merged Functions

    • Most functions shouldn’t be merged

    • Some should:

      clock, language, sleep timer…

    • Future work…

      Baudisch’s flat volume control

      General merging

Some merged functions


Discussion l.jpg
Discussion

  • How can our use of content flow be extended?

    • Lighting in a presentation room

    • Video-conferencing rooms

    • Outside the home: factories & offices

    • Kitchen?


Future work l.jpg
Future Work

  • Extending to more detailed content types

    • Home theater: support for HD signals, encoded audio, etc.

  • Setup and Troubleshooting

    • Tool for specifying initial wiring

    • Helping with Problems

  • More aggregate interfaces

    • Usage-based aggregate?

  • Usability evaluations



Information triggers44 l.jpg
Information Triggers

  • Motivation

It’s easy to find stuff on the web

It’s even easy to re-find something you’ve previously found...

...if you remember to re-find it when you need it

“Participants [in a re-finding study] complained that they often forgot to look for an information item until the period of its usefulness had passed.” (Jones et al. 2005)


Information triggers45 l.jpg
Information Triggers

  • What we’ve built so far

  • A Firefox extension that allows people to associate web pages with future events at which that page will be particularly relevant

  • When the event occurs, the web page is automatically opened for the user

  • Goal is to make triggers as easy and lightweight to configure as possible

  • Supports three different types of triggers:

    • Date & time

    • Arrival of an e-mail

    • Visit to a web page


Beyond current reminder systems l.jpg
Beyond Current Reminder Systems

  • Existing reminder systems provide a short message about their topic, but info triggers provides the information itself

  • Context-based reminder systems (e.g. Cybreminder) have mainly used location for context


Scenarios l.jpg
Scenarios

  • Waiting to login to a website to configure account information (date & time trigger)


Scenarios48 l.jpg
Scenarios

  • Resuming a task that can’t be completed in one sitting, e.g. waiting for manager approval (new e-mail trigger)


Scenarios49 l.jpg
Scenarios

  • Opening a note-taking site when visiting a page with specific keywords (page visit trigger)

information retrieval

web pages


Scenarios50 l.jpg
Scenarios

  • Delaying a task until a leisure period (page visit trigger)


Scenarios51 l.jpg
Scenarios

  • Opening a set of pages all at once, e.g. opening one bill payment site opens others (page visit trigger)


Scenarios52 l.jpg
Scenarios

  • Automating routine browsing tasks (date & time trigger)



User feedback l.jpg
User Feedback

  • Distributed the extension within IBM, currently have several regular users

  • Needed features

    • Means for configuring mail triggers within mail reader

    • Ability to share triggers with others

  • Uses include:

    • Reminders of information related to a current event

      • Automatically display the web page/blog of a person you are about to meet with

    • Regular reminders of a task prior to a deadline

      • Make the conference registration page appear every morning on the week before the early registration deadline

    • Automating phone call forwarding

      • One user has a set of triggers that are able to automatically start and stop call forwarding from the office on days that he regularly works from home

    • Automating regular browsing tasks

      • Weekly entry of status reports


Discussion55 l.jpg
Discussion

  • Programming triggers can be hard or require being “clever”

    • Can’t always predict when some information will be useful

    • Current set of triggers is powerful, but making them do what you want may not be straight-forward

      • Delaying until leisure time trigger

      • Triggering the next time you open your browser

      • Triggers that open pages that trigger other things

  • Where should triggers be set?

    • Triggered information vs. triggering information

    • Need usability studies to improve authoring of triggers

  • Deferring Triggered Information

    • Moving the window to the background can be sufficient

    • May need an additional explicit mechanism


Future work56 l.jpg
Future Work

  • Other Trigger Events?

    • What other context information might be useful to trigger on?

  • Sharing Triggers

    • Can some of the overhead of sharing triggers be offset using social mechanisms?

  • Distributed Triggers

    • No one is always at their browser

    • Trigger on mobile devices or across a user’s machines

  • Trigger Queues

    • A trigger that fires regularly associated with a queue of pages

    • Each time the trigger fires the next item from the queue is displayed

    • “Interesting new articles from today”, “Quick online to-do items”, “Papers to read”, etc.


Future work cont l.jpg
Future Work, cont.

  • Usability and Usage Studies

    • Improve the interaction

    • Is it useful? When?

  • Deeper Questions

    • How well can people anticipate when they will need information?

    • How well can people anticipate when they will next be interruptable?


Ibm almaden research center l.jpg
IBM Almaden Research Center

  • 400+ research employees; 100+ students and postdocs

  • Research in Computer Science, Storage Systems, Science and Technology, Services Science

  • HCI-related Departments:

    • User Focused Systems group in CS

    • Services Sciences research


User group at almaden l.jpg
USER Group at Almaden

  • Four main research thrusts:

    Activities

    • Unified Activity Management

    • Koala

      Web 2.0 and Social Computing

    • BlueMail

    • Fringe

      Content Management

    • AACS, etc.

      Interaction in Healthcare

    • Data entry for EMR

    • Communication management (awareness & context)

    • Patient-centered healthcare


Thanks for listening l.jpg

Thanks for listening!

For more information…

http://www.pebbles.hcii.cmu.edu/puc/

http://www.almaden.ibm.com/

http://www.jeffreynichols.com/


Huddle and information triggers helping users automate complex and temporally distant tasks61 l.jpg

Huddle and Information Triggers:Helping Users Automate Complex and Temporally-Distant Tasks

Jeffrey Nichols

IBM Almaden Research Center

UC Berkeley

June 14, 2007