huddle and information triggers helping users automate complex and temporally distant tasks
Download
Skip this Video
Download Presentation
Huddle and Information Triggers: Helping Users Automate Complex and Temporally-Distant Tasks

Loading in 2 Seconds...

play fullscreen
1 / 72

Huddle and Information Triggers: - PowerPoint PPT Presentation


  • 279 Views
  • Uploaded on

Huddle and Information Triggers: Helping Users Automate Complex and Temporally-Distant Tasks. Jeffrey Nichols IBM Almaden Research Center UC Berkeley June 14, 2007. Today – Two Mini Talks. Huddle Automatic generation of interfaces for systems of connected appliances

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Huddle and Information Triggers: ' - ryanadan


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
huddle and information triggers helping users automate complex and temporally distant tasks

Huddle and Information Triggers:Helping Users Automate Complex and Temporally-Distant Tasks

Jeffrey Nichols

IBM Almaden Research Center

UC Berkeley

June 14, 2007

today two mini talks
Today – Two Mini Talks
  • Huddle
    • Automatic generation of interfaces for systems of connected appliances
    • Part of my dissertation work on the Personal Universal Controller
  • Information Triggers
    • Helps users recall important information or tasks at the moment they are important
    • New project at IBM Almaden Research Center
    • In collaboration with Jake Wobbrock at the University of Washington
the multi appliance problem cont
The Multi-Appliance Problem, cont.

How can we provide improved interfaces for systems of connected appliances?

control of multiple appliances
Control of Multiple Appliances

Universal Remote Controls

Built-in Interfaces

Custom Installations

huddle
Huddle
  • Improved multi-appliance interfaces based on content flow

Flow-Based Interface

Aggregate User Interfaces

personal universal controller puc
Personal Universal Controller (PUC)
  • Automatically generates interfaces for remotely controlling appliances [Nichols, UIST 2002]
  • Specification language for describing appliance functionality

Dependency information that describes the availability of functions based on appliance state

  • Automatically generate new interfaces to be consistent with previous interfaces [Nichols, CHI 2006]

Knowledge base that identifies and stores similarities between appliances

related work
Related Work
  • Roadie [Lieberman, IUI 2006]

Goal-oriented natural language interface for appliances

Uses planner to configure appliances based on common sense database (EventNet)

  • ICrafter [Ponnekanti, Ubicomp 2002]

Builds aggregate user interfaces based on “service interfaces”

Hand-coded interface aggregators

Limited automatic generation capabilities

  • Omojokun, et al. [Omojokun 2006]

Examined how remote control interfaces might be generated from recordings of actual usage

Recorded remote control usage with IR receiver

  • General Work on Interface Generation
    • Task-based work: TERESA [Mori 2004], XIML [Puerta, IUI 2002], others…
    • SUPPLE[Gajos, IUI 2004]
huddle overview
Huddle Overview
  • Introduction
  • Content Flow Modeling
  • Flow-Based Interface
  • Aggregate User Interfaces
  • Discussion and Future Work
content flow model
Content Flow Model
  • Home Theater

DVD

Receiver

BroadcastTelevision

TV

VCR #1

VCR #2

modeling tasks
Modeling Tasks
  • Home Theater: Watching Television

DVD

Receiver

BroadcastTelevision

TV

VCR #1

VCR #2

modeling tasks12
Modeling Tasks
  • Home Theater: Watching Television

DVD

Receiver

BroadcastTelevision

TV

VCR #1

VCR #2

modeling tasks13
Modeling Tasks
  • Home Theater: Watching Television

DVD

Receiver

BroadcastTelevision

TV

VCR #1

VCR #2

modeling tasks14
Modeling Tasks
  • Home Theater: Watch DVD

DVD

Receiver

BroadcastTelevision

TV

VCR #1

VCR #2

modeling tasks15
Modeling Tasks
  • Home Theater: Watch DVD + Copy Tape

DVD

Receiver

BroadcastTelevision

TV

VCR #1

VCR #2

modeling tasks16
Modeling Tasks
  • Presentation Room

VCR

PowerPoint

Projector

Laptop Task Manager

External Video Control

Media Player

modeling tasks17
Modeling Tasks
  • Presentation Room: Presenting PowerPoint

VCR

PowerPoint

Projector

Laptop Task Manager

External Video Control

Media Player

where does the model come from
Where does the model come from?

DVD

Receiver

BroadcastTelevision

TV

VCR #1

VCR #2

where does the model come from19
Where does the model come from?

DVD

Receiver

TV

VCR #1

VCR #2

where does the model come from20
Where does the model come from?

DVD

Receiver

TV

VCR #1

VCR #2

flow based interface fbi
Flow-Based Interface (FBI)

Flow-Based Interface

  • Allows user to manage high-level tasks
  • Quick specification of flows
  • AI Planner automatically configures appliances
  • Question/Answer interface to address conflicts
aggregate user interfaces auis
Aggregate User Interfaces (AUIs)

Aggregate Controls for “Watch DVD”

  • Allows user to manage low-level details
  • Control or setup of active flows
  • General setup of all appliances
  • Merged functions
aggregate user interfaces
Aggregate User Interfaces
  • Three kinds
    • Control of Active Flows

DVD

Receiver

BroadcastTelevision

TV

VCR #2

VCR #1

aggregate user interfaces33
Aggregate User Interfaces
  • Three kinds
    • Control of Active Flows

DVD

Receiver

BroadcastTelevision

TV

VCR #2

VCR #1

aggregate user interfaces34
Aggregate User Interfaces
  • Three kinds
    • Control of Active Flows

DVD

Receiver

BroadcastTelevision

TV

VCR #2

VCR #1

aggregate user interfaces35
Aggregate User Interfaces
  • Three kinds
    • Control of Active Flows

DVD

Receiver

BroadcastTelevision

TV

VCR #2

VCR #1

aggregate user interfaces36
Aggregate User Interfaces
  • Three kinds
    • Control of Active Flows

DVD

Receiver

BroadcastTelevision

TV

VCR #2

VCR #1

aggregate user interfaces37
Aggregate User Interfaces

Volume for TV

Volume for Receiver

  • Three kinds
    • Control of Active Flows

“Copy Tape”

“Watch TV using TV speakers”

“Watch TV using Receiver speakers”

aggregate user interfaces38
Aggregate User Interfaces
  • Three kinds
    • Control of Active Flows
    • Setup of Active Flows

Two screens of “Watch DVD” setup

aggregate user interfaces39
Aggregate User Interfaces
  • Three kinds
    • Control of Active Flows
    • Setup of Active Flows
    • General Setup of all appliances

Two screens of General Setup for a home theater config.

aggregate user interfaces40
Aggregate User Interfaces
  • Three kinds
    • Control of Active Flows
    • Setup of Active Flows
    • General Setup of all appliances
  • Merged Functions
    • Most functions shouldn’t be merged
    • Some should:

clock, language, sleep timer…

    • Future work…

Baudisch’s flat volume control

General merging

Some merged functions

discussion
Discussion
  • How can our use of content flow be extended?
    • Lighting in a presentation room
    • Video-conferencing rooms
    • Outside the home: factories & offices
    • Kitchen?
future work
Future Work
  • Extending to more detailed content types
    • Home theater: support for HD signals, encoded audio, etc.
  • Setup and Troubleshooting
    • Tool for specifying initial wiring
    • Helping with Problems
  • More aggregate interfaces
    • Usage-based aggregate?
  • Usability evaluations
information triggers44
Information Triggers
  • Motivation

It’s easy to find stuff on the web

It’s even easy to re-find something you’ve previously found...

...if you remember to re-find it when you need it

“Participants [in a re-finding study] complained that they often forgot to look for an information item until the period of its usefulness had passed.” (Jones et al. 2005)

information triggers45
Information Triggers
  • What we’ve built so far
  • A Firefox extension that allows people to associate web pages with future events at which that page will be particularly relevant
  • When the event occurs, the web page is automatically opened for the user
  • Goal is to make triggers as easy and lightweight to configure as possible
  • Supports three different types of triggers:
    • Date & time
    • Arrival of an e-mail
    • Visit to a web page
beyond current reminder systems
Beyond Current Reminder Systems
  • Existing reminder systems provide a short message about their topic, but info triggers provides the information itself
  • Context-based reminder systems (e.g. Cybreminder) have mainly used location for context
scenarios
Scenarios
  • Waiting to login to a website to configure account information (date & time trigger)
scenarios48
Scenarios
  • Resuming a task that can’t be completed in one sitting, e.g. waiting for manager approval (new e-mail trigger)
scenarios49
Scenarios
  • Opening a note-taking site when visiting a page with specific keywords (page visit trigger)

information retrieval

web pages

scenarios50
Scenarios
  • Delaying a task until a leisure period (page visit trigger)
scenarios51
Scenarios
  • Opening a set of pages all at once, e.g. opening one bill payment site opens others (page visit trigger)
scenarios52
Scenarios
  • Automating routine browsing tasks (date & time trigger)
user feedback
User Feedback
  • Distributed the extension within IBM, currently have several regular users
  • Needed features
    • Means for configuring mail triggers within mail reader
    • Ability to share triggers with others
  • Uses include:
    • Reminders of information related to a current event
      • Automatically display the web page/blog of a person you are about to meet with
    • Regular reminders of a task prior to a deadline
      • Make the conference registration page appear every morning on the week before the early registration deadline
    • Automating phone call forwarding
      • One user has a set of triggers that are able to automatically start and stop call forwarding from the office on days that he regularly works from home
    • Automating regular browsing tasks
      • Weekly entry of status reports
discussion55
Discussion
  • Programming triggers can be hard or require being “clever”
    • Can’t always predict when some information will be useful
    • Current set of triggers is powerful, but making them do what you want may not be straight-forward
      • Delaying until leisure time trigger
      • Triggering the next time you open your browser
      • Triggers that open pages that trigger other things
  • Where should triggers be set?
    • Triggered information vs. triggering information
    • Need usability studies to improve authoring of triggers
  • Deferring Triggered Information
    • Moving the window to the background can be sufficient
    • May need an additional explicit mechanism
future work56
Future Work
  • Other Trigger Events?
    • What other context information might be useful to trigger on?
  • Sharing Triggers
    • Can some of the overhead of sharing triggers be offset using social mechanisms?
  • Distributed Triggers
    • No one is always at their browser
    • Trigger on mobile devices or across a user’s machines
  • Trigger Queues
    • A trigger that fires regularly associated with a queue of pages
    • Each time the trigger fires the next item from the queue is displayed
    • “Interesting new articles from today”, “Quick online to-do items”, “Papers to read”, etc.
future work cont
Future Work, cont.
  • Usability and Usage Studies
    • Improve the interaction
    • Is it useful? When?
  • Deeper Questions
    • How well can people anticipate when they will need information?
    • How well can people anticipate when they will next be interruptable?
ibm almaden research center
IBM Almaden Research Center
  • 400+ research employees; 100+ students and postdocs
  • Research in Computer Science, Storage Systems, Science and Technology, Services Science
  • HCI-related Departments:
    • User Focused Systems group in CS
    • Services Sciences research
user group at almaden
USER Group at Almaden
  • Four main research thrusts:

Activities

      • Unified Activity Management
      • Koala

Web 2.0 and Social Computing

      • BlueMail
      • Fringe

Content Management

      • AACS, etc.

Interaction in Healthcare

      • Data entry for EMR
      • Communication management (awareness & context)
      • Patient-centered healthcare
thanks for listening

Thanks for listening!

For more information…

http://www.pebbles.hcii.cmu.edu/puc/

http://www.almaden.ibm.com/

http://www.jeffreynichols.com/

huddle and information triggers helping users automate complex and temporally distant tasks61

Huddle and Information Triggers:Helping Users Automate Complex and Temporally-Distant Tasks

Jeffrey Nichols

IBM Almaden Research Center

UC Berkeley

June 14, 2007

ad