1 / 72

Huddle and Information Triggers: Helping Users Automate Complex and Temporally-Distant Tasks

Huddle and Information Triggers: Helping Users Automate Complex and Temporally-Distant Tasks. Jeffrey Nichols IBM Almaden Research Center UC Berkeley June 14, 2007. Today – Two Mini Talks. Huddle Automatic generation of interfaces for systems of connected appliances

violet
Download Presentation

Huddle and Information Triggers: Helping Users Automate Complex and Temporally-Distant Tasks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Huddle and Information Triggers:Helping Users Automate Complex and Temporally-Distant Tasks Jeffrey Nichols IBM Almaden Research Center UC Berkeley June 14, 2007

  2. Today – Two Mini Talks • Huddle • Automatic generation of interfaces for systems of connected appliances • Part of my dissertation work on the Personal Universal Controller • Information Triggers • Helps users recall important information or tasks at the moment they are important • New project at IBM Almaden Research Center • In collaboration with Jake Wobbrock at the University of Washington

  3. The Multi-Appliance Problem

  4. The Multi-Appliance Problem, cont. How can we provide improved interfaces for systems of connected appliances?

  5. Control of Multiple Appliances Universal Remote Controls Built-in Interfaces Custom Installations

  6. Huddle • Improved multi-appliance interfaces based on content flow Flow-Based Interface Aggregate User Interfaces

  7. Personal Universal Controller (PUC) • Automatically generates interfaces for remotely controlling appliances [Nichols, UIST 2002] • Specification language for describing appliance functionality Dependency information that describes the availability of functions based on appliance state • Automatically generate new interfaces to be consistent with previous interfaces [Nichols, CHI 2006] Knowledge base that identifies and stores similarities between appliances

  8. Related Work • Roadie [Lieberman, IUI 2006] Goal-oriented natural language interface for appliances Uses planner to configure appliances based on common sense database (EventNet) • ICrafter [Ponnekanti, Ubicomp 2002] Builds aggregate user interfaces based on “service interfaces” Hand-coded interface aggregators Limited automatic generation capabilities • Omojokun, et al. [Omojokun 2006] Examined how remote control interfaces might be generated from recordings of actual usage Recorded remote control usage with IR receiver • General Work on Interface Generation • Task-based work: TERESA [Mori 2004], XIML [Puerta, IUI 2002], others… • SUPPLE[Gajos, IUI 2004]

  9. Huddle Overview • Introduction • Content Flow Modeling • Flow-Based Interface • Aggregate User Interfaces • Discussion and Future Work

  10. Content Flow Model • Home Theater DVD Receiver BroadcastTelevision TV VCR #1 VCR #2

  11. Modeling Tasks • Home Theater: Watching Television DVD Receiver BroadcastTelevision TV VCR #1 VCR #2

  12. Modeling Tasks • Home Theater: Watching Television DVD Receiver BroadcastTelevision TV VCR #1 VCR #2

  13. Modeling Tasks • Home Theater: Watching Television DVD Receiver BroadcastTelevision TV VCR #1 VCR #2

  14. Modeling Tasks • Home Theater: Watch DVD DVD Receiver BroadcastTelevision TV VCR #1 VCR #2

  15. Modeling Tasks • Home Theater: Watch DVD + Copy Tape DVD Receiver BroadcastTelevision TV VCR #1 VCR #2

  16. Modeling Tasks • Presentation Room VCR PowerPoint Projector Laptop Task Manager External Video Control Media Player

  17. Modeling Tasks • Presentation Room: Presenting PowerPoint VCR PowerPoint Projector Laptop Task Manager External Video Control Media Player

  18. Where does the model come from? DVD Receiver BroadcastTelevision TV VCR #1 VCR #2

  19. Where does the model come from? DVD Receiver TV VCR #1 VCR #2

  20. Where does the model come from? DVD Receiver TV VCR #1 VCR #2

  21. Flow-Based Interface (FBI) Flow-Based Interface • Allows user to manage high-level tasks • Quick specification of flows • AI Planner automatically configures appliances • Question/Answer interface to address conflicts

  22. Flow-Based Interface

  23. Flow-Based Interface

  24. Flow-Based Interface

  25. Flow-Based Interface

  26. Flow-Based Interface

  27. Flow-Based Interface

  28. Flow-Based Interface

  29. Flow-Based Interface

  30. Flow-Based Interface

  31. Aggregate User Interfaces (AUIs) Aggregate Controls for “Watch DVD” • Allows user to manage low-level details • Control or setup of active flows • General setup of all appliances • Merged functions

  32. Aggregate User Interfaces • Three kinds • Control of Active Flows DVD Receiver BroadcastTelevision TV VCR #2 VCR #1

  33. Aggregate User Interfaces • Three kinds • Control of Active Flows DVD Receiver BroadcastTelevision TV VCR #2 VCR #1

  34. Aggregate User Interfaces • Three kinds • Control of Active Flows DVD Receiver BroadcastTelevision TV VCR #2 VCR #1

  35. Aggregate User Interfaces • Three kinds • Control of Active Flows DVD Receiver BroadcastTelevision TV VCR #2 VCR #1

  36. Aggregate User Interfaces • Three kinds • Control of Active Flows DVD Receiver BroadcastTelevision TV VCR #2 VCR #1

  37. Aggregate User Interfaces Volume for TV Volume for Receiver • Three kinds • Control of Active Flows “Copy Tape” “Watch TV using TV speakers” “Watch TV using Receiver speakers”

  38. Aggregate User Interfaces • Three kinds • Control of Active Flows • Setup of Active Flows Two screens of “Watch DVD” setup

  39. Aggregate User Interfaces • Three kinds • Control of Active Flows • Setup of Active Flows • General Setup of all appliances Two screens of General Setup for a home theater config.

  40. Aggregate User Interfaces • Three kinds • Control of Active Flows • Setup of Active Flows • General Setup of all appliances • Merged Functions • Most functions shouldn’t be merged • Some should: clock, language, sleep timer… • Future work… Baudisch’s flat volume control General merging Some merged functions

  41. Discussion • How can our use of content flow be extended? • Lighting in a presentation room • Video-conferencing rooms • Outside the home: factories & offices • Kitchen?

  42. Future Work • Extending to more detailed content types • Home theater: support for HD signals, encoded audio, etc. • Setup and Troubleshooting • Tool for specifying initial wiring • Helping with Problems • More aggregate interfaces • Usage-based aggregate? • Usability evaluations

  43. Information Triggers

  44. Information Triggers • Motivation It’s easy to find stuff on the web It’s even easy to re-find something you’ve previously found... ...if you remember to re-find it when you need it “Participants [in a re-finding study] complained that they often forgot to look for an information item until the period of its usefulness had passed.” (Jones et al. 2005)

  45. Information Triggers • What we’ve built so far • A Firefox extension that allows people to associate web pages with future events at which that page will be particularly relevant • When the event occurs, the web page is automatically opened for the user • Goal is to make triggers as easy and lightweight to configure as possible • Supports three different types of triggers: • Date & time • Arrival of an e-mail • Visit to a web page

  46. Beyond Current Reminder Systems • Existing reminder systems provide a short message about their topic, but info triggers provides the information itself • Context-based reminder systems (e.g. Cybreminder) have mainly used location for context

  47. Scenarios • Waiting to login to a website to configure account information (date & time trigger)

  48. Scenarios • Resuming a task that can’t be completed in one sitting, e.g. waiting for manager approval (new e-mail trigger)

  49. Scenarios • Opening a note-taking site when visiting a page with specific keywords (page visit trigger) information retrieval web pages

  50. Scenarios • Delaying a task until a leisure period (page visit trigger)

More Related