Instant Messaging and Interruption: Influence of Task Type on Performance

1. Instant Messaging and Interruption: Influence of Task Type on Performance Mary Czerwinski Ed Cutrell Eric Horvitz Microsoft Research

2. What Is Attention? • There are many definitions of attention • A function which selectively improves processing for one item, location, or task at the expense of others • Different modalities NRL April 2001—Czerwinski et al.

3. Some Motivation • Miyata & Norman (1986) • Predicted interruptions after important actions or between task execution and evaluation would be less harmful when multitasking NRL April 2001—Czerwinski et al.

4. Background 1 • Notifications are most distracting when they bear surface resemblance to the UI of the task at hand • Gillie & Broadbent (1989); Kreifelt & McCarthy (1981); Rhodes, Benoit & Payne (2000) • Auditory notifications can be more distracting than visual notifications • Mollenhauer, et al. (1994) NRL April 2001—Czerwinski et al.

5. Background 2 • People habituate to notifications over time, and training can help • Hess & Detweiler (1994); Altmann & Gray (2000) • Interruptions can be useful! • O’Conaill & Frohlich (1995) NRL April 2001—Czerwinski et al.

6. McFarlane (1999) • Examined 4 methods for instant messages • immediate (requiring immediate user response) • negotiated (user chooses when to attend) • mediated (an intelligent agent might determine when best to interrupt) • scheduled (interruptions come at prearranged time intervals) interruption methods • Negotiated resulted in good performance • users may postpone attending to interrupting messages in these cases • Immediate is fast but users are less efficient overall

7. Previous Work 1 • Degree of disruption depends on what task a user is doing and when the notification arrives • Relevant notifications are less disruptive than irrelevant • Czerwinski, Cutrell & Horvitz (2000) NRL April 2001—Czerwinski et al.

8. Previous Work 2 • Notifs during Web search task “phases” • Planning—deciding what search terms to use • Execution—entering the search terms • Evaluation—search the list looking for target • Disruption worst during: • Execution (“chunking”) • Evaluation (? ? ?) NRL April 2001—Czerwinski et al.

9. Messaging on the Web • Some trials interrupted with MSN’s Messenger service (v. 2.0) • Relevant messages (design category) • Irrelevant (factoid about target site) • Interruptions occurred during one of the three phases NRL April 2001—Czerwinski et al.

12. Why is List Scanning So Susceptible to Interruption? • 2 hypotheses: 1) Visual reorienting is hard to do People just lose their place in the list“Where was I?” 2) Problem with “conceptual reacquisition” Delay from accessing memory of goal“What was I doing?” NRL April 2001—Czerwinski et al.

13. How do we find out? • Sent IM messages to participants while they were scanning lists of book titles • Two kinds of targets: • Verbatim title (easy visual scan) • Gist of title (difficult semantic-based search) • Can we help? • Used a visual marker to save place in list NRL April 2001—Czerwinski et al.

14. Method • 12 experienced Microsoft Office 2000 users, aged 25-54, participated in this study. 6 had some experience with MSN’s Messenger. • 64 sets of 80 book titles obtained from MS library. Each set was an Excel spreadsheet. Targets were distinctive within a set of 80 titles. NRL April 2001—Czerwinski et al.

15. List Search • Target always visible at top • Navigation with Cursor Up/Down or Page Up/Down keys • Cursor—>marker • Search target either verbatim title or “gist” of title NRL April 2001—Czerwinski et al.

16. Procedure • In half of all trials, participants’ search task was interrupted with an instant message asking them a simple math problem. • Half of all trials had “gist” targets and half had title targets • Navigation was blocked, with half of the participants using Cursor Up/Down (Marked) first and the other half using Page Up/Down (Unmarked). NRL April 2001—Czerwinski et al.

17. Results • Only report time data; accuracy was quite good • Used log response times to normalize common skewing & variability of RT data NRL April 2001—Czerwinski et al.

18. Overall task times • IMs slow down task times • Searches for Gists are slower than for Titles • No difference in navigation NRL April 2001—Czerwinski et al.

19. Task Times Minus IM Time • Same pattern as overall task times—effects not due to device switching time • Marker only helped title search a little—navigational confound NRL April 2001—Czerwinski et al.

20. Summary • IM is disruptive • More disruptive for fast, stimulus-driven search than for slower, semantic-based search tasks • Marker didn’t seem to help, but was confounded by navigation style • Reran study w/o confound, same result NRL April 2001—Czerwinski et al.

21. Memory Effects & Notification • Reran book title search study correcting for navigation confounds • Removed title from top of page and added a “Remind Me” button to list • Users could use button any time • Recorded where and when users needed to be reminded of search target NRL April 2001—Czerwinski et al.

22. Methods • 16 Ss (9 female) • Intermediate to advanced PC users • All but 1 had tried IM before • 2 (marker or not) x 2 (IM or not) within subjects design • 64 search trials NRL April 2001—Czerwinski et al.

23. Results NRL April 2001—Czerwinski et al.

24. Attention-Based Principles of Notifications 1 • Unless you are absolutely sure the user wants to know what you’re telling them at that moment, be careful of very salient notifications (this is from previous work) • Autoarchive in Outlook • Frequent audio alerts from Messenger • Users’ trust is fragile. Once they perceive a system is unreliable, it is very hard to win them back (from ongoing work) • Be cautious repeating information –it might be outdated or irritating NRL April 2001—Czerwinski et al.

25. Attention-Based Principles of Notifications 2 • Make notifications situation-aware. • Look for breakpoints and pauses in users’ interactions. We’ve identified a few: Open… or Save as… dialog boxes probably good places to interrupt; typing, selecting, and other direct interactions probably bad • When possible, use smart monitoring. • Monitor the user (what are they doing?) • Content of interruption • Obvious privacy issues, etc. NRL April 2001—Czerwinski et al.

26. Adaptive Systems and Interaction Group at MSR • Complementary work on modeling and decision making for alerts going on in our group • An information-theoretic perspective with supportive infrastructure. • Work tends to rely on theories that consider direct preference assessments about outcomes. • User studies will hopefully minimize the needs for preference elicitation • Results of the work will be useful to the cost-benefit modeling, decision making work. NRL April 2001—Czerwinski et al.

27. Future Work • User models of distractibility • Better cost/benefit user models of the value of delaying information • Better UI for notifying and reminding user of what they were doing before the notification • Field studies with teens • Longitudinal studies of our beta Mobile Manager software w/cell phones NRL April 2001—Czerwinski et al.