Enhancing Instant Messaging Responsiveness Through Predictive Modeling

1. Responsiveness in IM: Predictive Models Supporting Inter-Personal Communication Daniel Avrahami, Scott E. HudsonCarnegie Mellon University www.cs.cmu.edu/~nx6

2. Q: if an instant message were to arrive right now, would the user respond to it? in how long? • collected field data • 5200 hours • 90,000 messages • IM and desktop events • models predicting responsiveness • as high as 90.1%

3. why should we care?

4. why should we care? • IM is one of the most popular communication mediums • no longer a medium just for kids (work / parents) • sending messages is “cheap” but the potential for interruptions is great • unsuccessful communication can have a negative effect on both sender and receiver • can disrupt the receiver’s work • can leave the sender waiting for information • true not only for IM

5. message awareness sender receiver how can such models help? q intercept w alert e mask r enhance

6. how can such models help? q q intercept w alert e mask r enhance message sender receiver

7. how can such models help? q q intercept w alert e mask r enhance message sender receiver

8. how can such models help? q q intercept w alert e mask r enhance w message sender receiver

9. shhhh how can such models help? q q intercept w alert e mask r enhance w awareness e sender receiver

10. how can such models help? q q intercept w alert e mask r enhance (carefully) r not now w awareness e sender receiver

11. related work • instant messaging • [Nardi’00 , Isaacs’02 , Voida’02] • interruptions and disruptions • [Gillie’89 , Cutrell’01 , Hudson’02 , Dabbish’04] • models of presence and interruptibility • [Horvitz’02 , Begole’02 , Hudson’03 , Begole’04, Horvitz’04 , Fogarty’05 , Iqbal’06]

12. coming up… • data collection • participants • responsiveness overview • predictive models • how (features and classes) • results • a closer look (new! not in the paper) • future work

13. data collection • a plugin for Trillian Pro (written in C) • non-intrusive collection of IM and desktop events

14. data collection (cont.) • privacy of data • masking messages • for example, the message “This is my secret number: 1234 :-)” was recorded as “AAAA AA AA AAAAAA AAAAAA: DDDD :-)”. • alerting buddies • hashing buddy-names • 4 participants provided full content

15. participants • 16 participants • Researchers: 6 full-time employees at an industrial research lab (mean age=40.33) • Interns: 2 summer interns at the industrial research lab (mean age=34.5) • Students: 8 Masters students (mean age=24.5) • nearly 5200 hours recorded • over 90,000 messages

16. responsiveness 50%

17. responsiveness 92% 50%

18. session defining “IM Sessions” 92%

19. session defining “Session Initiation Attempts” used two subsets: 5 minutes (similar to Isaacs’02) and 10 minutes

20. features • for every message: • features describing IM state. including: • Day of week • Hour • Is the Message-Window open • Buddy status (e.g., “Away”) • Buddy status duration • Time since msg to buddy • Time since msg from another buddy • Any msg from other in the last 5 minutes • log(time since msg with any buddy) • Is an SIA-5

21. features (cont.) • for every message: • features describing desktop state (following Horvitz et al. Fogarty et al. and others). including: • Application in focus • Application in focus duration • Previous application in focus • Previous application in focus duration • Most used application in past m minutes • Duration for most used application in past m minutes • Number of application switches in past m minutes • Amount of keyboard activity in past m minutes • Amount of mouse activity in past m minutes • Mouse movement distance in past m minutes

22. what are we predicting? • “Seconds until Response” • computed, for every incoming message from a buddy, by noting the time it took until a message was sent to the same buddy • examined five responsiveness thresholds • 30 seconds, 1, 2, 5, and 10 minutes

23. modeling method • features selected using a wrapper-based selection technique • AdaBoosting on Decision-Tree models • 10-fold cross-validation • 10 trials: train on 90%, test on 10% • next we report combined accuracy

24. results

25. results (full feature-set models) all significantly better than the prior probability (p<.001)

26. results (user-centric models) • previous models used information about the buddy (e.g., time since messing that buddy) • can predict different responsiveness for different buddies • but what if you wanted just one level of responsiveness? • built models that did not use any buddy-related features

27. results (user-centric models) all significantly better than the prior probability (p<.001)

28. a closer look(new! not in the paper)

29. a closer look (new! not in the paper) • analysis of the continuous measure: • log(Time Until Response) • repeated measures ANOVA • Independent Variables: features subset • ParticipantID [Group] as random effect

32. “those in the back can’t see, and those in the front can’t understand…” Robert Kraut

33. a closer look (new! not in the paper) • work fragmentation • longer time in previous app …. slower • more switching (30sec) …. faster • longer mouse movements (60sec) …. faster • more keyboard activity (30 sec) …. faster • more message windows …. slower • longer time since messaging with buddy… faster • buddy ID had significant effect

34. implications for practice(in the paper)

35. implications for practice • preserving plausible deniability • making predictions about the receiver, visibleto the receiver • multiple concurrent levels of responsiveness

36. message awareness summary & future work • presented statistical models that accurately predict responsiveness to incoming IM based on naturally occurring behavior • we plan to examine using message-content to improve modeling • intercept • alert • mask • enhance

37. we would like to thank • Mike T (Terry) • James Fogarty • Darren Gergle • Laura Dabbish, and • Jennifer Lai

38. for more info visit:www.cs.cmu.edu/~nx6 or email:nx6@cmu.edu thank you this work was funded in part by NSF Grants IIS-0121560, IIS-0325351, and by DARPA Contract No. NBCHD030010