1 / 101

Enhancing Technology-Mediated Communication Tools, Analyses, and Predictive Models

Enhancing Technology-Mediated Communication Tools, Analyses, and Predictive Models. Daniel Avrahami Human-Computer Interaction Institute School of Computer Science Carnegie Mellon University. Illustration.

xantha-beck
Download Presentation

Enhancing Technology-Mediated Communication Tools, Analyses, and Predictive Models

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. EnhancingTechnology-Mediated CommunicationTools, Analyses, and Predictive Models Daniel Avrahami Human-Computer Interaction Institute School of Computer Science Carnegie Mellon University Daniel Avrahami - Date - Location

  2. Illustration Anne is making final changes to a presentation for a client visit. Her team member John, working at a different site, sends her an instant message asking for some urgent information. Since Anne is pressed for time, she decides to ignore all incoming messages until after she’s done, leaving John unable to finish his task. Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  3. Illustration Anne is making final changes to a presentation for a client visit. Her team member John, working at a different site, sends her an instant message asking for some urgent information. Since Anne is pressed for time, she decides to ignore all incoming messages until after she’s done, leaving John unable to finish his task. tries to call her cell phone to ask calls Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  4. tries to call her cell phone to ask calls Illustration Anne is making final changes to a presentation for a client visit. Her team member John, working at a different site, sends her an instant message asking for some urgent information. Since Anne is pressed for time, she decides to ignore all incoming messages until after she’s done, leaving John unable to finish his task. sends her an email asking emails Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  5. Illustration (cont) Consider now if we were able to automatically: • Predict, based on her activity, that Anne was not likely to respond to John’s message for some time • Predict, based on past communication patterns, that Anne and John are co-workers • Identify John’s need for a response We could, for example • Increase the salience of particular communications Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  6. Research approach • An interdisciplinary approach with two primary goals: • Provide predictive statistical models and tools that enhance interpersonal communication • Predict responsiveness with accuracy as high as 90% • Provide a better understanding of human-behavior and factors that influence the use of communication tools • work-fragmentation and responsiveness Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  7. Key aspects • Responsiveness(when) • Create accurate models that predict responsiveness to incoming Instant Messages (IM), and investigate the factors affecting responsiveness • Interpersonal relationships(who) • Investigate the effect of interpersonal relationships on IM interaction, and create statistical models that use this knowledge to predict relationships • Use properties of human dialogue(what) • Responsiveness • Create accurate models that predict responsiveness to incoming Instant Messages (IM), and investigate the factors affecting responsiveness • Interpersonal relationships • Investigate the effect of interpersonal relationships on IM interaction, and create statistical models that use this knowledge to predict relationships • Use properties of human dialogue( • Use basic properties of human dialogue to provide support for balancing responsiveness and performance Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  8. Why Instant Messaging? • Instant Messaging, or IM, is one of the most popular communication mediums today • No longer a medium only for social communication • 12 billion instant messages are sent each day. • Nearly 1 billion of those are exchanged by 28 million business users [IDC Market Analysis’05] • Useful in many ways: from quick questions and clarifications, coordination and scheduling, to discussions of complex work [Bradner’99; Nardi’00; Handel’02; Herbsleb’02; Isaacs’02] Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  9. Buddy-list Message window Why Instant Messaging? • Instant Messaging, or IM, is one of the most popular communication mediums today • No longer a medium only for social communication • 12 billion instant messages are sent each day. • Nearly 1 billion of those are exchanged by 28 million business users [IDC Market Analysis’05] • Useful in many ways: from quick questions and clarifications, coordination and scheduling, to discussions of complex work [Bradner’99; Nardi’00; Handel’02; Herbsleb’02; Isaacs’02] Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  10. Instant Messaging email phone Asynchronous Synchronous Background • Some characteristics of IM: • Sending messages is “lightweight” • People can choose when/whether to respond • Asynchrony means that people can (and do) multitask [Nardi’00, Isaacs’02, Grinter’02] • Can tell whether a receiver is present • But… Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  11. Background • Especially in the workplace, means that messages may often arrive at inconvenient times • Presence is not enough • Unsuccessful communication can have a negative effect on both sender and receiver • Can disrupt the receiver’s work • Can leave the sender waiting for information • True not only for IM • Presence is not enough • Unsuccessful communication can have a negative effect on both sender and receiver • Can disrupt the receiver’s work • Can leave the sender waiting for information • True not only for IM Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  12. Predicting responsiveness to IM(when) [Avrahami & Hudson, CHI 2006] Daniel Avrahami - Date - Location

  13. Outgoing User state Time until response Responsiveness Models that predict the answer to the following: • If an instant message were to arrive right now, would the user respond to it? In how long? • Observable behavior (demonstrated availability) • “Objective” Incoming time Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  14. message awareness sender receiver How can such models help? q intercept w alert e mask r enhance Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  15. How can such models help? q q intercept w alert e mask r enhance message sender receiver Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  16. How can such models help? q q intercept w alert e mask r enhance message sender receiver Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  17. How can such models help? q q intercept w alert e mask r enhance w message sender receiver Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  18. shhhh How can such models help? q q intercept w alert e mask r enhance w awareness e sender receiver Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  19. How can such models help? q q intercept w alert e mask r enhance (carefully) r not now w awareness e sender receiver Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  20. Data collection Daniel Avrahami - Date - Location

  21. Data collection • Created a plugin for Trillian Pro (written in C) • Non-intrusive collection of IM and desktop events • Why this setup? Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  22. Data collection (cont) • What gets collected: • IM Events • Messages, status changes, etc. • Desktop Events • Applications, events, etc. • Each participants records for at least 4 weeks Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  23. Data collection (cont.) • Privacy of data • Masking messages • for example, the message:“This is my secret number: 1234 :-)” was recorded as “AAAA AA AA AAAAAA AAAAAA: DDDD :-)”. • Temporary masking • Alerting buddies • Hashing buddy-names Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  24. Participants • 16 participants • Researchers: 6 full-time employees at IBM research (mean age=40.33) • Interns: 2 summer interns at IBM research (mean age=34.5) • Students: 8 Masters students (mean age=24.5) Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  25. Participants • Nearly 5200 hours recorded • Over 90,000 messages • Over 400 buddies • 4 participants provided full text • More than doubled with new data:13359 additional hours, 34610 messages, 250 buddies • On average, participants exchanged a message every: 8.1, 2.2, 3.1 minutes (when client open) Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  26. Responsiveness Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  27. Responsiveness 50% Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  28. Responsiveness 92% 50% Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  29. session Defining “IM Sessions” 92% Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  30. session Defining “Session Initiation Attempts” used two subsets: 5 minutes (similar to Isaacs’02) and 10 minutes Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  31. Features • For every message: • Fixed number of features describing IM state. including: • Day of week • Hour • Is the Message-Window open • Buddy status (e.g., “Away”) • Buddy status duration • Time since msg to buddy • Time since msg from another buddy • Any msg from other in the last 5 minutes • log(time since msg with any buddy) • Is an SIA-5 Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  32. Features (cont.) • For every message: • Features describing desktop state(following Horvitz et al. Fogarty et al. and others). including: • Application in focus • Application in focus duration • Previous application in focus • Previous application in focus duration • Most used application in past m minutes • Duration for most used application in past m minutes • Number of application switches in past m minutes • Amount of keyboard activity in past m minutes • Amount of mouse activity in past m minutes • Mouse movement distance in past m minutes Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  33. What are we predicting? • “Seconds Until Response” • computed, for every incoming message from a buddy, by noting the time it took until a message was sent to the same buddy • Examined five responsiveness thresholds • 30 seconds, 1, 2, 5, and 10 minutes Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  34. Modeling method • Features selected using a wrapper-based selection technique • AdaBoosting on Decision-Tree models • 10-fold cross-validation • 10 trials: train on 90%, test on 10% • Next we report combined accuracy Weka ML java-based toolkit Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  35. Results (full feature-setmodels) All significantly better than the prior probability (p<.001) Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  36. Results • But…what if we want just one level of responsiveness? • e.g., to protect privacy / save face • Models that don’t use any buddy-related features • Previous models used information about the buddy (e.g., time since messaging that buddy) • Can predict different responsiveness for different buddies • But…what if we want just one level of responsiveness? • e.g., to protect privacy / save face • Models that don’t use any buddy-related features Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  37. Results (buddy-independentmodels) all significantly better than the prior probability (p<.001) BUT not sig. diff. from previous set Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  38. Practical considerations • Preserving plausible deniability • Making predictions about the receiver, visibleto the receiver • Multipleconcurrent levels of responsiveness Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  39. Understanding responsiveness We know that we can predict it. But… • What do we know about it? • What affects it? • (Can we manipulate it?) Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  40. Understanding responsiveness Daniel Avrahami - Date - Location

  41. Analysis method • The dependent measure:(the thing we are interested to see how it is influenced by other measures) • Time until response(log-transformed) • The independent measures:(those that may or may not affect the dependent measure) • Environment, computer activity, IM activity, Message characteristics [Mixed-model analysis] Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  42. Results • Simultaneous communicationresults in slower responsiveness • (on average, 11% slower per additional IM window) • Basic message-characteristics affect responsiveness • Length, URLs, questions • The relationshipwith the buddy showed significant effect on responsiveness only when the window was already open but out of focus • May explain accuracy of Buddy-independent models Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  43. Results: Work-fragmentation • Work-Fragmentation appears to be a strong indicator of faster responsiveness • More window-switching • Shorter time in focused app • More mouse movements ! Remember: we are talking about work-fragmentation before the message arrives Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  44. Results: State of the window • The state of the window when the message arrives has significant effect on responsiveness [F=1163, p<.0001; 12sec, 30, 72, 120, 190] • The effect of the visibilityof a message is stronger than that of an ongoing exchange • This may not be surprising, but is important Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  45. message awareness Summary: Responsiveness • Statistical models that accurately predict responsiveness to incoming IM based on naturally occurring behavior • Analysis that revealed major factors that influence responsiveness to IM communication • Work-fragmentation, window state, relationships - intercept - alert - mask - enhance Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  46. Related work • Interruptions and disruptions • [Gillie’89 , Cutrell’01 , Bailey’01, Hudson’02 , Dabbish’04, Mark’05, Czerwinski’05] • Interruptibility and cost of interruption • [Horvitz’99 , Horvitz’03, Hudson’03 , Begole’04, Horvitz’04, Fogarty’05, Iqbal’06] • Models of presence • [Horvitz’02, Begole’03] • Responsiveness to Email • [Horvitz’02, Tyler’03] Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  47. Relationships and communication patterns(who) [Avrahami & Hudson, CSCW 2006] Daniel Avrahami - Date - Location

  48. Related Work • Relationship type has significant effects on communication, including the quality, purpose and perceived value [Duck’91] • Cues, such as tempo, pauses, speech rates and the frequency of turns, affect the way in which conversation partners perceive each other [Feldstein’94] • Frequency affects communication [FTF:Whittaker’94, IM:Isaacs’02] Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  49. Goals • Investigate the effect of interpersonal relationships on IM interaction, and • Create statistical models that use this knowledge to classify relationships Daniel Avrahami – 28 March 2007 – Intel Research Seattle

  50. Buddy Coder Co-worker (Senior) Co-worker (Peer) Co-worker (Junior) Co-worker (Other) Friend Family Spouse Significant Other Acquaintance Friend & Co-worker Self Bot [Unknown/Unused] Daniel Avrahami – 28 March 2007 – Intel Research Seattle

More Related