1 / 97

Enhancing Technology-Mediated Communication Tools, Analyses, and Predictive Models

Ph.D. Thesis Defense. Enhancing Technology-Mediated Communication Tools, Analyses, and Predictive Models. Daniel Avrahami Committee: Scott Hudson (Chair) Susan Fussell Robert Kraut Eric Horvitz. Six months ago (March 11 th , 3pm). Time goes by fast…. Illustration.

teige
Download Presentation

Enhancing Technology-Mediated Communication Tools, Analyses, and Predictive Models

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Ph.D. Thesis Defense EnhancingTechnology-Mediated CommunicationTools, Analyses, and Predictive Models Daniel Avrahami Committee: Scott Hudson (Chair) Susan Fussell Robert Kraut Eric Horvitz Daniel Avrahami – Dissertation Defense - 11 September 2007

  2. Six months ago (March 11th, 3pm) Daniel Avrahami – Dissertation Defense - 11 September 2007

  3. Time goes by fast… Daniel Avrahami – Dissertation Defense - 11 September 2007

  4. Illustration Anne, a senior HCI Ph.D. student, is making final changes to a presentation. John, a junior Ph.D. student, is looking for help with an early draft of his CHI paper. Since John and Anne are located in different buildings, he must choose between asking a faculty member to read the draft, or sending Anne an instant message asking for her help. Daniel Avrahami – Dissertation Defense - 11 September 2007

  5. Illustration John sends Anne an instant message asking for her help. Since Anne is pressed for time, and having been interrupted a number of times already, she decides to ignore all incoming messages until after she’s done, leaving John unable to finish his task successfully. Daniel Avrahami – Dissertation Defense - 11 September 2007

  6. Illustration John sends Anne an instant message asking for her help. Since Anne is pressed for time, and having been interrupted a number of times already, she decides to ignore all incoming messages until after she’s done, leaving John unable to finish his task successfully. tries to call Anne’s cell phone to ask calls Daniel Avrahami – Dissertation Defense - 11 September 2007

  7. tries to call Anne’s cell phone to ask calls Illustration John sends Anne an instant message asking for her help. Since Anne is pressed for time, and having been interrupted a number of times already, she decides to ignore all incoming messages until after she’s done, leaving John unable to finish his task successfully. sends Anne an email asking emails Daniel Avrahami – Dissertation Defense - 11 September 2007

  8. Illustration (cont) Consider now if we were able to automatically: • Predict, based on her activity, that Anne was not likely to respond to John’s message for some time • Predict, based on past communication patterns, that Anne and John are co-workers • Identify John’s need for a response We could, for example • Increase the salience of particular communications Daniel Avrahami – Dissertation Defense - 11 September 2007

  9. Research approach • An interdisciplinary approach with two primary goals: • Provide predictive statistical models and tools that enhance interpersonal communication • Predict responsiveness with accuracy as high as 90% • Provide a better understanding of human-behavior and factors that influence the use of communication tools • The effect of work-fragmentation on responsiveness Daniel Avrahami – Dissertation Defense - 11 September 2007

  10. Key aspects • Responsiveness(when) • Create accurate models that predict responsiveness to incoming Instant Messages (IM), and investigate the factors affecting responsiveness • Interpersonal relationships(who) • Investigate the effect of interpersonal relationships on IM interaction, and create statistical models that use this knowledge to predict relationships • Use properties of human dialogue(what) • Responsiveness • Create accurate models that predict responsiveness to incoming Instant Messages (IM), and investigate the factors affecting responsiveness • Interpersonal relationships • Investigate the effect of interpersonal relationships on IM interaction, and create statistical models that use this knowledge to predict relationships • Use properties of human dialogue( • Use basic properties of human dialogue to provide support for balancing responsiveness and performance Daniel Avrahami – Dissertation Defense - 11 September 2007

  11. Why Instant Messaging? • Instant Messaging, or IM, is one of the most popular communication mediums today • No longer a medium only for social communication • 12 billion instant messages are sent each day • Nearly 1 billion of those are exchanged by 28 million business users [IDC Market Analysis’05] • Useful in many ways: from quick questions and clarifications, coordination and scheduling, to discussions of complex work [Bradner’99; Nardi’00; Handel’02; Herbsleb’02; Isaacs’02] Daniel Avrahami – Dissertation Defense - 11 September 2007

  12. Buddy-list Message window Why Instant Messaging? • Instant Messaging, or IM, is one of the most popular communication mediums today • No longer a medium only for social communication • 12 billion instant messages are sent each day • Nearly 1 billion of those are exchanged by 28 million business users [IDC Market Analysis’05] • Useful in many ways: from quick questions and clarifications, coordination and scheduling, to discussions of complex work [Bradner’99; Nardi’00; Handel’02; Herbsleb’02; Isaacs’02] Daniel Avrahami – Dissertation Defense - 11 September 2007

  13. Instant Messaging email phone Asynchronous Synchronous Background • Some characteristics of IM: • Sending messages is “lightweight” • People can choose when/whether to respond • Asynchrony means that people can (and do) multitask [Nardi’00, Isaacs’02, Grinter’02] • Can tell whether a receiver is present • But… Daniel Avrahami – Dissertation Defense - 11 September 2007

  14. Background • Especially in the workplace, means that messages may often arrive at inconvenient times • Presence is not enough • Unsuccessful communication can have a negative effect on both sender and receiver • Can disrupt the receiver’s work • Can leave the sender waiting for information • True not only for IM • Presence is not enough • Unsuccessful communication can have a negative effect on both sender and receiver • Can disrupt the receiver’s work • Can leave the sender waiting for information • True not only for IM Daniel Avrahami – Dissertation Defense - 11 September 2007

  15. Predicting responsiveness to IM(when) [Avrahami & Hudson, CHI 2006] Daniel Avrahami – Dissertation Defense - 11 September 2007

  16. Outgoing User state Time until response Responsiveness Models that predict the answer to the following: • If an instant message were to arrive right now, would the user respond to it? In how long? • Observable behavior • “Objective” Incoming time Daniel Avrahami – Dissertation Defense - 11 September 2007

  17. message awareness sender receiver How can such models help? q intercept messages before they are delivered w alert the receiver to important messages e hide the receiver r enhance awareness indicators Daniel Avrahami – Dissertation Defense - 11 September 2007

  18. Data collection Daniel Avrahami – Dissertation Defense - 11 September 2007

  19. Data collection • Created a plugin for Trillian Pro (written in C) • Non-intrusive collection of IM and desktop events • Each participant records for at least 4 weeks • Why this setup? Daniel Avrahami – Dissertation Defense - 11 September 2007

  20. Participants • 19 participants • Researchers: 6 full-time employees at IBM research (mean age=40.33) • Interns: 2 summer interns at IBM research (mean age=34.5) • Students: 8 Masters students (mean age=24.5) • NEW: Startup: 3 employees at a local startup(mean age=32) Daniel Avrahami – Dissertation Defense - 11 September 2007

  21. Participants • Nearly 6,600 hours recorded • Over 126,000 messages • Over 400 buddies • 7 participants provided full text • On average, participants exchanged a message every: 8.1, 2.2, 3.1, 2.4 minutes (when client open) Daniel Avrahami – Dissertation Defense - 11 September 2007

  22. Responsiveness Daniel Avrahami – Dissertation Defense - 11 September 2007

  23. Responsiveness 50% Daniel Avrahami – Dissertation Defense - 11 September 2007

  24. Responsiveness 90% 50% Daniel Avrahami – Dissertation Defense - 11 September 2007

  25. session Defining “IM Sessions” 90% Daniel Avrahami – Dissertation Defense - 11 September 2007

  26. session Defining “Session Initiation Attempts” used two subsets: 5 minutes (similar to Isaacs’02) and 10 minutes Daniel Avrahami – Dissertation Defense - 11 September 2007

  27. Features • For every message: • Fixed number of features describing IM state. including: • Is the Message-Window open • Buddy status (e.g., “Away”) • Buddy status duration • Time since msg to buddy • Time since msg from another buddy • Any msg from other in the last 5 minutes • log(time since msg with any buddy) • Is an SIA-5 • Day of week • Hour Daniel Avrahami – Dissertation Defense - 11 September 2007

  28. Features (cont.) • For every message: • Features describing desktop state(following Horvitz et al. Fogarty et al. and others) including: • Application in focus • Application in focus duration • Previous application in focus • Previous application in focus duration • Most used application in past m minutes • Duration for most used application in past m minutes • Number of application switches in past m minutes • Amount of keyboard activity in past m minutes • Amount of mouse activity in past m minutes • Mouse movement distance in past m minutes Daniel Avrahami – Dissertation Defense - 11 September 2007

  29. What are we predicting? • “Seconds Until Response” • computed, for every incoming message from a buddy, by noting the time it took until a message was sent to the same buddy • Examined five responsiveness thresholds • 30 seconds, 1, 2, 5, and 10 minutes Daniel Avrahami – Dissertation Defense - 11 September 2007

  30. Modeling method • Features selected using a wrapper-based selection technique • AdaBoosting on Decision-Tree models • 10-fold cross-validation • 10 trials: train on 90%, test on 10% • Next I report combined accuracy Weka ML java-based toolkit Daniel Avrahami – Dissertation Defense - 11 September 2007

  31. Results (full feature-setmodels) All significantly better than the prior probability (p<.001) (F-measures for less frequent class all around 0.8) SIA-5 models Daniel Avrahami – Dissertation Defense - 11 September 2007

  32. Results • But…what if we want just one level of responsiveness? • e.g., to protect privacy / save face • Models that don’t use any buddy-related features • Previous models used information about the buddy (e.g., time since messaging that buddy) • Can predict different responsiveness for different buddies • But…what if we want just one level of responsiveness? • e.g., to protect privacy / save face • Models that don’t use any buddy-related features Daniel Avrahami – Dissertation Defense - 11 September 2007

  33. Results (buddy-independentmodels) all significantly better than the prior probability (p<.001) BUT not sig. diff. from previous set Daniel Avrahami – Dissertation Defense - 11 September 2007

  34. Using old data to train models for new Comparing models trained on old data vs. new data(testing done on new data) SIA-10 models Daniel Avrahami – Dissertation Defense - 11 September 2007

  35. Forecasts of responsiveness Daniel Avrahami – Dissertation Defense - 11 September 2007

  36. waited additional wait IM Sent Query time Exploring forecasts of IM responses • Models presented earlier predict responsiveness before a message is sent • Consider the case where a user has already sent a message and is now waiting for a response • May wish to know, given that they have already been waiting for some time, the likelihood that a response will (or will not) arrive within some time period. P(response) [similar to Horvitz’02] Daniel Avrahami – Dissertation Defense - 11 September 2007

  37. Exploring forecasts of IM responses Likelihood of receiving a responsehaving already waited T Daniel Avrahami – Dissertation Defense - 11 September 2007

  38. Exploring forecasts of IM responses Likelihood of receiving a response within 2 minutes, having already waited T 50% 19% Daniel Avrahami – Dissertation Defense - 11 September 2007

  39. Understanding IM responsiveness Daniel Avrahami – Dissertation Defense - 11 September 2007

  40. Understanding IM responsiveness We know that we can predict it. But… • What do we know about it? • What affects it? • (Can we manipulate it?) Daniel Avrahami – Dissertation Defense - 11 September 2007

  41. Analysis method • The dependent measure:(the thing we are interested in seeing how it depends on other measures) • Time until response(log-transformed) • The independent measures:(those that may or may not affect the dependent measure) • Context(IM, Desktop) • Communication (Relationship, Time since last comm.) • Content (Length, Question, URL, Emoticon) • Control (Gender, Age, Group, lag) [Mixed-model analysis] Daniel Avrahami – Dissertation Defense - 11 September 2007

  42. Results Spoiler Alert!! Daniel Avrahami – Dissertation Defense - 11 September 2007

  43. Results • Simultaneous communicationresults in slower responsiveness, but only for messages that arrive in a window that is out-of-focus(17% slower on average) • The relationshipwith the buddy did not show a significant effect on responsiveness • (although significant differences in responsiveness to different individuals) Daniel Avrahami – Dissertation Defense - 11 September 2007

  44. Results: Work-fragmentation • Work-fragmentation appears to be a strong indicator of faster responsiveness • More window-switching • Shorter time in focused app • More mouse movements ! Remember: we are talking about work-fragmentation before the message arrives Daniel Avrahami – Dissertation Defense - 11 September 2007

  45. ? ? ? Results: Content • Measures of content were found to have significant effect on responsiveness: • Faster responses to messages with a question(55s vs. 89s) • Slower responses to messages with a URL(103s vs. 48s) • Slower responses to messages with an emoticon(74s vs. 67s) [Related to Burke et al. 2007] Daniel Avrahami – Dissertation Defense - 11 September 2007

  46. Results: State of the window • The state of the window when the message arrives has significant effect on responsiveness [F=560, p<.001; 24sec, 55, 91, 123, 156] • (The effect of the visibilityof a message was stronger than that of an ongoing exchange) • This may not be surprising, but is important Daniel Avrahami – Dissertation Defense - 11 September 2007

  47. message awareness Summary: Responsiveness • Statistical models that accurately predict responsiveness to incoming IM based on naturally occurring behavior • Explored forecasts of responses • Analysis that revealed major factors that influence responsiveness to IM communication • Work-fragmentation, window state, multiple communication > - intercept - alert - hide - enhance Daniel Avrahami – Dissertation Defense - 11 September 2007

  48. Related work • Interruptions and disruptions • [Gillie’89 , Cutrell’01 , Bailey’01, Hudson’02 , Dabbish’04, Mark’05, Czerwinski’05] • Interruptibility and cost of interruption • [Horvitz’99 , Horvitz’03, Hudson’03 , Begole’04, Horvitz’04, Fogarty’05, Iqbal’06] • Models of presence • [Horvitz’02, Begole’03] • Responsiveness to Email • [Horvitz’02, Tyler’03] Daniel Avrahami – Dissertation Defense - 11 September 2007

  49. Relationships and communication patterns(who) [Avrahami & Hudson, CSCW 2006] To Summary Daniel Avrahami – Dissertation Defense - 11 September 2007

  50. Related work • Relationship type has significant effects on communication, including the quality, purpose and perceived value [Duck’91] • Cues, such as tempo, pauses, speech rates and the frequency of turns, affect the way in which conversation partners perceive each other [Feldstein’94] • Frequency of communication affects communication [FTF:Whittaker’94, IM:Isaacs’02] Daniel Avrahami – Dissertation Defense - 11 September 2007

More Related