Summarization and Personal Information Management

1. Summarization and Personal Information Management Carolyn Penstein Rosé Language Technologies Institute/ Human-Computer Interaction Institute

2. Announcements • Reminders. Questions? • Plan for Today • Bellotti et al., 2005 • Student presentation – Gagene Gweon • Short discussion of discourse structure from Rosé et al., 1995

3. Meet your classmates!

4. Homework 2: Due Feb 19 in class

5. Homework 2: Due Feb 19 in class

6. Issues • Unanswered questions • Not enough technology? • Pace? • Amount of work? • Amount of feedback? Problem Human Behavior Solution Design Technology Problem?

7. Plan for next 3 lectures • Last Week: High level overview • Methodology focus: survey research • Specifics of email management strategies • Tuesday: In-depth look at the process of task management through email • Relationships between group dynamics and email management • Thursday: In-depth look at an email product development life-cycle in a large corporation

8. What do I really like about this work? • Grounded in user research • Needs based • Not limited to current paradigms • Criticized common IR solutions for being too limited by current paradigms • Classification, prioritization • People don’t trust it? (findings from the 90s)

9. What do I really like about this work? • Developed, tested, and refined • Nice connection with summarization • See section on Aggregation of Information

10. Gahgene Gweon

11. My role today… • i was particularly impressed with how they turned their data into meaningful quantifiable data. For instance, in figure 2, you can see that they coded different activities and broke down the percentages of each category. It is hard to come up with the categories in my experience, and they did a good job of doing that. • Say a little bit about my experience in coming up with categories

12. Analyzing interview data • Problem: What do instructors look for to evaluate student groups? • Our approach: Interview instructors -> analyze data to come up with categories • Timeline • Started to preparing questions & scheduling in June • Conducted 9 interviews late June & early July • Transcribed interviews… • Almost finalized codes late August • Still evolving

13. Our journey :How our categories came about • Process • Conducted 9 interviews late June & early July • 3 interviewers, at least 2 present for each interview • IMPORTANT: different perspectives & comprehensiveness • After each interview come up with categories based on interview data • Obtain transcripts of the interview for further inspection

14. The challenge • Categories don’t fall into clean-cut ones • Think about organizing your emails into folders! • Think about organizing 10 people’s email into folders! • How many folders? Labels? No overlapping? • Need agreement among interviewers • Need external validity from reviewers

15. Our journey :How our categories came about • Multiple iterations of categories • Communication, task, effort • Communication, ownership, participation, engagement, cohesiveness • Group vs. Individual

16. 4 participants : BOLD Process Categories Individual Group Process • Personal goal setting • Personal progress • Discussion contribution • Participation • Engagement • Team goal setting • Group progress • Group knowledge building • Division of labor • Interpersonal dynamics

17. What next? • The Challenge • Need agreement among interviewers • Need external validity from reviewers • Writing a manual for evaluating 10 categories by watching groups • For each category, have a series of questions • Participation: Did the student attend the meeting?

18. 4 participants : BOLD Process Categories Individual Group • Personal goal setting • Personal progress • Discussion contribution • Participation + Engagement • Team player • Team goal setting • Group progress • Group knowledge building • Division of labor • Interpersonal dynamics

19. Oh, one more thing.. • PLEASE don’t share this with friends yet! • We are looking for more groups to watch!

20. Back to Bellotti…

21. What do people do with email?

22. Group vs. Individual Email

23. Keeping things “in your face”

24. Task Management Behaviors

25. Thrasks • Collections of emails • Organized around a task • Visible status information • Put everything related to that task in the same place

26. Quotes related to Managing Attention • Tying emails into threads and associating those with tasks is a good way to add context to things, and centralize task related information.

27. Student Quote: Impact of Thrasking • The most useful is the 55% composed of Dialogue/discussion/negotiation & Organizing/arranging/coordinating/scheduling. • Thrasking Announcements (35%) seems less useful, since that's mostly "put X in your calendar“ • Thrasking Not-an-activity stuff is useless. • So in the best case, thrasks would help with about 90% of your email (assuming 10% is spam), and in the worst case, maybe about 50%. I wonder if people would be more likely to ignore the other 50% now, because it's not part of the thrask list.

28. Pros and Cons • Discussion about the TaskManager as just a “fat email client” • Other products solve same problems • Many information overload tools seem to suffer from similar design problems -- you want to be able to tie all your information together as easily as possible, but mandating a single tool to manage all your digital information -- emails, chat logs, calendar, todo list, contact list, etc comes with a huge barrier to entry and demands a very large scope of whatever application gets developed. What if we replace the Finder?

29. Student Quote • I am usually biased against relying on email for reminders for oneself. It doesn't seem that enforcing users to process one single email several times, and use their email client as their agenda is the most efficient way to relief information overload. • Is that what they did?

30. ? Question unanswered… • But did the design have an impact on overload or coordination?

31. Student Quote • …e-mails that involve co-ordination and collaboration with multiple people are the ones leading to a sense of overload. While this seems quite intuitive and not-so-surprising in retrospect, … • Let’s take a step back and think more about what this finding means…

32. What is the evidence? • Stories from users about how they were compensating for the difficulties of extended tasks in a fast paced environment • Source of context switching and window management • Tasks are vulnerable when they are put on hold • Might forget something • Computer might crash, and you lose work

33. Work people do to avoid forgetting

34. How is this related to the Dabbish and Kraut findings?

35. Student Quote • I think [the] major contribution of this paper is the level of detail in its analysis of threads. • This allowed the researchers to measure how many days threads last (most last more than a few days). They cite one of their earlier relevant finding that incomplete to-do items rarely get done if left longer than 2 weeks. They discuss this in conjunction with the ways widely spaced threads get pushed out of the inbox, threatening to be forgotten.

36. What do threads have to do with it? • Bellotti et al. offer evidence that when multiple email threads are managed in parallel, people feel overloaded • Is it about the complexity of the task? • Or is it about the complexity of the discourse? • Possible interpretation: Discourse structure more important than volume of messages • Has potential implications then for non-task related communication

37. What do threads have to do with it? Use patterns of anaphoric references as evidence for the underlying structure of discourse

38. What do threads have to do with it? DS 0 DS 4 DS 2 DS 1 DS 3

39. What do threads have to do with it?

40. What do threads have to do with it?

41. Length of Participation

42. Evaluation: Was it a failure? • Only 9 people participated • OK for a formative user study, not testing a hypothesis • One person returned to Outlook when he felt overwhelmed with work • Interesting finding that if you’re away from the TaskManager for a few days, getting back into it is hard • Problems with windows compatibility and missing features (printing and feedback about address completion)

43. Student Quote • I was surprised that they found “To-Do” folders to work for some of their participants, since it was shown in the Whittaker and Sidner study that these folders failed. • How do you reconcile this?

44. Findings Related to Summarization • Not all forms of aggregation turned out to be useful/successful • Very successful: Warning bars that say what is most urgent • Not very successful: action balls that aggregate “to dos” • Maybe because they don’t tell you what is most urgent • Back to attention management – what should I pay attention to now?

45. Different kinds of publications • Conference papers versus journal articles • Formative versus summative evaluations • Result papers versus insight papers • Insight papers are valuable, even if the results are not great, if the insights that are offered are new and well substantiated • What do we take away from this paper?

46. Questions?

47. Homework 2 • Project proposal • Poster will be presented in class on Feb 19 • The Problem • Potential Solution • Research questions and needs • Convince other students that they have something valuable to contribute • PowerPoint, 8inX14in, print to Spectrum or other color printer • Ask Rohit if you need help