Flash Forums and ForumReader: Crafting an Interface for a New Kind of Online Discussion

1. Flash Forums and ForumReader:Crafting an Interface for a New Kind of Online Discussion Kushal Dave, kushal@google.com* Martin Wattenberg, mwatten@us.ibm.com Michael J. Muller, michael_muller@us.ibm.com IBM Research / Collaborative User Experience Cambridge, MA USA *Work done at IBM

2. Outline • Flash Forums • What are they? • Why do they matter? • ForumReader • Design decisions • Demo • User feedback • Jam trial • Lab study • Conclusions

3. Reading 892 of these is hard!

4. Flash forum examples • Slashdot • “News for nerds” portal • Several articles discussed daily • Hundreds of posts per topic within a day • IBM Jams • Company-wide discussions • Several broad forums • Thousands of posts over 3 days • Blog comments, news discussions, et al. • As much text as a small novel

5. Flash Forums In contrast to Usenet and other ongoing forums… • Diffuse authorship • Large size • Focused topic • Short duration • Cf. flash mobs: “a large group of people who gather in a usually predetermined location, perform some brief action, and then quickly disperse” [wordspy.com] • Often, shallower threads

6. Flash forums are less conversational Flash forums 90.00 80.00 Usenet 70.00 Slashdot Jam 60.00 50.00 Percent of authors posting more than once 40.00 30.00 20.00 10.00 0.00 0.00 20.00 40.00 60.00 80.00 100.00 Percent of messages that are replies

7. Flash forum threads are shallower VJam (Role) VJam (Impact) Flash forums Slashdot (MySQL) Slashdot (Windows) Usenet (Linux) Usenet (Prog) 70 60 50 40 Percent of messages 30 20 10 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 Thread depth

8. Why diffuse authorship matters • Authors are relevant… • Some known users on Slashdot • Some key personalities in IBM • Metadata about users (reputation, seniority) provide cues • …but ideas become focus • Users indicate ideas trump authors (unlike Usenet) • Official and distributed moderation (try to) create meritocracy

9. Why time limits matter “A Jam’s authenticity is derived from the fact that it’s a real-time and finite event, and that there are real, often serendipitous ‘knowledge accidents’ among participants that emerge because of the time constraint imposed.” [Dorsett, Fontaine, O’Driscoll]

10. Why size and topic matter • Many simultaneous posts, constrained topic, and shallow threads lead to: • Thread drift • Particular themes split across threads • Diminished utility of threads as filters

11. Discussion interfaces • Much work on Usenet • Conversation Map • Netscan • Our problem is different • Authors are inadequate filters • Threads are inadequate filters • Everything happens at once • Basically, the discussion is one big mess

12. ForumReader • Easily move around the discussion • Many ways to enter discussion • Sense of orientation • Integrate visualization and text analytics • Builds on existing work in thumbnail interfaces • SeeSoft/SeeSys, Reader’s Helper, Context Lens

13. DEMO!

14. Jam feedback • 8,973 posts, 22,000 participants • Survey of 1,248 participants: • 16% used ForumReader successfully • Important (3.5 / 5) and satisfying (3.2 / 5) • Value in orientation • Ability to find themes came up repeatedly “Amazing. To be able to locate commonalities, etc., and analyze the worth of this VALUABLE effort IS GREAT!”

15. But we still didn’t know… • How do users navigate discussions? What cues do they use? • Does the interface really help them understand the scope of discussions and find information? Which features are most valuable?

16. Lab study design 2x2 varying visualization, text analysis Data Collected • Describe expectation • Explore discussion • Identify key arguments • Generate mindmap or outline • Indicate relative amounts of discussion of topics • Argue for or against • Evaluation Visualization Y N 4 4 Text Analytics N Y 4 4

17. Preference Users consider map, search and moderation highlighting as valuable as scrollbar and text view.

18. Performance • Visualization and text analytics might improve performance individually, but detract together. Purple: Text analytics Blue: No text Left columns: No viz. Right columns: Visualization F(1,12)=1.95 N.S. F(1,12)=6.57 p<.03 F(1,12)=5.20 P<.04 F(1,12)=4.26 P<.05

19. Navigation patterns Users used the map (red dots) extensively, often nonlinearly. Rows are conditions: map (top), NLP (second), control (third), map + NLP (bottom) Dots show navigation events: scrollbar/arrow keys (blue), map (red), tree (cyan) Lines show click search (black), typed search (cyan), highlighting (all others)

20. Reading patterns Users spent more time viewing starts of threads, highly-moderated posts, and starts of discussions.

21. Summary • Flash forums present novel structure and dynamics (diffuse, big, focused, short) • Users value cues like readwear, moderation, threads, authors (especially themselves!) • Navigation is idiosyncratic • Our visualization and text analytics appear to help users see more of the discussion • But too much complexity may be distracting

22. Future Work • Much more to learn about flash forums • Textual analysis • User goals • Moderation systems • Opportunities for better interfaces • Multi-dimensional filtering • Anti-filtering: emphasize novelty, variety

23. Thanks!