1 / 57

If You Build It? Benefits and Costs of Creating Your Own Online Community

If You Build It? Benefits and Costs of Creating Your Own Online Community. Loren Terveen Computer Science & Engineering The University of Minnesota August 2011. Background: ways of knowing. Theory Simulation Lab studies Surveys Qualitative studies Build and learn

sine
Download Presentation

If You Build It? Benefits and Costs of Creating Your Own Online Community

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. If You Build It? Benefits and Costs of Creating Your Own Online Community Loren Terveen Computer Science & Engineering The University of Minnesota August 2011

  2. Background: ways of knowing • Theory • Simulation • Lab studies • Surveys • Qualitative studies • Build and learn • (e.g., Google, Facebook, Wikipedia) • Build To Learn

  3. Build to learn • GroupLens Research • Create new interaction / social computing techniques • Do empirical, quantitative research • Learn from what we and others build

  4. To answer the kinds of research questions we like to ask, we need: • Data • Experimental Control

  5. The rest of the talk • Learning from others’ data • Learning from our own data • Exercising experimental control

  6. 1. Learning from others’ data • Q&A systems • Wikipedia

  7. GroupLens Wikipedia Research • WP:Clubhouse? An Exploration of Wikipedia’s Gender Imbalance. Lam, S.K., Uduwage, A., Dong, Z., Sen, S., Musicant, D.R., Terveen, L., Riedl, J. WikiSym 2011. • NICE: Social translucence through UI intervention. A. Halfaker, B. Song, D. A. Stuart, A. Kittur and J. Riedl. Wikisym 2011. • Don't bite the Newbies: How Reverts Affect the Quantity and Quality of Wikipedia Work. A. Halfaker, A. Kittur and J. Riedl. Wikisym2011. • Mentoring in Wikipedia: A Clash of Cultures. D. Musicant, Y. Ren, J. Johnson and J. Riedl. Wikisym2011. • The Effects of Group Composition on Decision Quality in a Social Production Community, Lam, S.K., Karim, J., Riedl, J. Group 2010. • The Effects of Diversity on Group Productivity and Member Withdrawal in Online Volunteer Groups, Chen, J., Ren, Y., Riedl, J. CHI 2010. • rv you're dumb: Identifying Discarded Work in Wiki Article History, Ekstrand, M.D., Riedl, J.T. Wikisym 2009. • A Jury of Your Peers: Quality, Experience and Ownership in Wikipedia, Halfaker, A., Kittur, N., Kraut, R., Riedl, J. Wikisym 2009. • Is Wikipedia Growing a Longer Tail?, Lam, S.K., Riedl, J. Group 2009. • Wikipedians are born, not made: a study of power editors on Wikipedia, Panciera, K., Halfaker, A., Terveen, L. Group 2009. • SuggestBot: Using Intelligent Task Routing to Help People Find Work in Wikipedia, Cosley, D., Frankowski, D., Terveen, L., Riedl, J. IUI 2007. • Creating, Destroying, and Restoring Value in Wikipedia, Priedhorsky, R., Chen, J., Lam, S.K., Panciera, K., Terveen, L., Riedl, J. Group 2007.

  8. WP:Clubhouse? An Exploration of Wikipedia’s Gender Imbalance. Lam, S.K., Uduwage, A., Dong, Z., Sen, S., Musicant, D.R., Terveen, L., Riedl, J. • www.grouplens.org/node/466

  9. Trigger… • http://www.nytimes.com/2011/01/31/business/media/31link.html?_r=1&src=busln • A topic generally restricted to teenage girls, like friendship bracelets, can seem short at four paragraphs when compared with lengthy articles on something boys might favor, like, toy soldiers or baseball cards, whose voluminous entry includes a detailed chronological history of the subject. • (BTW, it’s not about the friendship bracelets)

  10. Findings • Only 16% of new editors joining Wikipedia during 2009 identified themselves as women • Women made only 9% of the edits by this cohort • New women editors are more likely to stop editing and leave Wikipedia when their edits are reverted • Topics of particular interest to women appear to get less (and poorer) coverage in Wikipedia • (Hmm… maybe Wikipedia has a low collective IQ!) • Come to Wikisym to get the details!

  11. 2. Learning from our own data

  12. GroupLens online communities • MovieLens • Cyclopath

  13. 200 Union St SE Lagoon Theatre

  14. Research Question • How do contributors to open content systems become contributors? • Inspired by…

  15. Becoming WikipedianBryant, Forte, & Bruckman 2005 • Wikipedians fill different niches than non-Wikipedians • Wikipedians branch out to new areas and topics as they mature • Wikipedians take on more “community work” as they mature Qualitative study with nine participants self-reporting

  16. Our goal: test these findings quantitatively Evidence for “becoming”? Quantity of work Quality of work Nature of work Are Wikipedians Born or Made?

  17. Wikipedian A registered editor with 250+ edits over his/her lifetime If editors reach 250 edits within our data set, they are labeled Wikipedian from the beginning

  18. Data English Wikipedia dump (January 13, 2008) Edits from bots and other non-human means removed We counted: Only registered editors Wikipedians (users with 250+ edits) - 38K Non-wikipedians - random sample of 38K Edits per day per editor (“User days”) (“Day 1”)

  19. Quantity

  20. Is a user’s fate sealed? Wikipedians are Born Made Quantity

  21. Quality • Measure: Persistent Word Revisions (PWRs) • Proportion of words added that persist five revisions

  22. Other quality metrics? Wikipedians are Born Made Quality

  23. Nature of Work • Conjecture: Wikipedians take on community maintenance work over time • Several ways to formalize • Editing in “talk” (and other) namespaces • (Nope: still “born”) • Referring to “community norms” (Wikipedia policies) to explain edits

  24. Community Learning norms vs. learning to appeal to the norms? Training: effective editing Wikipedians are Born Made

  25. Summary of findings • Common pattern: Initial burst of activity, decline, steady state • Wikipedians look different from day one • Little evidence for “Becoming Wikipedian”: Wikipedians are born, not made • Can we reconcile? • This is depressing! • Possible responses: • Early interventions • Change the culture • Systemic initiatives, e.g., APS Wikipedia Initiative: http://www.psychologicalscience.org/index.php/members/aps-wikipedia-initiative • Accept the reality of the long tail

  26. But: methodological worries • We can’t ask Wikipedia users about our interpretations • What if the learning happened before users registered?

  27. Cyclopath: viewing and pre-registration activities are visible • As of September 2009, we identified: • 1172 “unambiguous” users • 268 of these users made some edits • 440 “ambiguous” users • For unambiguous users • Day 1 = First time a user came to the site (not the day they registered)

  28. Same pattern as for Wikipedia

  29. And few users edited before registration

  30. Some viewing before registration A minute or two <= 5 min. <= 15 <= 30 <= 60

  31. But amount of viewing before registration (or before editing) does not predict subsequent behavior “Born, Not Made” still seems true

  32. Followups • Cyclopath user surveys – Wikisym 2011 paper • Why these patterns? • What ‘triggers’ initial contribution? • And how might we nurture ongoing participation? • Cyclopath contextual interviews • planned

  33. 3. Exercising Experimental Control

  34. Research Question • Motivating participation: How can we get more work done in open content systems? • Idea: match users with tasks they’re likely to be interested in and capable of doing • Requirements: • Introduce tasks matching algorithms/interfaces • Assign users to different conditions • Gather data necessary for evaluation • Survey users

  35. Tools Recommender algorithms Interaction design Intelligent Task Routing Goals Get work done Nurture new users Serve community Theory Collective Effort Model Social Influence

  36. MovieLens Task: Edit movie content

  37. Four strategies to suggest movies to a user theory-based

  38. The experiment • Assign ML users to four groups, one per algorithm • About 2,000 subjects, 200 contributors • Count # editors, contributions, fields

  39. Rare rated: dominant Needs work: bang for buck Random: not bad here High prediction: lousy

More Related