Retaining Participants in Volunteer Computing Projects

1. Retaining Participants in Volunteer Computing Projects Peter Darch, Annamaria Carusi Oxford e-Research Centre, University of Oxford 8 December 2009 peter.darch@ccc.ox.ac.uk

2. Introduction • Volunteer computing projects • Explosion of scientific projects requiring large-scale computing capacity (Welsh et al. 2006) • Drive for greater public engagement on the part of scientists • VCPs since 1996 • In a variety of scientific fields

3. Goals of Volunteer Computing Projects • To do science • e.g. BRaTS@Home • To educate and engage the public • e.g. climateprediction.net; Rosetta@home • To achieve these, they need to recruit and retain volunteers

4. Set up in 1999, in order to study uncertainty in climate models • Became volunteer computing project, September 2003 • With the support of UK e-Science (EPSRC), now ‘by far the largest full-resolution climate modelling experiment in the world‘ (Martin et al. 2005, p. 2) • ~57 000 active users

5. A ‘work unit’ • A single climate model (1920-2080) • Online forums • Feedback from the project • ‘Credits’ • Volunteers form teams, often with own web pages and forums • League tables on climateprediction.net’s website • Screensaver showing progress of the model • Visualization

6. http://attribution.cpdn.org/images/cloudshot_win0.gif (accessed 17 November 2008)

7. Two overriding (inter-related) aims: • To produce scientific results • Results published in scientific journals, including Nature (e.g. Stainforth et al. (2005) has been cited 375 times) • To educate the public about climate science • Materials on project website • Talks in schools, universities • Also seeks to be ongoing • New projects developed

8. The case study • Data collected: • climateprediction.net online forums • Semi-structured interviews with software engineers and scientists involed in developing and running the project • Online questionnaires for project volunteers • Papers published by technoscientists

9. Retaining project volunteers • Different groups of project volunteers: • “Super-crunchers” • Those who do a great deal of work for the project • Those with little prior familiarity with scientific institutions • “Alpha-testers” • Those who test the new models and work units • These groups valuable to climateprediction.net in different ways • And have different motivations for taking part

10. The “super-crunchers” • Enjoy the prestige of doing a great deal of scientific work • Post on forums about how much they’ve done • Posts when milestones reached • Signatures in forum posts: • Signatures

11. The “super-crunchers” • Do a great deal of work for the project • ~10% of credit awarded to ~0.2% of active volunteers • ~60% of credits awarded to ~10% of volunteers • How they are retained: • Forums and teams • And credit system • Stability • Consistency

12. Those with few links to scientific institutions • Very important group from point of view of public outreach • No other apparent links to technoscientific institutions • Relatively low credit scores • Participate in only a handful of other BOINC projects, if at all • ~75% of volunteers; but <25% of credits assigned to them

13. Those with few links to scientific institutions • Generally believed by those launching VCPs that they are interested in: • Visualizations • Being educated about the science behind the VCP • More interested • Being reassured that they are making a contribution • Cumulative credit system • Regular feedback about scientific results

14. The “alpha-testers” • Important contribution to the ongoing nature of the project • Test new models and work units • 15, 20, 30 or more BOINC projects • So may not be available for testing

15. The “alpha-testers” • Little trust in the other volunteers • Belief that these volunteers know little about science • Instead, belief that volunteers are motivated primarily by credit system • Need credit system with consistent/inflexible rules

16. Conclusions • Goals of running a VCP • To do science • To engage/educate the public • Decisions to be made • Ongoing vs one-off • What to offer the volunteers • Educational material • Visualizations • Policies regarding systems of credit

17. Conclusions • Different groups of volunteers • “Super-crunchers” • Those with little prior engagement with science • “Alpha-testers” • Contribution towards different goals • Different ways of engaging and retaining them • All motivated by the belief that the project produces worthwhile science

18. Acknowledgments • Particular thanks to: • Milo Thurston & Tolu Aina, who work on running and maintaining climateprediction.net, for their time and for cyber-introductions to climateprediction.net volunteers • mo.v and Thyme Lawn, two climateprediction.net forum moderators, for advice on how to approach their online community and for promoting my project on the climateprediction.net forums • The many climateprediction.net volunteers

19. References • Martin, A., Aina, T., Christensen, C., Kettleborough, J. & Stainforth, D. (2005) ‘On two kinds of public-resource distributed computing’, Prodceedings of Fourth UK e-Science All-Hands Meeting. • Stainforth, D., Aina, T., Christensen, C., Collins, M., Faull, N., Frame, D., Kettleborough, J., Knight, S., Martin, A., Murphy, J., Piani, C., Sexton, D., Smith, L., Spicer, R., Thorpe, A. & Allen, M.. (2005). ‘Uncertainty in predictions of the climate response to rising levels of greenhouse gases.’ Nature, vol. 433, pp. 404-406. • Welsh, E., Jirotka, M. & Gavaghan, D. (2006) ‘Post-genomic science: cross-disciplinary and large-scale collaborative research and its organizational and technological challenges for the scientific research process’, Philosophical Transactions of the Royal Society A, vol. 364, no. 1843, pp. 1533-1549.