The age of virtual economy

1. Evaluating the Effectiveness of a Virtual Economy System Within an Advanced Scientific CyberinfrastructureDirected Project, WMP in TechnologyA. Nedossekina. Chair: Dr. M. Sutton

2. The age of virtual economy A system of economic activities of users within a virtualcommunity. Total value of assets in virtual worlds (including online games) approaching US$2 billion MMOG, virtual gaming communities trading virtual goods Social network sites where virtual trading & rewards help build communities A virtual marketplace for real goods and real money Grid computing projects using market-based resource allocation

3. nanoHUB.org Online simulation and more Serving a scientific community of over 90,000 users around the globe

4. nanoHUB.org Online simulation and more A resource to the entire nanotechnology discovery and learning community 819 resource reviews Over 1400 “and more” resources 137 simulation tools (plus 97 in development) 433 questions & answers

5. nanoHUB.org Online simulation and more Resources come from over 600 contributors, community rates them and asks questions

6. Problem: low contribution rates The site has over 24,000 total registered users, but on average only about 2% contribute < 2% of registered users contributed a review 2.5% of registered users contributed a resource < 1% of registered users participated in the Answers forum

7. Solution: virtual economy Introducing virtual reward points as a way to motivate contributions Earn by contributing Spend on merchandise and high-end services

8. nanoHUB virtual economy Resources Reviews Questions & Answers … Merchandise Faster computation cycles More data storage More interactive sessions …

9. nanoHUB virtual economy Goal: to help virtual community growth and sustainability

10. nanoHUB virtual economy Current development stage Resources Reviews Questions & Answers … Merchandise Faster computation cycles More data storage More interactive sessions …

11. 2002 Jun 2007 Apr 2008 nanoHUB started Answers forum launched Points introduced

12. What was the impact on user contribution activity? Did the system change user behavior? Do users contribute more than before? ? Why? To improve current system To help future development To know users better Need for an evaluation model

13. What was the impact on user contribution activity? Our project looked for an answer in usage data, measuring impact on activity in the Answers forum Jun 2007 Apr 2008 ? PRE 2007 Jul – Sep (A) Oct – Dec (B) Jul – Dec (C) POST 2008 Jul – Sep (A) Oct – Dec (B) Jul – Dec (C)

14. Closer look at our users PRE POST Simulation users Minimal Average Heavy Expert 0 300 400 3000 100 200 Number of simulation runs

15. What percentage of users contributed Q&A? Percentage of users who contributed Q&A increased in all categories.

16. How big is the change in percentage of users who contributed Q&A? 8.3 increase among Heavy simulation users!

17. What are the ratios of Q&A per user? Ratios of Q&A contributions per user increased in all categories, With expert users having the highest Q&A/user ratios

18. How big is the change in the ratios of Q&A per user? 9.6 increase among Heavy simulation users!

19. How does the impact on Q&A activity compare with changes in other contribution activities? Change in percentage of contributing users Q&A Reviews Resources

20. How does the impact on Q&A activity compare with changes in other contribution activities? Change in percentage of contributing users Change in ratio of contributions per user Q&A Reviews Resources The impact on Q&A contribution activity among Heavy users is obvious!

21. Research process Data collection Data analysis Descriptive statistics Correlation analysis Ratio analysis in user categories Spearman rank correlation between contribution and usage levels

22. Correlation analysis examined the relationship between usage level and contribution activity usage nanoHUB contributions Data for each user in a group: Usage N of simulation runs N of sessions CPU Session time N of downloads N of web clicks N of unique module views Login time N of logins Correlation? Contributions N of Q&A N of resources N of reviews Simulation usage level Non-simulation usage level • Anderson-Darling Test for normality • non-normal distribution • need Spearman Rank • Spearman Rank correlation (Pearson correlation on ranked data) • Correlation coefficients compared in PRE & POST for several user groups

23. Correlation analysis showed increase of positive correlation between usage levels and Q&A contributions in POST periods for ALL studied user groups Yet, the correlation coefficients were statistically insignificant (<0.3)

24. Conclusions about the evaluation model Two data analysis approaches utilized Correlation analysis Ratio analysis in user categories Taking advantage of complex statistical tests Looking at effect on overall population Unbiased assessment Robust assessment Comprehensive comparison of pre and post situation Looking at impact on different user categories Complex procedures Studied impact saturated Arbitrary user classification, thus more bias May appear oversimplified Both methods provided some valuable insight into user behavior Both may be used for future system assessment

25. What’s next? Expand system to earn points on reviews Tie Q&A with non-simulation resources to increase participation of non-simulation, minimal and average simulation users Repeat assessment when new components are deployed: both for testing of evaluation approaches & for new insight about impact of system on user behavior May publish in Journal of Virtual Worlds Research Special issue on virtual economies Deadlines: Abstract - June15, 2009.
Full manuscript - November 1, 2009
Publication Date: December 15, 2009

26. The end.

27. Studied User Groups