1 / 66

Music

Game Optimization through Large-Scale Experimentation Erik Andersen Ph.D. student Center for Game Science University of Washington Computer Science Dept. Music. Art. Sound effects. Gameplay. ?. engagement retention revenue. A/B Testing. A/B Testing. A. B. Metrics. Levels completed

dot
Download Presentation

Music

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Game Optimization through Large-Scale ExperimentationErik AndersenPh.D. studentCenter for Game ScienceUniversity of Washington Computer Science Dept.

  2. Music Art Sound effects Gameplay

  3. ? engagement retention revenue

  4. A/B Testing

  5. A/B Testing A B

  6. Metrics • Levels completed • Time played • Return rate

  7. Aesthetics Secondary Objectives Tutorials

  8. Foldit Refraction Hello Worlds

  9. Refraction

  10. Rating: 3.8 / 5 400,000 plays

  11. Hello Worlds!

  12. Rating: 4.2 / 5 1,385,000 plays

  13. Foldit

  14. 240,000 players

  15. Player Tracking • Flash cache / login name • New players only

  16. Statistical significance 95% confidence level (p<0.05)

  17. Music Experiment #1: Audio Sound Effects

  18. Result: Music and sound effects did not matter

  19. Experiment #2: Animations

  20. Result: Animations improved engagement

  21. Refraction No Animations

  22. Refraction No Animations

  23. Refraction No Animations

  24. Hello Worlds No Animations

  25. Hello Worlds No Animations

  26. Experiment #3: Secondary Objectives (Super Mario Bros., Nintendo 1985)

  27. 100 % of players 0 Levels

  28. Hypothesis 100 With Coins % of players 0 Levels

  29. Reality 100 With Coins % of players 0 Levels

  30. What about time played? 100 % of players 0 Time

  31. Hypothesis 100 With Coins % of players 0 Time

  32. Hypothesis 100 With Coins % of players 0 Time

  33. Reality: Many Players Quit Sooner 100 With Coins % of players 0 Time

  34. 100 % of players 0 Time

  35. Result: secondary objectives harmed engagement

  36. (Assassin’s Creed, Ubisoft 2007)

  37. Maybe easier is better? (Super Mario Bros., Nintendo 1985)

  38. 100 Off-path Coins % of players 0 Time

  39. 100 On-path Coins % of players 0 Time

More Related