Using Game Elements to Teach Computer Science - Jesse Hartloff

1. USING GAME ELEMENTS TO TEACH COMPUTER SCIENCE ‘- Dr. Jesse Hartloff hartloff@buffalo.edu 1

2. The App • • • Introductory Computer Science course Procedurally generated problem sets Questions based on level and prior performance ‘- • Earn points by answering question correctly • No punishment for incorrect answers • 100% Automated grading 2

3. BUT FIRST… ‘- How did we get here? 3

4. My [Gaming] Background ‘- 4

5. My [Gaming] Background ‘- 5

6. My [Gaming] Background ‘- 6

7. Can education be as compelling as games? ‘- 7

8. THE PROBLEM ‘- What are we trying to solve? 8

9. Low Motivation • Students are not motivated to learn the material • Many come to Computer Science for the jobs • Some non-majors who are “forced” to take our intro course • Computer Science can be dry ‘- 9

10. Difficulty – Application of Material • • There is very little memorization Mostly at the level of Apply on Bloom’s Taxonomy with implicit Understanding • Many students are not prepared for this in their first semester Common complaint • This is way too hard for a 100- level course ‘- • 10

11. Difficulty – Cumulative Material • Most topics in the course require the students to understand most of the previous topics Ex. You cannot apply data structures in a program without understanding variables, expressions, and functions Similar to a math classes • You can’t learn algebra without understanding arithmetic Students like to skip hard topics • ‘- • • 11

12. Academic Integrity • Typically 4-7% are caught are convicted each semester • Presumably many more don’t get caught • Difficult to access the full scale of this problem ‘- 12

13. Scale • This Spring there were 350 student in the intro course • First semester using game elements Last Fall: 650 students This Fall: 800 students expected across 4 sections Fall Support • Co-taught by 2 faculty members (Dr. Carl Alphonce and myself) • 30 undergraduate teaching assistants working 10 hours/week • • • ‘- 13

14. POTENTIAL SOLUTION? ‘- Badges & Leaderboards 14

15. Badges • • Can be motivating Ex. Xbox achievements can be great • They provide new challenges and goals in games that you already want to play ‘- • Adding badges to an app does not always motivate a user • Should be added with a specific purpose to be effective Badges in the Audible mobile app 15

16. Leaderboards I have tried this in upper level course • Only motivates the top students (~5- 10%) • ‘- Does nothing for the strugling students • In some cases it can demoralize them • Majority of students just want to get through the assignment and shift their focus • Runtime leaderboard in a 300-level algorithms course 16

17. Badges, Leaderboards, etc. • Where are they? • Not in my games. ‘- 17

18. A BETTER APPROACH ‘- What can we learn from games 18

19. Common Features of Compelling Games • Flow • Challenging, but not frustrating • It’s OK to fail • Try again? • The game will wait for us ‘- • The player has complete control • A failure feels like the players shortcoming • Can overcome with practice 19

20. Flow ‘- image: whats-in-a-game.com 20

21. Single-Player Games • Flow • Player progresses as far as they are able • Game then “waits” for the player as the practice with the game mechanics • With enough practice the player can overcome the next obstacle ‘- • OK to fail • Worst Case: Game over and restart from beginning • Practice easier levels • Control • Responsive control over your sprite 21

22. Multi-Player Games • Flow • Depends on opponent • Evenly matched opponents can experience flow • Unbalanced matches diminish fun ‘- • OK to fail • Worst Case: Find a new opponent • Control • Limited by luck • Over time the more skilled players will win more often 22

23. A BETTER APPROACH ‘- What can we learn from Science? 23

24. Self-Determination Theory • • A leading theory of motivation Motivation depends on: • Competency • Autonomy • Relatedness ‘- Deci, E. L., & Ryan, R. M. (2008). Self-determination theory: A macrotheory of human motivation, development, and health. Canadian Psychology/Psychologie canadienne, 49(3), 182-185. Núñez, J. L., & León, J. (2015). Autonomy support in the classroom: A review from self-determination theory. European Psychologist, 20(4), 275-283. 24

25. Self-Determination Theory • Competency • Progressing towards mastery in a task or skill • Autonomy • In control • Can make meaningful decisions ‘- • Relatedness • Connection to other people • Does the action matter to others? 25

26. Mixed Practice • • Mixing concepts together while studying improves long-term memory As opposed to practicing 1 topic at a time ‘- Shea, J. B., & Morgan, R. L. (1979). Contextual interference effects on the acquisition, retention, and transfer of a motor skill. Journal of Experimental Psychology: Human Learning and Memory, 5(2), 179-187. Hatala, R.M., Brooks, L.R., & Norman, G. (2003). Practice makes perfect: the critical role of mixed practice in the acquisition of ECG interpretation skills. Advances in health sciences education: theory and practice, 8(1), 17-26. 26

27. Compiled Checklist of Goals  Keep students in flow  It’s OK to fail  Player is in control  Competency  Autonomy  Relatedness  Mixed Practice  Scales Well ‘- 28

28. Condensed Checklist of Goals  Keep students in flow / Competency  Player is in control / Autonomy  Relatedness  It’s OK to fail  Mixed Practice  Scales Well ‘- 29

29. THE SYSTEM ‘- 30

30. The App • • • Introductory Computer Science course Procedurally generated problem sets Questions based on level and prior performance ‘- • Earn points by answering question correctly • No punishment for incorrect answers • 100% Automated grading 31

31. Condensed Checklist of Goals  Keep students in flow / Competency  Player is in control / Autonomy  Relatedness  It’s OK to fail  Mixed Practice  Scales Well ‘- Problem Set Performance? 32

32. Keep students in flow / Competency • • Questions depend on their prior performance in the course Can’t progress until they practice/study enough to complete the current question types ‘- • Visual progress bar and level to show their accomplishment thus far 33

33. Player is in control / Autonomy • • Player has limited control Pros • • • Can decide when to use consumable multipliers Can complete problem sets any time they’d like Can complete the question with any approach they’d like as long as their code is correct ‘- • Cons • No choice in the question types or topics 34

34. Relatedness ‘- 35

35. It’s OK to Fail • • • No punishment for getting a question wrong Only reward for answering a question correctly If a student can’t answer a question correctly they will get another similar question ‘- • No game overs • Try as many problem sets as they’d like 36

36. Mixed Practice • • • Previous questions will come back in later levels Especially true if they skip question types Level 13 is 6000 point of mixed practice with no new concepts or question types ‘- 37

37. Scales Well • • 100% Automated Scales with ease to our large class sizes ‘- 38

38. Results ✓ −  Keep students in flow / Competency  Player is in control / Autonomy  Relatedness  It’s OK to fail  Mixed Practice  Scales Well ✓ ‘- ✓ ✓ 39

39. Next Version • Improve autonomy with “skill trees” • Students can practice questions for any topic as long as they’ve completed the required prereq topics • Allows student to choose between different branches in the course • Students can switch between branches at any time ‘- • Improved game loop • Current loop is slow and tedious • New loop will all be in browser on a single page app 40

40. MISTAKES ‘- Learning Experiences 41

41. Deadlines • • • • • Must have. No deadlines last semester Students procastinated until the last few weeks ‘- The course is not [yet] compelling enough to get them to start early Difficult to tell if the system was effective when so many students didn’t even explore it until it was too late • Will have several deadlines/check points next semester 42

42. Communication • Make sure all student understand the mechanics of the system early in the course • I had very basic questions very late in the semester (Ex. How do I earn multipliers) ‘- • One very early deadline could resolve this to make sure everyone completes a few problem sets 43

43. FUTURE TECH ‘- and random ideas 44

44. Beyond Computer Science • Very little of the system, and this talk, is specifically about Computer Science • Adding other subject can be as simple as adding questions for that subject ‘- 45

45. Artificial Intelligence • • Predict exactly what question will keep the student in flow Can personalize the theme of questions to each student based on their provided interest, or determine their interests by what themes they answer correct more often ‘- • Warning: Ethics must be considered when adding AI 46

46. Story • Add a story to the course where student progress by completing sets of questions • • Occasionally they get story questions that advace the narrative ‘- Some story questions can have multiple answers that branch the story depending on the decision/code of the student • Implemmented story in Spring 17’ Art by Angus Lam 47

47. QUESTIONS? ‘- 48

48. Data Analysis • • We have data (timestamps, scores, activity, submissions) New system will collect even more data ‘- 49

49. Game Loop • • What is the primary loop the students repeat throughout the course Check out a problem set, complete the problem set, submit for score and feedback, check out another problem set ‘- • Passing students went through this game loop on average 73 times 50

50. Multipliers • • Apply to problem sets and multiply the score of all correct answers Multipliers stack multiplicitivly • Students doing very well (bored) can quickly progress to a challenge ‘- 53