Webinar Series

1. Webinar Series Session 10 July 2013 Towards a framework for games and simulations in STEM subject assessments Gary Wills (University of Southampton, UK). Session spoken by Ashok Ranchhod (University of Southampton, UK). Please test your audio set up using the wizard: Tools ->Audio -> Audio setup wizard. Your Hosts Professor Geoff Crisp, Dean Learning and Teaching, RMIT University geoffrey.crisp@rmit.edu.au Dr Mathew Hillier, Teaching and Educational Development Institute, University of Queensland mathew.hillier@uq.edu.au

2. Research into Serious Games for Academic Assessment Talk by Ashok Ranchhod Acknowledgement: Gary Wills, James Stallwood, James Baker, Lester Gilbert, Alejandra Recio Saucedo, Alex Frazer, Teresa Binks

3. What is a Game? • Nobody is really sure… • “A series of meaningful choices” – Sid Meier, designer “Civilization” • “One or more causally linked series of challenges in a simulated environment” – Ernest Adams & Andrew Rollings, game design authors • “Conflicts in which players work to foil each other’s goals” – Chris Crawford, game designer and theorist • “A system in which players engage in an artificial conflict, defined by rules, that results in a quantifiable outcome” – Katie Salen & Eric Zimmerman, game design authors

4. What is a Game? “a rule-based formal system with a variable and quantifiable outcome, where different outcomes are assigned different values, the player exerts effort in order to influence the outcome, the player feels attached to the outcome, and the consequence of the activity are optional and negotiable” – JesperJuul, video game theorist • There are some useful key concepts here • Choices, simulated environments, conflicts, goals, rules, engagement, rewards

5. What is Game Design? • Creating a game concept, then defining how that game works • Defining the underlying rules and behaviours which govern the game world • This condition can be completed competitively or cooperatively Or both! • Rules set games apart from toys and roleplay sets them aside from puzzles • Weaving an engaging narrative into the game • Deciding how the player will interact with the game world

6. Educational Games: Why Bother? • Initially, games were used for educational purposes for purely motivational reasons • Games are fun, if we make a learning game, it’ll make the learning fun • Not necessarily true…

7. Knowledge Contextualisation • Knowledge is arguably useless in isolation • We need to be able to apply it to a given context • Games will typically introduce new skills and ideas in isolation • In tutorials at the beginning of the game, or at various points throughout • Once these skills are taught, the player will be called upon to use them in progressively more complex scenarios • As more skills are learnt, the more complex the scenarios can become…

8. Game Balance • Maintaining the “Flow” zone • Make game challenges difficult enough to stop the player becoming bored, without being so difficult as to be impossible • Difficulty curve • The player is likely to become better at a game as they progress • Challenges will become relatively easier • If they become too easy, the player will grow bored • Increase the difficulty accordingly, to maintain a state of Flow • The rate of change of this curve can be difficult to engineer!

9. Game Balance • Feedback loops • The rewards a player receives for completing a task could make it easier to complete the same task in future… • …earning even more rewards… • …making the task even easier still! • This can make the game boring (or frustratingly difficult for a multiplayer opponent), disrupting the Flow state • We can overcome this in a number of ways • Make the reward less powerful • Gradually make the reward less powerful as the player progresses • Offer rewards for one task which help another

10. Moving to Learning • Understanding and the construction of knowledge comes through learners undertaking learning activities and receiving feedbackfrom taking the activity. • In some subjects this will include serious games, simulations, and the building and running of models.

11. Moving to Learning • Learning activities inherently engage and motivate the learner through their characteristic combination of learning actions and consequential feedback. • These are part of the wider formative learning activates used to motivate, and provide a contextual understanding of the subject domain.

12. Craft industry • Providing games and simulations to address specific educational objectives in different subject areas may • require custom-built applications • That are difficult-to-share • difficult-to-reuse solutions whose effectiveness is highly dependent upon the personal skills of specialist developers

13. Benefits to STEM • During the past decades, researchers have explored the effectiveness and impact of games that can be used as educational tools and in supporting learning of STEM at any educational level: • facilitate students’ inquiry, • encourage them to engage in “What if” explorations, • allow multiple and dynamic external visualisations through multimedia support, • optimally communicate STEM abstract, complex and non-perceptible concepts, • promote metacognitive skills, and • support social interaction and collaborative learning.

14. Intended Learning Outcomes: • A pedagogically informed learning activity is based upon the existing competence of the learner, is coupled with prerequisite competences, and seeks to develop one or more articulated desired competences.

15. Intended Learning Outcomes: • A pedagogically informed (top level) competence is conceived as an acyclic directed graph of enabling competences • Each competence (top level and enabling) comprising a contextualised intended learning outcome (ILO). • An ILO in turn comprises a learned capability with respect to a specified topic or item of subject matter. • A number of PhDs have been and are being conducted in the area of computer based competence structures in the group

16. Intended Learning Outcomes: • Pedagogically informed teaching activities associated with the specified learning activities include: • the provision of appropriate materials, • the assessment of learner performance on the desired competences, • the provision of feedback which is well-timed, contingent, and specific.

17. Model of an ILO Courtesy of Lester Gilbert

18. Merrill’s works

19. Goal Definition • ILOs are a an educational way of setting an achievable and measurable goal. • Goals are useful for learner motivation • Game designers consider them equally important • “The 400 List” puts “provide clear short-term goals” and “provide an enticing long-term goal” at positions 6 and 25 respectively • Long term goals can be used to imply a context for more immediate, short-term goals

20. Framework • The question we asked was how can we construct and then take a machine-processable expression of an educational intention and (either automatically or semi-automatically) turn it into a computer-deliverable serious game, simulation, or adaptive formative assessment with (some elements of) pedagogical validity? • As a response, we propose a framework for Pedagogically Effective Games & Simulations (PEGS).

21. Learners • In such a framework, learners will be able to assess their current learning needs and navigate a machine-readable ILO structure to meet their learning goals. • Each ILO will be associated with one or more learning activities. • Where a learning activity is designed to feed the curiosity and reasoning processes of the learner, engagement and motivation is likely, allowing the learner to gain understanding and construct knowledge through their learning activity and consequential feedback.

22. Learners • The pedagogical strategy is to invoke active learner participation supported by multiple resources made available in the learning environment. • The learner participation involves searching for, evaluating, and using authentic information. • This learning experience mimics real life in targeting the learner as the routine information hunter and interpreter who constructs knowledge by problem solving with information tools. • The advantages to this strategy are the adoption of a student-centred approach to learning and the promotion of thinking skills (problem solving, reasoning, and critical evaluation).

23. PEGS Framework • A framework should allow for the 'routine' construction of cost-effective, shareable and reusable games and simulations as pedagogically-informed learning activities by teachers and learners who do not necessarily have gaming experience or skills. • In the same way that that learning materials are routinely constructed in support of one or more ILOs and assessments are routinely constructed to test the achievement of one or more ILOs. • The framework should use open standards in the creation of games and simulations, thus ensuring continuous development and uptake by interested communities

25. Framework • The ‘PEGS-I’ tool is an application which supports a teacher in identifying and structuring intended student competences from syllabus and curriculum descriptions, from teacher expectations and desires, and possibly from national and international prescriptions. • ‘PEGS-D’ is a standards-defined interoperable competences database which represents the topics of interest to a particular community. • Visualising the PEGS-D competences uses the ‘PEGS-V’ visualisation tool, which also supports exchanging PEGS-D competences with others, tagging key competences with supporting learning and teaching materials, etc.

26. Framework • One or more game and simulation templates are populated from PEGS-D to generate the desired learning activities using ‘PEGS-E’, the Editor. • Templates may be pre-defined or developed from a ‘blank sheet’ or by adapting existing templates. The templates are standards-defined and interoperable (IMS-LD, IMS LTI, with extensions), yielding games and simulations which can be played using a compatible software application, the PEGS-P player. The games and simulations are held in the PEGS-GS repository. • The simpler games and simulations are single-user, the more advanced multi-player.

27. Framework • Assessments are generated using the ‘PEGS-A’ (assessment) tool and are held in a compatible repository, PEGS-X, as a standards-defined and interoperable (IMS-QTI, with possible extensions, and/or IMS LTI) file. • Finally, we envision a future ‘PEGS-ID’ tool to provide suggestions for teaching strategies and instructional designs which are appropriate to constituent competence sub-graphs in the topic, and an intelligent tutor, ‘PEGS-ITS’, to adaptively deliver materials and assessments.

28. PEGS toolkit • The framework will be delivered through a series of easily adoptable tool-kits. • Toolkits “provide a pragmatically-based approach to applying theory to practice” and can be used to support decision-making. (Conole et al., 2005) • Enable the 'routine' construction of cost-effective, shareable and reusable games and simulations as pedagogically-informed learning activities by teachers and learners, • Similar to the way that learning materials are routinely constructed in support of one or more ILOs and assessments are routinely constructed to test the achievement of one or more ILOs.

29. Research Agenda • Maintaining the flow in an Educational Game • How can we use the underlying competence structure to automatically provide the balance of a game. • Visualizing the graphs structure • Mapping the competence structures in such a way that learners can map out their own route for learning.

30. Research • Connecting Games to competences • What affordances do games offer and how can we relate these to the competences we require from learners. • In a collaborative world what affordances can MMROPGs offer education. • How can the learning know (trust) if an activity will meet the goals. • Automatic adaption based on the experience of the user.

31. Research Agenda • Navigating the competence structure • Obtain appropriate learning activities for each of the competences and path to take to meet their goals. • Work by Onjira Sitthisak and Lester Gilbert suggested that we can automatic generate assessment questions from the competency tree • Can we build a list of automatic activates to meet a given competence • Feedback: can we use the structure to give automatic feedback

32. EUFLORIA-Strategy Game on colonising the earth

33. PROTEUS-A Sandbox game-explore enjoy learn Can these be formalised and competencies worked out?

34. ACTING Win Challenge COMPETE Showoff Compare Taunt Build Purchase Design Create Decorate EXPRESS Customise Choose CONTENT PLAYERS View Collect Rate EXPLORE Vote Curate Review Comment Like COLLABORATE Greet Help Share Give INTERACTING

36. Conclusions • Games for learning have the potential to engage and motivate learning • They are fun • They can provide (instance) feedback • They require clear learning goals • They can maintain the flow • They can provoke curiosity (what if scenarios ) • They can be played individually or within a group and you can collaborate or compete.

37. Conclusions • We have presented a framework to support the 'routine' development of effective games and simulations from pedagogically-informed statements of intended learning outcomes. • That would be machine-processable • turn it into a computer-deliverable serious game, simulation, or adaptive formative assessment with pedagogical validity.

38. Webinar Series Webinar Feedback Please kindly fill in the feedback survey by clicking this link: With Thanks from your hosts Professor Geoff Crisp, Dean Learning and Teaching, RMIT University geoffrey.crisp@rmit.edu.au Dr Mathew Hillier, Teaching and Educational Development Institute, University of Queensland mathew.hillier@uq.edu.au