Agents in QuizMASter

1. Intelligent Education Systems Group Agents in QuizMASter Dr Fuhua Lin Fuhua Lin, SCIS, FST, Athabasca University

2. UT Austin Villa Wins World RoboCupChampionships 2011 http://www.utexas.edu/news/2011/07/19/villa_wins/ The key to victory, says Peter Stone, was that he and his graduate and undergraduate students taught their robots to teach themselves. Fuhua Lin, SCIS, FST, Athabasca University

3. Immersive Learning Environments • Commercial platforms such as: • World of Warcraftfor online gaming • Second Life for online social networking • Positive outcomes of these environments • a high level of realism • associated levels of engagement • supporting and encouraging social interaction • Whether these positive outcomes can be generalized and applied to the education community and weather institution can adopt these environments and provide them as part of their online ICT infrastructure ? Fuhua Lin, SCIS, FST, Athabasca University

4. Game-based E-Learning • the use of a computer games based approach to deliver, support, and enhance teaching, learning, assessment, and evaluation. (Connolly et al., 2004) Fuhua Lin, SCIS, FST, Athabasca University

5. Goal • Devise virtual learning environments that integrate AI and game engines. • 3Es: Effective, Efficient, Engaging • 3Is: Intelligent, Interactive, Immersive • Adaptive • Motivational elements: • 4Cs: challenge, curiosity, control, and context, • creativity is the new emerging “C” a (Lepper& Henderlong, 2000) • Social relationship • Play and learn • Target users: • For all learners, both within and outside of the classroom. • What to do? Infrastructure for Building Virtual Classrooms • Research • Development • Evaluation Fuhua Lin, SCIS, FST, Athabasca University

6. Requirements (1) --- Believability • The believability of what we communicate is influenced 55% by body language • For a six month baby, when you smile to him/her, he/she may smile to you. • Body Language • how to detect it • how to express it adaptively and automatically. Fuhua Lin, SCIS, FST, Athabasca University

7. Requirements (2) ---- Autonomy • Game engines are still too complex for most educators to implement their own learning games. • avatar animation and facial expressions, but these features must be controlled manually – there is no way to associate them with game events. • stops short of providing tools essential to many educational game activities, such as question banks, score keeping, and user modeling. Fuhua Lin, SCIS, FST, Athabasca University

8. Requirements (3) Easy to Build • The time and expertise required to create believable NPCs and engaging learning activities based on virtual-world technologies remains a significant barrier. • How to incorporate intelligence into NPCs? • How to make the agents learn? Fuhua Lin, SCIS, FST, Athabasca University

9. Why Agents and MAS? • Complexity • it is not practical to analyse and code for every possible game state and every possible interaction between the various game elements. • Why Multiagent Systems? • For cooperative problem solving. • For emergent behaviours. • For global optimization. • ... Fuhua Lin, SCIS, FST, Athabasca University

10. Agents Agent Reactive agent Goal-oriented agent Fuhua Lin, SCIS, FST, Athabasca University

11. Agents Agent Reactive agent Goal-oriented agent NPC Agent System Agent Student Agent Fuhua Lin, SCIS, FST, Athabasca University

12. Agents Agent Reactive agent Goal-oriented agent NPC Avatar NPC Agent System Agent Student Agent Fuhua Lin, SCIS, FST, Athabasca University

13. Agents Agent Reactive agent Goal-oriented agent NPC Avatar NPC Agent System Agent Student Agent Pedagogical Agent Virtual Student Virtual Audience Timer Scorekeeper AMA Fuhua Lin, SCIS, FST, Athabasca University

14. Agents Agent Reactive agent Goal-oriented agent NPC Avatar NPC Agent System Agent Student Agent Pedagogical Agent Virtual Student Virtual Audience Timer Scorekeeper AMA Tasks 1. Expressing verbal/non-verbal communication actions 2. Learning from the student agents about the students 3. Generating quizzes and hints 4. Reasoning about the situation. 5. Providing narrative and dialogue functions that provide increased engagement and immersion in the environment

15. Agents Agent Reactive agent Goal-oriented agent NPC Avatar NPC Agent System Agent Student Agent TSI-enhanced Pedagogical Agent Virtual Student Virtual Audience Timer Scorekeeper AMA Tasks 1. Expressing verbal/non-verbal communication actions Adaptively to increase user engagement level. 2. Learning from the student agents about the students 3. Generating quizzes and hints 4. Reasoning about the situation. 5. Providing narrative and dialogue functions that provide increased engagement and immersion in the environment

16. Agents Agent Reactive agent Goal-oriented agent NPC Avatar NPC Agent System Agent Student Agent TSI-enhanced Pedagogical Agent Virtual Student Virtual Audience Timer Scorekeeper AMA Tasks: 1. Detecting and analyzing verbal/non-verbal communication and behavior(e.g. emotion, gesture) 2. Learning from other agents 3. Modeling student (level, preferences, learning styles, profiles) 4. Identifying social relations Tasks 1. Expressing verbal/non-verbal communication actions Adaptively to increase user engagement level. 2. Learning from the student agents about the students 3. Generating quizzes and hints 4. Reasoning about the situation. 5. Providing narrative and dialogue functions that provide increased engagement and immersion in the environment

17. Agents Agent Artifact Reactive agent Goal-oriented agent NPC Avatar NPC Agent System Agent Student Agent TSI-enhanced Pedagogical Agent Virtual Student Virtual Audience Timer Scorekeeper AMA Tasks: 1. Detecting and analyzing verbal/non-verbal communication and behavior(e.g. emotion, gesture) 2. Learning from other agents 3. Modeling student (level, preferences, learning styles, profiles) 4. Identifying social relations Tasks 1. Expressing verbal/non-verbal communication actions Adaptively to increase user engagement level. 2. Learning from the student agents about the students 3. Generating quizzes and hints 4. Reasoning about the situation. 5. Providing narrative and dialogue functions that provide increased engagement and immersion in the environment

18. Environment Programming in MAS Environment Fuhua Lin, SCIS, FST, Athabasca University

19. Environment Programming in MAS Environment Fuhua Lin, SCIS, FST, Athabasca University

20. Environment Programming in MAS Environment Fuhua Lin, SCIS, FST, Athabasca University

21. Reactive Agents • Perceive events • Simple set of rules event  action (i.e., activation of a specific behavior) • Actions are often known as “behaviours” • Example of a simple “mail agent”: • if send mail then check virus • If new mail then check spam • If spam then send message to friends agents • If new message then get new spam information • Pros: • simple and efficient • Cons: • Action depending only on stimuli • Not flexible • Not really autonomous Fuhua Lin, SCIS, FST, Athabasca University

22. Reactive Agents with State • Internal state (internal knowledge) • Update of internal state • New state = actual perception + old state • The update may require • Knowledge on how the world evolves – which can also dynamically acquired by the agent • Knowledge on how the agent actions influence the world • Select action (i.e., behavior) accordingly • Example • A mail agents that keeps track of the users marking some messages as “spams” and take these into account in future actions • An object is a sort of reactive agents, but • - It has no rule for action selection • - It actions are directly commanded by the external Fuhua Lin, SCIS, FST, Athabasca University

23. Goal-oriented agents • Goal  a desired situation to eventually achieve • The agent exploits the goal and its knowledge select actions whose effect would be that of approaching the goal • How can a goal be selected? • Search in the state space • Plannings • Heuristics  sub-optimal actions • Example: an agent to minimize fragmentation in a hard disk • - Knapsack problem • - Do not know the future but know the past • - Select allocation of new files based on some heuristics • - An action does not necessarily minimize the current fragmentation • - Perform de-fragmentation action when the computer is idle Fuhua Lin, SCIS, FST, Athabasca University

24. Utility-oriented Agents • The Goal is that of maximizing the current utility • opportunistic behavior • Utility • A function of some parameter, measuring the state of “goodness” (with respect to the agent) of a situation • Often, it measures a trade-off between contrasting objectives • Example • An agent to maximize CPU utilization • Always select the ready process • The current choice may be sub-optimal with regard to the global execution time of processes Fuhua Lin, SCIS, FST, Athabasca University

25. Hybrid Architectures • Mixing utility and goals • An agent that has to achieve a goal and, at the same time, has to maximize a specific utility function • Trade-off between the two goals, which may be contrasting • Often, the various ways to approach a goal can be quantified by a utility function • Do the actions that approach the goal with the maximal utility • Mixing reactive and goal-oriented behavior • A long terms goal that include several short term actions on the environment • That could lead to sub-optimal choices Fuhua Lin, SCIS, FST, Athabasca University

26. BDI agent model • BDI agents are most suitable to implement intelligent behavior in games • The use of goal-oriented action planning in gaming • They make explicit use of goals and planning • They incorporate mechanisms to effectively use communication and other interaction mechanisms in their action deliberation A Plan library Beliefs Emotions Action output Sensor input Interpreter Desires Intentions Agent EBDI Model Emotion reasoning --- one of the common sense reasoning! Fuhua Lin, SCIS, FST, Athabasca University

27. OCC Emotion Model • The theory of human emotions and emotional reactions to events proposed by A. Ortony, G. L. Clore, and A. Collins. (1988) (OCC model) • Emotion classes, each consisting of several emotions or emotional reactions, for each emotion, eliciting conditions determine under what circumstance the emotion is elicited • Well-being • Fortunes-of-others • Prospect-based • Attribute • Compound • Attraction Fuhua Lin, SCIS, FST, Athabasca University

28. Formalization of Part of OCC model (Mueller, 2006) • Agent sort: a, a1, a2, … • Belief sort: a, b1, b2, … • Event sort: e, e1, e2, … , and Not(e) • Fluent sort: f, f1, f2, … • Object sort: o, o1, o2, … • Real number sort: x, x1, x2, … Fuhua Lin, SCIS, FST, Athabasca University

29. Fluent --- represent facts (factors) used to specify the electing conditions of emotions • Believe(a, e) --- agent a believes that b has occurred • Believe(a, Not(e) ---- agent a believes that e has not occurred • Desirability(a1, a2, e, x): agent a1 believes that the desirability of event e to agent a2 is x, where -1 ≤ x ≤ 1. a1 and a2 may be the same. • Praiseworthiness(a1, a2, e, x): agent a1 believes that the praiseworthiness of event e performed by agent a2 is x, where -1 ≤ x ≤ 1. a1 and a2 may be the same. • Anticipate(a, e, x): agent a anticipates that event e will occur with likelihood x, where 0≤ x ≤ 1 Fuhua Lin, SCIS, FST, Athabasca University

30. Emotion expression functions • Joy(a, e): agent a is joyful about event e • Distress(a, e): agent a is distressed about event e. • Happyfor(a1, a2, e): agent a1 is happy for agent a2 regarding event e • SorryFor(a1, a2, e): agent a1 is sorry for agent a2 regarding event e • Resentment(a1, a2, e): agent a1 is resentful of agent a2 regarding event e • Gloating(a1, a2, e): agent a1 gloats toward agent a2 regarding event e. • … Fuhua Lin, SCIS, FST, Athabasca University

31. Learning • What to learn • Body language (emotions) • Communication patterns • Domain knowledge (quiz bank, determine the degree of difficulty, students levels, • Game play knowledge • How to learn • Centralized learning • Distributed learning • Case-based learning • Student agents --- how to build student models, learn from human users, pedagogical agents, and other agents, • Pedagogical agents --- how to group students, how to generate quizzes, how to provide hints, how to score • NPCs --- how to learn from humans and their agents? Fuhua Lin, SCIS, FST, Athabasca University

32. Pedagogical Agents’ Algorithms • How to determine a student 's level and to assign a correct game room; • Decide what kinds of peer virtual students will be best for this student, assuming we have a repository of virtual agents (NPCs) available; • During the game-play, quiz generation and sequencing, given a group of real players and virtual players. Fuhua Lin, SCIS, FST, Athabasca University

33. Question Item Metadata Fuhua Lin, SCIS, FST, Athabasca University

34. Simulating Human Communicative Strategies • Simulating Human communicative strategies • Compose explanations spoken or textual; • Deliver encouragement, critiques and maintain a mixed initiative dialogue; • Analyze a student explanation, spoken or textual; • Question student’s approach • Recognize student’s affect (emotion, focus of attention, or motivation) • Engage students in role playing; hire partners for training interactive skills. • … (Woolf, 2009) Fuhua Lin, SCIS, FST, Athabasca University

35. Our Publications • S. Leung, S. Virwaney, F. Lin (2011, Submitted), TSI-enhanced Pedagogical Agents, TESL 2011, Dalian, China. • Martin Weng, Fuhua Lin, Timothy K. Shih, Maiga Chang, IretiFakinlede, A Conceptual Design of Multi-Agent based Personalized Quiz Game, The 11th IEEE International Conference on Advanced Learning Technologies (ICALT 2011) July 6-8, 2011, Athens, Georgia, USA (accepted) • Ning Xia, Fuhua Lin, Aishuang Li, MODELING AND VISUALIZATION OF FRUIT TREES IN HORTICULTURE, in a book "Computers and Education" edited by Sergei Abramovich • Blair, Jeanne & F. Lin (2011). An Approach for Integrating 3D Virtual Worlds with Multiagent Systems, ISeRim workshop - IANA 2011 (March, 2011; Singapore)  • Armstrong, AJ & F. Lin, (2010) Modelling and Personalizing Curriculum with Colored Petri Nets, ICCE 2010 (WIPP). • F Lin, Kinshuk, & M Dutchuk, (2009). Multiagent architecture for incorporating adaptivity feature into 3D learning environments, The 6th International Workshop on Mobile and Ubiquitous Learning Environments (MULE 2009), Sept 8-12, 2009, Athabasca University, Canada, pp 33-35,  • Mark Dutchuk, Khalid Aziz Muhammadi, Fuhua Lin (2009), QuizMASter - A Multi-Agent Game-Style Learning Activity, EduTainment 2009, Aug 2009, Banff, Canada, Learning by Doing, (eds.), M Chang, R. Kuo, Kinshuk, G-D Chen, M. Hirose, LNCS 5670, 263-272.  Fuhua Lin, SCIS, FST, Athabasca University