It knows what you are going to do: Adding Anticipation to a Quakebot

1. It knows what you are going to do: Adding Anticipation to a Quakebot John E. Laird University of Michigan Ann Arbor Proceedings of the Fifth International Conference on Autonomous Agents, 2001

2. Other References • John E. Laird “Using a Computer Game to Develop Advanced AI” (2001) • J.E.Laird, A.Newell, P.S.Rosenbloom “SOAR: An Architecture for General Intelligence” (1987) • J.E.Laird, J.C.Duchi “Creating Human-Like Synthetic Characters with Multiple Skill Levels: A Case Study Using the SOAR Quakebot” (2001)

4. Game details • Can be played at several “levels” • At each level, players can: + Pick “powerups” viz. ammo, weapons, armor, first-aid boxes etc. + Attack an enemy with intent to injure • If player’s health reaches zero, player “dies” • Dead players are “re-spawned” at specific spawning sites. • First player to rack up a determined number of “kills” wins.

5. Motivation • Virtual characters in the game are not “smart” • They should be able to: “beat you by outthinking you not out-shooting you”.

6. Drives of the SOAR Quakebot • To win at the game Quake II. Created by id Software. • Winning + Maximum “kills”. + Staying alive.

7. Environment • Inaccessible • Continuous • Deterministic • Dynamic

8. SOAR Quakebot - Architecture

9. Sensory Inputs • Sensory Inputs are: 1.Visual: Only unobstructed objects in a View Cone. 2.Audio: Only “nearby” sounds. 3.Health • Obtained from underlying Quake II game engine

10. Controls • Implemented with SOAR • DLL written in C • SOAR I/O • SOAR Quakebot Rules • Decision cycle: 30 to 50 times/second. • Multiple reasoning steps for every change.

11. Actions • Known as operators • Operators comprise: + Primitive actions: Move, turn, shoot etc. + Internal Actions: Remembering enemy’s last position etc. + Abstract: Attack, Get_Item, Go_to_Next_Room etc.

12. Decision Cycle of the SOAR Quakebot

13. Elaboration • Some rules may cause certain aspects of sensory input to be elaborated. • Example Rules may require current health level to be categorized as either critical, low, medium or high.

14. Action Selection • Operator Proposal, Evaluation and Selection. • Operators are continually proposed, evaluated, selected and implemented. • This corresponds to sensed changes in game environment • This need not be sequential

15. Operator Proposal • IF-THEN Rules propose operators based on sensory data. • Antecedent matching • Several operators maybe proposed at a given time

16. Operator Evaluation • Priorities are assigned based upon situation. • Example: IF Enemy is armed with superior weapon AND Health is NOT good THEN Retreat to safer position or Hide (Highest) Look for First-Aid Kit Power-up. (High) Attack Enemy (Low) • Additional rule firings

17. Operator Selection and Implementation • Operator Selection: Based upon priorities. • Operator Implementation • Can be a simple motor action • If high-level or abstract operator • Generate sub-states • Decompose into primitive operators Example: <Obtain_First_Aid_Box> comprises <Move_Towards_Box> comprises <Turn> <Walk_Forward> <Pick_Up_Object>

18. Operator Hierarchy – An Excerpt

19. Operator Decomposition

20. Trace of Operator Selections

21. Anticipation In The SOAR Quakebot • More complicated than anticipation in chess, checkers etc. • Reasons + Inaccessibility to entire game state + Breadth of possible actions makes search intractable

22. Anticipation - Requirements • Internal Representation • Prediction • Must be as generalized as possible • Usage Only when enemy is in view but not facing the SOAR Quakebot and far away (else the SOAR Quakebot must attack).

23. Internal Representation • Data gathered from exploration used to construct 2-D internal representation. • Helps SOAR Quakebot remember rooms, corridors, staircases etc. • SOAR Quakebot uses chunking.

24. Prediction • Based upon sensory data pertaining to enemy • SOAR Quakebot creates representation of enemy’s state, partly hypothesis. • Anticipates enemy action based upon what it (SOAR Quakebot) would do. • Comes up with counter-strategy

25. Prediction (2) • Used to: + Ambush enemy + Deny “power-ups” to enemy • Rules used to distinguish enemy’s state from SOAR Quakebot’s own. • Rules to test if prediction is going on. • Time limits on prediction based strategies.

26. Anticipation based Strategy

27. Anticipation based Strategy

28. SOAR Quakebot – A Performance Evaluation Parameters used: • Decision Time (5 Levels) • Complexity of Tactics (3 Levels) • Aiming Skill (3 Levels) • Aggressiveness (3 Levels)

29. Performance Evaluation Methodology • Have SOAR Quakebot challenge a human expert • Videotape action from SOAR Quakebot’s perspective • Have a novice, average and good (not AS good) player challenge human expert. • Record action from challengers’ perspective • Have eight judges evaluate (and compare) performance by looking at videotapes

30. Conclusions • Insignificant impact: aggressiveness, tactical complexity • Significant impact: decision time, aiming skill • Aiming skills most relevant to SOAR Quakebot’s performance.