From the Ground Up: A.I. Architecture and Design Patterns

1. From the Ground Up: A.I. Architecture and Design Patterns Dr Brett Laming Rockstar Leeds

2. Contents • Basic Principles • Think/Act - The I/O Divide • An Algorithmic Approach • A Functional Approach • Movement • Animation • Conclusion

3. Basic Principles

4. It’s a Competitive Market… Higher Expectations Tighter Schedules Multiple Platforms Simultaneous Titles …But Patterns Are Everywhere Identify, generalise, reuse, evolve  Reliability  Production Speed  More Fun Stuff Why Do We Care?

5. Inspirations Marvin Minsky Different representations for different views Daniel Dennet Behaviour can be viewed at the physical, design and intentional levels David Marr Computational, algorithmic and implementational Damian Isla Cognitive Maps Spatial Relations Semantics Noam Chomsky Hierarchical decomposition Structure vs Meaning “Colorless green ideas sleep furiously” Chris Hecker Style vs Structure [2008] What is the textured triangle of A.I. Craig Reynolds Simple rules Complex behaviour

6. What Am I Looking For? Algorithms Concepts Hierarchy Components Reusability Best Practice Commutability

7. Observation How might it work? Introspection What would I do? Generalisation I’ve done this before They do the same Bad Experience Lets not do that again Background I studied this once? Could I apply this? How Do I Find Them?

8. Best Practice

9. Best Practice • Prototype new ideas where possible • Get visual and design direction • Mock-ups • Prove (or disprove) the concept • Quick and dirty programming • Play to peoples strengths • Maths  Physics Guys • Navigation Mesh  Collision Guys

10. Best Practice • Program Defensively • Assert and Unit Test • Automated Scripts as soak tests • One co-ordinate system and S.I. units • Maximise Workflow • Cater tools to their needs • Put new functionality on bypass • Think of the man-hour cost! • Minimise potential for human error

11. Best Practice • Build A Debugging Suite • Instant Pause • Flyable Camera • Layered Information • Action Histories • Maintain Player Immersion • A.I. should not be too bad or too good • Limit ourselves to what the player would know • Constrain them to the same actions

12. The Algorithmic Approach

13. Sensory Data Think Controller Act The Think-Act Loop • Key tenants • Use only what we might know • Mimic the player Player A.I. Sensory Receptors Brain Controller Game

14. Sensory Input Sensory Data Think Controller Act

15. Sensory Data • Detail level • Depends on genre and perceived communication • Thief vs Medieval Total War II • General Model • Visual component an arc • Auditory component a radius • Auditory targets less official • Occlusion too expensive? • Shoot a weapon to get info • Theorise using ghost images

16. Blackboards • Used To Share Information • Static blackboard stores defined types of info • Dynamic blackboard stores arbitrary data • Agents write to the board • Generally read it as well Agent A Agent B {10,20,15} {-30,20,13} Scout {3, 17, 10} Cover {17, 11,5} {-10,15,12} {17,11,5}

17. Control Output cSensorCone Think Controller Act

18. Purpose Carries control instructions Provides a strict I/O Divide Notice the const correctness Unifies player and A.I. Control mapping for the player A.I. fills it in from Think() Key Properties Never stored in it’s entirety Created on the stack Lifetime of a single Process() Virtual Controller (yoke) Process() cVirtualYoke yoke Think(yoke&) const Controller Act(const yoke&)

19. Process() cBlackboard cVirtualYokeyoke Think(yoke&) const cSensorCone Act(const yoke&) … So Actually

20. Composite Collection of smaller yokes Allows selective storage cLocomotiveYoke fp32 mGas; fp32 mSteering; fp32 mPitch; cEntity *mpDontAvoid; Context Based Controller Input Logic Flow Control cWeaponYoke enum eFire { F_PRESSED, F_WHEN_IN_CONE, F_WHEN_LOCKED }; eFire mFireButton; Conditional signals Virtual Yoke cVirtualYoke cLocomotionYoke mLocomotion; cWeaponYoke mWeapons; cAnimYoke mAnimation;

21. Taking Action Process() cBlackboard cVirtualYokeyoke Think(yoke&) const cSensorCone Act(const yoke&)

22. Act() yoke iWeaponMgr* iLocomotive* Act • Applies Yoke Commands • Composite yokes  Subsystems • Object Models • Supply common interfaces • Could be a turret mounted weapon or my pistol. • Could be driving a car, a plane, a boat or myself!

23. Self Contained Instructions for Think Actions for Act Commutable Plug and Play Downloadable Content Maybe broadcast use Think Sims 2! Object Model iLocomotive // Fills in yoke ComputeMotion(const cTarget &, cLocomotiveYoke&) // Computes forces ApplyMotion(const cTarget &, const cLocomotiveYoke&, v3 &force, v3 &torque)

24. Compute Compute Compute Apply Apply Apply More Coolness ApplyMotion(const cTarget& targ, cLocomotiveYoke& yoke) { v3 force(Zero); v3 torque(Zero); if (Drivable()) { Drivable()->ApplyMotion( targ, yoke, force, torque); } if (Floatable()) { Floatable()->ApplyMotion( targ, yoke, force, torque); } ApplyForceTorque(force, torque); } Compute Apply

25. Frame Time Process() cBlackboard cVirtualYokeyoke dt() Think(yoke&, dt) const cSensorCone Update(dt) Act(const yoke&, dt)

26. Work into everything Including fixed time steps No more s+=v, s+=v*dt() Benefits Integration Implicit forward Euler Rough Smoothing Closed Feedback Pause dt=0 Level Of Detail dt=2dt Special Effects dt() smoothing feedback control dt()

27. The Functional Approach

28. Problem Domain Needs a problem  • Examine the Terminology • Feeling, Knowledge, Goals, Beliefs, Needs • Examine the Concepts • Decisions, Facts, Uncertainty, Exploration, Verb-Noun Actions, Repetition, Sequencing

29. What Are We Doing? • Goal based reasoning • Working to solve a goal • Thinking about and realising smaller tasks • Taking a hierarchical approach • Using a limited number of short verb-noun pairings to form a plan • We’ve seen this before • An old pattern • Colossal Cave Adventure and MUDs • Verb-noun actions like “get axe, wield axe” separated by movement “n, e, e, s, e” • We use a container object model

30. Know of “Mine, Smithy” Goto “Mine” Know of “Wall” Get Ore From “Wall” Goto “Smithy” Know of “Door” Use “Door” Play “Open Door” Warp Inside Know of “Owner”, “Forge” Play “Close Door” Goto “Owner” Put 10 gp In “Owner” Get Time From “Owner” Goto “Forge” Put Time In “Forge” Put Iron In “Forge” Use “Forge” Use Enchant Use “Forge” Get Sword From “Anvil” Applying The Pattern Taking the computational to the algorithmic

31. Easy Questions • Why did I choose to do this again? • Because we were driven to it by personality and need • What happens when I get the treasure? • I’ll probably choose to do something else depending on my mood • Why stop at get iron? • Because its reached the atomic level - there are no more questions, just results

32. Explicit Order “Script here, Please greet the player” Putting It Together Ambient Controller “Get treasure” Planning “Get magic sword”

33. Explicit Order “Script here, Please greet the player” Explicit Orders • Script commands • Script on A.I. • Autopilot • Player on Player • Player instructions • Player on A.I. • Squad Commander • A.I. on A.I. Ambient Controller “Get treasure” Planning “Get magic sword”

34. Scripting Notes • Don’t mix styles • Script has immediate control • Script waits for an opportunity • Keep common properties separate • No sharing memory locations • A script global population density • A code global population density • Maintain a set order of calculation • Generally consistent with style

35. Explicit Order “Script here, Please greet the player” Ambient Controller • Generates sensible actions autonomously • Maybe Idle • Maybe Full Daily Routine • Daily Routines • Character properties • Needs/Drives • Scheduling • Time of day • nice emergent behaviour Ambient Controller “Get treasure” Planning “Get magic sword”

36. Hunger Libido dt() Daily Routine Time Of Day Sleep Drive Model Goto Work Work Character Leave Work Relax Schedule Go Home

37. Planning Goto Position “Goto Forge” Explicit Order “Script here, Please greet the player” Perform Action “Use Forge” Planning Continuous bar dt() • Defines Think() Ambient Controller “Get treasure” Planning “Get magic sword”

38. Get/Take “Object” From “Container” Primary world manipulation Contents determine state Goto “Location” Waits on dt() Key A.I. output Complex Warrants special attention later Plan Components • Play “Animation” • Waits on dt() • Use “Object” • Object model again • Broadcast actions. • Change world based on state • May wait on dt()

39. Search Based Planning • Traditional academic approach • See STRIPS, Hierarchical Task Networks, Bratko • The Good • Mimics our introspective reasoning • Seeks to fully realise a plan to the goal • Directed search for optimal solutions • Post processing even more so • The Bad • Knowledge representation • Scalability - difficult for video games • Lots of storage

40. Procedural Planning • Industry preferred approach • Hierarchical • Easy to comprehend • Limited Language • Goals • Sub-goals • Conditions • Actions • Transitions • Sequential • Decision Based • Powerful Results Get Object Buy Short of money? Make Thief skills? Steal Goto Mine Get Ore

41. Procedural Planning - Issues • Competing children • We have to make a best guess from the options • A* might help • But we could still end up with case of a basic sword being bought but not affording the forge. • Incomplete plan means no post process • Not good for player supporting A.I. • More action for generic A.I.

42. The Curve Ball • Task Interruption • I’m returning to my gang hideout • I see an enemy • I engage the enemy • I roll out of the way of a car • I recover to my feet • I re-engage the enemy • I continue to return to my gang hideout • … Is A Key Requirement of our A.I.

43. Death Examine Kill Death Examine Kill Gain Range Gain Range • HFSM came along • Eased transitions • But history still an issue Look At Attack Look At Attack Finite State Machines • No plan history • No idea of context • No generalised exit. • Hideous state history workarounds • Don’t scale well • Many transitions

44. Behaviour Trees Get Object Decorators Selection Crime Allowed? Create Buy Low money? Get Material Ok money? Steal Sequencing Thief Class? Goto Source Take Material Preconditions Actions

45. Modelling Interruption? Ambient Behaviour Return Home Kill Enemy Parallel? Fire Weapon Locate Hideout Goto Location Gain Range

46. Simple and powerful Limited vocabulary Most situations handled Highly flexible Plug and play Customisable Nice design tools Handy child evaluation Lends itself to directed decision making Issues Interruption handling Where to return Amount of flexibility Trees get complicated Behaviour Trees

47. MARPO Get Object Thief class? Thieving allowed? Not enough money? Money? Steal Create Buy Get Material Goto Source Take Material

48. MARPO Long term Reactive Immediate Get Object Create Get Material Take Material

49. Create Gain Distance Get Material Take Material MARPO Long term Reactive Immediate Get Object Kill Enemy

50. Gain Distance MARPO Long term Reactive Immediate Get Object Kill Enemy Roll