Lecture 22 – Decision Making

1. Lecture 22 – Decision Making CS 490/590 Wesley Kerr

2. Acknowledgements • Ian Millington. Artificial Intelligence for Games. Morgan Kaufman Publishers. 2006 • images and organization • Mat Buckland. Programming Game AI By Example. Jones & Bartlett Publishers. 2004

3. AI Model AI gets information AI gets a slice of processor time AI turned into on-screen action

4. Decision Making Internal Changes Internal Knowledge Decision Maker Action Request External Knowledge External Changes

5. Decision Trees Security Guard • Fast, simple, easy to implement, easy to understand • Arguably the simplest technique we will look at • Doesn’t mean that you can’t achieve realistic AI with it • Idea • A tree of connected decision points • Starting decision is the root of the tree • Each decision leads down a path grounding out in the leaves of the tree which are actions sleeping? true false noise? idle? false true true false sleep patrol perform stir

6. Goomba Decision Tree

7. Decision Tree

8. Key Points • Decisions are simple • Decisions do not contain complex boolean logic • Types of tests • boolean (the value is true or false) • enumeration (the value is one of the set) • numeric (the value is within some range of values) • 3D vector (the vector is within some length) if (path clear AND no chasm) then go forward if (path blocked OR chasm ahead) then turn around

9. Code Structure

10. Complexity (for binary decision tree) • Time to reach action • Balanced tree • O(log n) • Not necessarily always the case • Try to keep most common path higher up in the tree

11. Interesting Ideas • Reusing nodes in the decision tree (merging) • Simply reconnect the decision node to another point in the tree • Be careful not to create loops in your tree • Loops result in infinite loops • Random decisions

12. State Machines • Character occupies a single state • Normally a set of actions/behaviors are associated with a state • States are connected by transitions • Each transition has a set of conditions

13. Pseudocode class FSM // maps from a state to the list of transitions leaving it transitionMap currentState // the current state of the FSM startState // the state that the FSM always starts in public void addState(State s) // add a state to the FSM public void addTransition(State from, State to, Transition t) public void update() { // update the active state // test all of the transitions from the active state // if a transition is active // then // call exitState() on the active state // call enterState() the endState of the transition // set the active state to be the end state of the transition

14. Hardcoded FSM class GuardFSM : enum State : PATROL, DEFEND, SLEEP // holds the current state of the FSM current-state = SLEEP function update(elapsed): // check for transitions if (current-state == PATROL) : if (canSeePlayer()) : current-state = DEFEND if (tired()) : current-state = SLEEP elseif (current-state == DEFEND) : if ( not canSeePlayer() ) : current-state = PATROL elseif (current-state == SLEEP) : if ( not tired() ) : current-state = PATROL function receive-sound-event(event) : if (current-state == SLEEP and loud(event)) : current-state = DEFEND function get-action(): if (current-state == PATROL) : return patrol-action if (current-state == DEFEND) : return defend-action if (current-state == SLEEP) : return sleep-action

15. FSM Benefits • Easy to write • Conveniently breaks down the challenges into manageable chunks • Additions are made by adding more transitions and states • Easy to picture • Easy to implement • Straightforward to debug • Deterministic – makes replication easier • Visual debugging – rendering the current state to the screen • Offline debugging – record all state transitions to a file • Applicable to a variety of situations • Menu system • Character AI

16. FSM Cons • When time is shortened you can easily short circuit the system • Adding in additional “switch” and “case” methods instead of proper transitions • Can grow to be very complex • The number of transitions tends to grow exponentially with the number of states • State oscillation

17. Hierarchical FSMby example

18. Hierarchical FSMby example

19. Hierarchical FSMby example

20. Hierarchical FSMby example

21. Walkthrough • State machine composed of layers • Transitions that occur in layers higher up, have precedence • Slightly modified version of the FSM code presented earlier. • Each enterState, exitState, update, and transition method returns a single action • For each update of the HFSM we have a list of actions to perform

22. 1. Start in State L 2. Transition from H*  A return: update-L, enter-A state: [L,A] 3. Transition 1 trace: top-level – no transitions active-state (L) – 1 fires return: exit-A, 1-actions, enter-B, update-L state: [L,B] 4. Transition 4 trace: top-level – 4 fires active-state (L) – ignored return: exit-L, 4-actions, enter-M state: [M] 7. Transition 3 trace: top-level – no transitions active-state (L) – 3 fires special case Level 1  Level 0 pass the transition up to parent unfinished return: exit-B, exit-L, 3-actions, enter-N state: [N] 5. Transition 5 trace: top-level – 5 fires return: exit-M, 5-actions, enter-N state: [N] 8. Transition 7 return: exit-N, 7-actions, enter-M state: [M] 6. Transition 6 trace: top-level – 6 fires return: exit-N, 6-actions, enter-L state: [L,B] 9. Transition 2 special case Level 0  Level 1 return: exit-M, enter-C, enter-L, 2-actions state: [L,C]

23. public class HeirarchicalStateMachineextends HSMBase { _states; // a list of the states stored at this level _currentState; _initialState; function getStates() if (_currentState != null) return _currentState.getStates(); else return []; function update() if (_currentState == null) _currentState = _initialState; return _currentState.getEntryAction(); // check the transitions in the current state triggeredTransition = NONE for (transition in _currentState.getTransitions()) if (transition.isTriggered()) triggeredTransition = transition break; // if we have found an active transition if (triggeredTransition != NONE) result = UpdateResult() result.actions = [] result.transition = triggeredTransition result.level = triggeredTransition.getLevel() else // pass it on down result = _currentState.update() . . . To be continued

24. public class HeirarchicalStateMachineextends HSMBase { _currentState; . . . function update() . . . // variable result contains transition information if (result.transition) // decide what to do based on the level if (result.level = 0) // transition internal to this machine. Handle it now. targetState = result.transition.getTargetState() result.actions += _currentState.getExitAction() result.actions += result.transition.getAction() result.actions += targetState.getEntryAction() _currentState = targetState // this may be an instance of a state, so add our action result.actions += getAction() result.transition = NONE else if (result.level > 0) // destined for a higher level, so we exit our state result.actions += _currentState.getExitAction() _currentState = null result.level -= 1 else // the transition needs to be passed down targetState = result.transition.getTargetState() targetMachine = targetState.parent result.actions += result.transition.getAction() result.actions += targetMachine.updateDown( targetState, -result.level) result.transition = NONE else result.action += getAction() return result

25. publicclass HeirarchicalStateMachineextendsHSMBase { _currentState; . . . function updateDown(state, level) // if not at a top level, continue recursing if (level > 0) // pass ourselves off as the transition state actions = parent.updateDown(this, level-1) else actions = [] if (_currentState) actions += _currentState.getExitAction() currentState = state actions += state.getEntryAction() return actions

26. Other authors actually call this state blittingIf there are relatively few emergency transitions then have the FSM remember on transitions where it came from