Decision Trees & Rule-based AI

Decision Trees & Rule-based AI

Decision Tree • Advantages • Fast and easy to implement, Simple to understand • Modular, Re-usable • Can be learned  can be constructed dynamically from observations and actions in game, we will discuss this further in a future topic called ‘Learning’)

Decision Tree • Problem Setting • Given a set of knowledge, we need to generate a corresponding action from a set of possible actions • Some actions are triggered by a large set of inputs • E.g. For an ant AI to Evade, it requires player to be < 20 units away, player’s health to be > 50% and ant’s health to be < 25%. There are 3 input conditions. • Some of these input conditions may be more significant among the entire set of inputs • Computational redundancy

Decision Tree • Problem Setting • We need a method to group lots of inputs together under each action • We need to allow significant inputs to control the output actions (also non-significant inputs should be less utilized)

Decision Tree • A Decision Tree (DT) is made up of connected decision points • Tree has starting decision (root) • For each decision (starting from root), one of a set of ongoing options is chosen • Choice is made based on conditions derived from the character’s knowledge/values • Continue along the tree until the decision process has no more decisions to make • Each leaf is an action, to be executed

Decision Tree • A sample decision tree of a soldier character Root Input Conditions (Decision Points) Leaf

Decision Tree • The action for the character is determined by going through a series of decision points

Decisions • Decisions in a tree should be simple • Check on a single value or Boolean value (do not normally join inputs with Boolean logic) • Possible types of decisions and their data types • Boolean – True/False • Enumeration – Matches one of the given set of values • Numeric value – Value within given range • Vector (2D/3D) – Vector has length within a given range (for distance checking)

Combinations of Decisions • A decision tree is efficient because the decisions are simple – only one test condition at a time • When Boolean combination of tests (AND/OR) are required, some tree structures can be used to represent them

Combinations of Decisions • To AND two decisions together, place them in series, the 1st decision needs to be true to consider the 2nd, in order to get to action 1 • To OR two decisions together, place them in series, either 1st or 2nd decision can be true in order to carry out action 1

Decision Complexity • In a tree structure, the number of decisions that need to be considered is usually smaller than the number of decisions in the tree • Complexity issue? Space complexity? Time complexity?

Branching in DTs • Deep binary DT • The same value (color) may be checked up to 3 times • Slightly better: Order the checks so that the most likely state comes first • Flat DT with 4 branches • Using 4 branches, the structure is flatter and requires only one decision check • More efficient!

Binary Better? • It is still more common to find binary DT implementations • Underlying nature of codes usually simplifies down to a series of binary tests (if/else) • Speed savings not significantly better with higher order branching • Binary DTs are easier to optimize • Many tree optimization/compression techniques are for binary trees • Learning algorithms usually use binary DT

Performance • DTs (binary) usually take no memory and performance is linear with the number of nodes visited • Ideal case  If each decision takes a constant time and tree is balanced, performance is O(log2 n), where n is the number of decision nodes

Balancing the Tree • DTs run the fastest when trees are balanced • A balanced tree has about the same number of leaves on each branch • Both these trees have 8 behaviors and 7 decisions, but one is extremely unbalanced

Balanced vs. Unbalanced Tree • At its worst, performance of a severely unbalanced tree goes from O(log2 n) to O(n) • Although a balance tree is theoretically optimal, it may not be the fastest tree… • Why is this so?

Maximizing Performance? • Structuring a tree for maximum performance is a difficult task to accomplish • Since DTs are fast anyway, it is rarely important to squeeze out every drop of speed • General guidelines: Balance a tree (as balanced as possible), make commonly used branches shorter than rarely used ones, put expensive decisions late

Random Decision Tree • Sometimes, we want to inject a bit of randomness into decision making… • Randomly plugging in randomness into a DT can result in erratic behavior, especially when DTs are run frequently in the game loop (usually the case) • Imagine, a soldier randomly operating between guarding and attacking very frequently….

Random Decision Tree • To introduce random choices in a DT, decision making process needs to be stable • Rule: If there is no relevant changes in world state, there should be no change in decision • Consecutive frames should stay with the chosen random decision until some world state changes • Implementation: Allow the random decision to keep track of what it did last time, so that it knows the previous choice to take when the same decision is encountered.

Random Decision Tree • In the first decision (if not under attack), choose randomly to patrol or stand still • Subsequently, continue on with the previously chosen action • If the parent decision takes the ‘under attack’ choice (a different branch), get rid of the stored choice • Repeat…

Random Decision Tree • If the AI continues to do the same thing forever (because it is never under attack?), that may look strange too… • Use a time-out scheme (a stop timer) to reset the previous action, and initiate a new random choice • How about randomizing the timer as well…? 

Combining DT with FSM • We can replace transitions from a state (to another state) with a DT • The leaves are the actual transitions to new states

Combining DT with FSM • Note: If it cannot see the player, the transition (via the DT) ends, and no new state is reached • Otherwise, it tests for the player proximity and makes a transition to the “Raise Alarm” or “Defend” states

Combining DT with FSM • This FSM implements the same thing (as prev. slide), but without the DT nodes • Now, we have two complex conditions that need to be evaluated • If the condition involved a time-consuming test (such as LoS), then adding the DT would be much more efficient

Rule-based AI • Generally refer to AI systems that consist of a set of if-then (or if-else) style rules • Technically, FSMs and DTs are types of rule-based systems. Rules are used to handle state transitions and decision nodes • But more specifically, “rule-based systems” are also commonly referred to its usage in expert systems

Rule-based Expert Systems • Common usages/applications in real life: • Medical diagnosis, fraud protection, etc. • Advantage: • Rule-based systems can mimic the way people think and reason given a set of known facts and knowledge about a particular domain • Fairly easy to program and manage (in a computer application)

Rule-based Systems for Games • Rule-based systems are useful in GAMES… • Because knowledge encoded in rules is modular • Rules can be encoded in any order  flexible for coding and modifying the system at a later time • Let’s look at a game example that can use a rule-based system…

Example: Medieval RTS game • Technology Tree • An important element in RTS games • Shows the links between units to train, facilities to build and resources to harvest in order to expand influence in game

Example: Medieval RTS game • Aim: Enable the computer opponent to keep track of player’s current state of technology • By collection of knowledge of the player from the world state (resources, units, facilities) • “Cheating” and having perfect knowledge will not give fair and realistic AI behaviors • How to assess state of technology? • Sending scouts to collect information and observe (just like what human players do) • Make inferences via a rule-based system

Rule-based System Basics • Two main components • Working memory – Stores known facts and assertions made by the rules • Rules memory – Contains if-then style rules that operate over the facts stored in working memory • As rules as triggered or fired, • they can trigger some kind of action (such as in FSM and DT), or • they can modify contents of the working memory by adding new information

Rule-based System Basics • Sample working memory enumTMemoryValue{Yes, No, Maybe, Unknown}; TMemoryValue Peasants; TMemoryValue Woodcutter; TMemoryValue Stonemason; TMemoryValue Blacksmith; TMemoryValue Barracks; ...... • Contains elements that can take any one of the 4 values • Idea: Keep track of the current “perception” of the player’s state of technology

Rule-based System Basics • Computer can gather facts by sending out scouts to see if a player has built a temple (for e.g.) Temple element will be set to Yes. • In another way, we can use a set of if-then style rules to infer the technology that the player has (before a scout confirms it) • Example “temple” rule: if(Woodcutter == Yes && Stonemason == Yes && Temple == Unknown) Temple = Maybe

Rule-based System Basics • Inference can work the other way as well • If the player has been observed to have a priest, it can be inferred that the player also must have a temple, therefore, must have a barracks, a woodcutter, and a stonemason • Example “priest” rule: if(Priest == Yes) { Temple = Yes; Barracks = Yes; Woodcutter= Yes; Stonemason= Yes; }

Rule-based System Basics • You can have many more rules for this technology tree (More examples in textbook) • The main scheme: To write this set of rules and execute them continuously during the game (at each iteration of the game loop or in fixed intervals) • Maintain an up-to-date picture of the player’s technology capabilities • This knowledge can be used to decide when to attack/defend, what to build next and make other tactical/strategic plans

Rule-based System Basics • In reality, developers to not build rule-based systems using actual hard-coded if-statements • Some types of inferences are hard to achieve • Very inflexible as the rules need to be handcrafted one by one to strike a good balance among them • Definitely not efficient for future modifications! • Developers often use scripting languages or shells to create and modify rules without having to change the source code and recompile

Inference in Rule-based Systems • Forward Chaining • Match rules (if-parts) to facts in working memory • If a rule matches, it is fired and its then-part is executed • Potentially, if more than one rule matches the given set of facts, conflict resolutionphase required to figure out which rule to fire • Conflict resolution: Many possible ways – (1) first matched rule, (2) random rule, (3) largest weighted rule

Inference in Rule-based Systems • Forward Chaining • Example • If working memory indicates Peasants = Yes and Woodcutter = Unknown, the rule: if(Peasants == Yes && Woodcutter == Unknown) Woodcutter = Maybe; , matches. So, this rule can potentially be fired (depending on which other rules are also matched)

Inference in Rule-based Systems • Backward Chaining • Opposite of forward chaining • Match the then-parts, start with outcome/goal, and figure out which rules must be fired to arrive at the outcome/goal • E.g. Outcome is that the player has Calvary units. Work backwards – player must have Blacksmith to have Calvary, player must have Barracks to have Blacksmith, player must have Woodcutter to have Barracks and so on.

Inference in Rule-based Systems • Backward Chaining • So, all the rules required to reach the outcome are all fired. • Work the logic backward up the technology tree to the goal • In practice, backward chaining is recursive and more difficult to implement than forward chaining

Optimization of RBS • For small rule sets, • Forward chaining is fast • For large-scale rule-based systems, optimization is essential • Many rules may be matched for firing, so conflict resolution phase must be optimized  Rete Algorithm • Write your own scripting language (instead of 3rd party ones) to reduce implementation overhead

Decision Trees & Rule-based AI