1 / 61

Decision Making AI

John See 20 Dec 2010. Decision Making AI. Decision Making AI. Ability of a game character to decide what to do Decision Making in Millington’s Model. Decision Making AI. Decision Trees Finite State Machines (FSM) Rule-based Systems Fuzzy Logic & Neural Networks Blackboard Architecture.

hallie
Download Presentation

Decision Making AI

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. John See 20 Dec 2010 Decision Making AI

  2. Decision Making AI • Ability of a game character to decide what to do • Decision Making in Millington’s Model

  3. Decision Making AI • Decision Trees • Finite State Machines (FSM) • Rule-based Systems • Fuzzy Logic & Neural Networks • Blackboard Architecture

  4. Decision Trees • Fast, easy to understand • Simplest technique to implement, but extensions to the basic algorithm can be sophisticated • Typically used to control characters, animation or other in-game decision making • Can be learned, and learning is relatively fast (compared to fuzzy logic/NN)

  5. Decision Trees – Problem Statement • Given a set of knowledge, we need to generate a corresponding action from a set of actions • Map input and output – typically, a same action is used for many different sets of input • Need a method to easily group lots of inputs together under one particular action, allowing the input values that are significant to control the output

  6. Decision Trees – Problem Statement • Example: Grouping a set of inputs under an action Enemy is visible Enemy is now < 10m away Attack Enemy is visible Enemy is still far (> 10m), but not at flank Attack Enemy is visible Enemy is still far (> 10m), at flank Move Enemy is not visible, but audible Creep

  7. Decision Trees – Algorithm Overview • Made up of connected decision points • Tree has starting decision, its root • For each decision, starting from the root, one of a set of ongoing options is chosen. • Choice is made based on character’s knowledge (internal/external)  Fast! No prior representation! • Continues along the tree, making choices at each decision node until no more decisions to consider • At each leaf of the tree, an action is attached • Action is carried out immediately

  8. Decision Trees • Check a single value and don’t contain any Boolean logic (AND, OR) • Representative set • Boolean – Value is true • Enumeration – Matches one of the given set of values • Numeric value – Value is within given range • 3D Vector – Vector has a length within given range • Examples?

  9. Decision Trees – Combining Decisions • AND two decisions – place in series in the tree • OR two decisions – also use decisions in series, but the two actions are swapped over from the AND

  10. Decision Complexity • Number of decisions that need to be considered is usually much smaller than number of decisions in the tree. • Imagine using IF-ELSE statements to test each decision? • Method of building DTs: Start with simple tree, as AI is tested in game, additional decisions can be added to trap special cases or add new behaviors

  11. Decision Making - Branching • So far, we have considered only binary trees – decisions choose between 2 options. • It is possible to build DT with any number of options, or different decisions with different number of branches

  12. Decision Making - Branching • Deep DTs may result in a same alert being checked numerous times before a decision is found • Flat DTs are more efficient, requires less decision checking

  13. Decision Making - Branching • Still common to find DTs using only binary decisions • Why? • Underlying code for multiple branches simplifies down to a series of binary tests (IF-ELSE statements) • Binary DTs are easier to optimize. Some learning algorithms that work with DTs require them to be binary

  14. Decision Making – Performance • Takes no memory, performance is linear with number of nodes visited • Assume each decision takes constant amount of time, and tree is balanced, performance: O(log2 n), where n is number of decision nodes in tree • This DOES NOT consider the execution time of the different checks required in the DT, which can vary a lot!

  15. Decision Making – Balancing the Tree • DTs can run the fastest when atree is balanced • A balanced tree keeps the height of its branches approximately equal (within 1 to be considered balanced). In our context, it will have about the same number of decision making levels on each branch

  16. Decision Making – Balancing the Tree • 8 behaviors, 7 decisions • 1st tree – extremely unbalanced, 2nd tree – balanced • To get to H, 1st tree needs 8 decisions, 2nd tree needs 3 only

  17. Decision Making – Balancing the Tree • If all behaviors are equally likely, what is the average number of decisions needed for both trees?

  18. Decision Making – Balancing the Tree • If we were likely to end up at decision A majority of time, which is more efficient? • How do we treat decisions that are time-consuming to run? Let’s say A is the most time-consuming decision…

  19. Decision Making – Merging Patterns • DTs can be extended to allow multiple branches to merge into a new decision – efficient, but some care is needed! • A decision/action can be reached in more than one way • Avoid causing loops, can never find an action leaf

  20. Random Decision Trees • To provide some unpredictability and variation to making decisions in DTs • Simplest way: Generate a random number and choose a branch based on its value • DTs are normally intended to run frequently, reacting to the game state, random decisions can be a problem • What is a potential problem with the following DT?

  21. Random Decision Trees Possible considerations… • Allow random decision to keep track of what it did last time. • When a decision is considered, a choice is made at random, and that choice is stored. Next time the decision is considered, no more randomness, previous choice is maintained. • If something in the world changes, a different decision was arrived, the stored choice should be removed. • Timing Out: Allow the AI to “time out” after a set time, and a random choice is to be made again. Gives variety and realism.

  22. State Machines • Often, characters in a game act in one of a limited set of ways • Carry on doing the same thing until some event or influence makes them change • Can use decision trees, but it is easier to model this behavior using state machines (or finite state machines, FSM) • State machines take into consideration • the world around them • their internal state

  23. State Machines – Basics • Each AI character occupies one state at each instance • Actions or behaviors are associated with each state • So long as the character remains in that state, it will continue carrying out the same actions/behavior • States are connected by transitions • Each transitions leads from one state to another, the target state, and each has a set of associated conditions • Changing states: when the game determines that conditions of a transition are met, the conditions trigger and a new state is fired

  24. State Machines – Simple Example • State machine to model a soldier – 3 states • Each state has its own transitions • The solid circle (with a transition w/o trigger condition) points to the initial state that will be entered when the state machine is first run

  25. State Machines vs. Decision Trees • Now, name some obvious differences in making decisions using decision trees and state machines?

  26. Finite State Machines (FSM) • In game AI, a state machine with this kind of structure (as seen earlier) is usually called a finite state machine (FSM) • An FSM has a finite number of states and transitions • It has finite internal memory to store its states and transitions

  27. FSM – Generic Implementation • Use a generic state interface that keeps track of a set of possible states and records the current state it is in • With each state, a series of transitions are maintained. Each transition is also a generic interface with conditions • At each iteration (game loop), an update function is called to check if any of the transitions from the current state is triggered. • If a transition is triggered, then the transition will be fired • The separation of triggering and firing of transitions allows the transitions to have their own actions

  28. FSM – Generic Implementation • Refer to textbook or other references for a more in-depth code-level implementation of FSMs

  29. FSM – Complexity • State machines only requires memory to hold a triggered transition and the current state • O(1) in memory and O(m) in time, where m is the (average) number of transitions per state • The algorithm calls other supporting functions to perform action and etc. These probably account for most of the time spent in the algorithm. • Hard-coded FSM – inflexible, does not allow level designers the control over building the FSM logic

  30. Hard-coded FSM • Hard-coded FSM – • Consists of an enumerated value, indicating which state is currently occupied, and a function that checks if a transition is followed • States are HARD-CODED, and limited to what was HARD-CODED • Pros – Easy and quick implementation, useful for small FSMs • Cons: • Inflexible, does not allow level designers the control over building the FSM logic • Difficult to maintain (alter) – Large FSMs, messy code • Every character needs to be coded its own AI behaviors…

  31. Hierarchical State Machines • One state machine is a powerful tool, but will still face difficulty expressing some behaviors • Also if you wish to model somewhat different behaviors from more than one state machines for a single AI character • Example: Modeling alarm behaviors with hierarchical s/m (using a basic cleaning robot state machine)

  32. Hierarchical State Machines • If the robot needs to get power if it runs out of power & resume its original duties after recharging, these transition behaviors must be added to ALL existing states to ensure robot acts correctly

  33. Hierarchical State Machines • This is not exactly very efficient. Imagine if you had to add many more concurrent behaviors into your primary state machine?

  34. Hierarchical State Machines • A hierarchical state machine for the cleaning robot • Nested states – could be in more than one state at a time • States are arranged in a hierarchy  next state machine down is only considered when the higher level state machine is not responding to its alarm

  35. Hierarchical State Machines • H* - “history state” node • When the composite state (lower hierarchy) is first entered, the H* node indicates which sub-state should be entered • If composite state already entered, then previous sub-state is restored using the H* node

  36. Hierarchical State Machines • Hierarchical state machine with cross hierarchy transition • Most hierarchical s/m support transitions between levels of the hierarchy • Let’s say we want the robot to go back to refuel when it does not find any more trash to collect…

  37. Hierarchical State Machines • Refer to textbook for more details on its implementation • Performance: • O(n) in memory (n is number of layers in hierarchy) • O(nt) in time, where t is number is number of transitions per state

  38. DT + SM • Combining decision trees and state machines • One approach: Replace transitions from a state with a decision tree • Leaves of DT (rather than straightforward conditions/actions) are now transitions to other states

  39. DT + SM • To implement state machine without decision tree transitions… • We may need to model complex conditions that require more checking per transition • May be time-consuming as need to check all the time

  40. Fuzzy Logic • Founded by LotfiZadeh (1965) • “the essence of fuzzy logic is that everything is a matter of degree” • Imprecision in data… • and uncertainty in solving problems • Fuzzy logic vs. Boolean logic • 50%-80% less rules than traditional rule-based systems, to accomplish identical tasks • Examples: Air-conditioner thermostat or washing machine

  41. Fuzzy Logic in Games • Example of uses: • To control movement of bots/NPCs (to smooth out movements based on imprecise target areas) • To assess threats posed by players (to make further strategic decisions) • To classify player and NPCs in terms of some useful game information (such as combat or defensive prowess)

  42. Crisp data & Fuzzy data • Crisp data (real numbers, value) • Fuzzy data (a predicate or description, with degree value) • Fuzzy logic gives a predicate a degree value. Instead of belonging to a set of being excluded (1 or 0, Boolean logic), everything can partially belong to a set, and some things more belong than others

  43. Fuzzy sets • Fuzzy sets – the numeric value is called the degree of membership (these values are NOT probability values!) • For each set, a degree of membership of 1 given to something completely in the set. Degree membership of 0 given to something completely outside the fuzzy set • Typical to use integers in implementation instead of floating-point values (between 0 and 1), for fast computation in game • Note: Anything can be a member of multiple sets at the same time

  44. Fuzzy Control / Inference Process • 3 basic steps in a fuzzy control or fuzzy inference process

  45. Step 1 - Fuzzification • Mapping process – converts crisp data (real numbers) to fuzzy data (degree of membership) • E.g.: Given a person’s weight, find the degree to which a person is underweight, overweight or at ideal weight

  46. Membership Functions • Membership functions map input variables to a degree of membership, in a fuzzy set between 0 and 1 • Any function can be used, and the shape usually is governed by desired accuracy, the nature of problem, or ease of implementation. • Boolean logic m/f

  47. Membership Functions • Grade m/f • Reverse grade m/f • Triangular m/f • Trapezoid m/f

  48. Membership Functions • Earlier example of using a set of membership functions to represent a person’s weight

  49. Step 2 – Fuzzy rule base • Once all inputs are expressed in fuzzy set membership, combine them using logical fuzzy rules to determine degree to which each rule is true • E.g. • Given a person’s weight and activity level as input, define rules to make a health decision • If overweight AND NOT active then frequent exercise • If overweight AND active then moderate diet • But having a fuzzy output such as “frequency exercise” is not enough – need to quantify the amount of exercise (e.g. 3 hours per week)

  50. Fuzzy rules • Usually uses IF-THEN style rules • If A then B • A  antecedent / premise • B  consequent / conclusion • To apply usual logical operators to fuzzy input, we need the following fuzzy axioms: • A OR B = MAX(A, B) • A AND B = MIN(A, B) • NOT A = 1 – A

More Related