1 / 24

Artificial Intelligence in Game Design

Artificial Intelligence in Game Design. N-Grams and Decision Tree Learning. Predicting Player Actions. Type of games where works best: Player has choice between few possible actions Attack left Attack right Character can take simultaneous counteraction

Download Presentation

Artificial Intelligence in Game Design

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Artificial Intelligence in Game Design N-Grams and Decision Tree Learning

  2. Predicting Player Actions Type of games where works best: • Player has choice between few possible actions • Attack left • Attack right • Character can take simultaneous counteraction • Each player action has “correct” counteraction • Attack left  defend left • Attack right  defend right • Goal: Character should learn to “anticipate” current player action based on past actions

  3. Probabilistic Approach • Keep track of last n actions taken by player • Size of n is “window” of character memory • Compute probability of next player action based on these • Base decision about counteraction on that probability • Example: Last 10 player actionsL L R L R L L L R L • Estimated probabilities of next action:

  4. Probabilistic Actions • Simple majority approach • Since left attack has highest probability, defend left • Problem: Character will take same action for long periods • Too predictable! • Example: • Player’s next 4 attacks are from right • Character will still choose defend left since left attacks still have highest probability • Character looks very stupid! player: L L R L R L L L R L R R R Rcharacter: L L L L Left attack still majority player action in last 10 moves

  5. Probabilistic Actions • Probabilistic Approach:Choose action with same probability as corresponding player action • Biased towards recent player actions • Far less predictable • Player may notice that character is changing tactics without being aware of how this is done

  6. Window Size • Key question: What is good value for n?L L R L R L L L R L R R L • Can be too small (n = 2, for example)L L R L R L L L R L R R L • Can be too large (n = 20, for example)LLLLLLLLLLLLRRRRRRRR • No best solution • Will need to experiment Size of “window” used to determine probabilities Character has no “memory” of past player actions Too slow to react to changes in player tactics

  7. N-Grams • Conditional probabilities based on sequence of user actions • Example: • “Last two player attacks were left and then right” • “What has player done next after last times the attacked left then right?” • After last 4left then right attacks: • Attacked right3 times • Attacked left1 time • Conclusion: Player has 75% chance of attacking right next

  8. N-Grams Example • Example: • Window of memory = last 12 actions • Base decision on last two actions taken by player (past sequence length = 2) • Goal: Determine what action player is likely to take next given last two actions L R • Previous actions:L L R L R R L L R L L R? • Previous cases of L R: • Followed by L twice • Followed by R once

  9. N-Grams and Sequence Size • Number of statistics to keep grow exponentially with length of past sequences • Number of possible actions = a • Past sequences length = L • Number of possible configurations of past sequences = aL • L must be small (no more than 2 or 3)

  10. Storing N-Grams • Algorithm: • Keep statistics for all possible action strings of length L • Based on “window” of past actions • Example: L L L L R R L L R L L R

  11. N-Grams and Updating • When player takes new action • Add instance for that action and update statistics • Remove oldest action from list and update statistics • Move “window” • Example:L L L L R R L L R L L RL New action Now outside “window”

  12. Decision Trees Simple tree traversal • Node = question • Branch to follow = answer • Leaf = final action to take Can create dynamically from a set of examples Hit points < 5? no yes Obstacle between myself and player? Within one unit of player? yes yes no no Path to playerclear? hide run attack yes no run towardsplayer runsideways

  13. Decision Tree Learning • Based on a set of training examples • Attribute values about current situation • From world or from character state • Target action character should take on basis of those attribute values • Example: Estimating driving time • Hour of departure: 8, 9, or 10 • Weather: sunny, cloudy or rainy • Accident on route: yes or no • Stalled car on route: yes or no • Commute time: short, medium, or long attributes desired action

  14. Training Examples

  15. Decision Tree Building node BuildTree(examples[ ] E) { if (all examples in E have same target action T) { return new leaf with action T } else { choose “best” attribute A to create branches for examples E set question at this node to A for (all possible values V for attribute A) { create branch with value V EV = all examples in E that have valueV for attribute A attach BuildTree(EV ) to that branch } } }

  16. Decision Tree Building • Start tree by using BuildTree(all examples) to create root • Example: Suppose Hour were chosen at root: Examples 1 – 13 4 short, 2 medium, 7 long Hour 10 9 8 Examples 3, 6, 7, 10, 11 4 short, 0 medium, 1 long Examples 4, 5, 8, 9, 130 short, 2 medium, 3 long Examples 1, 2, 120 short, 0 medium, 3 long another question another question Long

  17. ID3 Decision Tree Learning • Choose attribute with highest information gain • Based on definition of entropy from information theory • Entropy over examples E with target actions T:Entropy(E) = - | ET | log2 | ET |T | E | | E | • Example: entropy for training set =-(4/13) log2(4/13) -(2/13) log2(2/13) -(7/13) log2(7/13) = 1.424 short examples 2 medium examples 7 medium examples 

  18. ID3 Decision Tree Learning • Expected entropy remaining after attribute A used = • Sum of entropies of child nodes down branches V • Weighted by percentage of examples down that branch | EV | Entropy (EV)V | E | • Information gain for attribute A = that value subtracted from original entropy 

  19. Best attribute at root

  20. Using ID3 in Games • ID3 efficient enough for on-line learning • Execution time proportional to number of examples • Best applied to games if: • Few possible actions for NPC to choose from(attack, retreat, defend) • Key attributes have few possible values(or continuous values partitioned into predefined ranges) • Have easy way to determine desired actions NPC should take based on significant number of player actions • May require tweaking rules of game!

  21. Black and White Game • Player given “creature” at beginning of game • Creature “trained” by player by observing player actions in different situations • What other armies to attack in what circumstances • Later in game (after creature grows), creature takes those same actions

  22. Black and White Game • Sample training set:

  23. Black and White Game • Tree created from examples: Allegiance friendly enemy No attack Defense weak strong medium Attack No attack Attack

  24. Black and White Game • Nature of game suited to decision tree learning: • Large number of examples created by player actions • Known target actions based on player actions • Small number of possible actions, attribute values • Game specifically designed around algorithm!

More Related