1 / 42

# Game Playing and Game Programming (Chapter 6) - PowerPoint PPT Presentation

Game Playing and Game Programming (Chapter 6). Outline. Strategies The minimax algorithm alpha-beta pruning Expectiminimax Applications of AI in game programming. Game vs Search Problems. “Unpredictable” opponent”  solution is a strategy specifying a move for every opponent play

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

## PowerPoint Slideshow about ' Game Playing and Game Programming (Chapter 6)' - laddie

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Game Playing and Game Programming(Chapter 6)

BUSI 0088 Handout 5 - Game Playing and Game Programming

• Strategies

• The minimax algorithm

• alpha-beta pruning

• Expectiminimax

• Applications of AI in game programming

BUSI 0088 Handout 5 - Game Playing and Game Programming

• “Unpredictable” opponent”  solution is a strategy specifying a move for every opponent play

• Because of time limits, it is unlikely to find all possibilities. Must approximate based on algorithms.

BUSI 0088 Handout 5 - Game Playing and Game Programming

• Single player vs two players vs multiple players

• Zero-sum vs non-zero-sum

• Perfect information vs imperfect information

• Deterministic vs chance

BUSI 0088 Handout 5 - Game Playing and Game Programming

deterministic

chance

perfect information

imperfect information

BUSI 0088 Handout 5 - Game Playing and Game Programming

• Considering games with 2 players called MAX and MIN. MAX moves first, and then they take turns moving until the game is over.

• A game can be formally defined as a search problem with the following components:

• Initial state: the board position and identifies the player to move.

• Successor function: a list of (move, state) pairs

• Terminal test: which determines when the game is over

• Utility function: Gives a numeric value for the terminal states. For example, in chess, the outcome is win, loss, or draw, which can be given a value 1, -1, or 0.

BUSI 0088 Handout 5 - Game Playing and Game Programming

• The initial state and the legal moves for each side define the game tree for the game.

• Consider the game of tic-tac-toe. From the initial state, MAX has nine possible moves.

• MAX and MIN take turns until we reach leaf nodes corresponding to terminal states — one player has three in a row or all the squares are filled.

BUSI 0088 Handout 5 - Game Playing and Game Programming

*The utility values are for MAX

BUSI 0088 Handout 5 - Game Playing and Game Programming

• MAX moves first and can choose the path a1, a2, or a3. MIN then makes the second move. The utility values are the values for MAX.

BUSI 0088 Handout 5 - Game Playing and Game Programming

• What is the optimal strategy?

• Roughly speaking, an optimal strategy leads to the best outcome assuming one is playing against the best opponent.

BUSI 0088 Handout 5 - Game Playing and Game Programming

• Given a game tree, the optimal strategy can be determined by examining the minimax value of each node

• Consider zero-sum games: MAX’s heaven is MIN’s hell.

• The minimax value of a node is the utility (for MAX) of being in the corresponding state, assuming that both players play optimally from there to the end of the game.

• Obviously, the minimax value of a terminal state is just its utility. At other nodes, MAX will prefer to move to a state of maximum value, whereas MIN prefers a state of minimum value.

BUSI 0088 Handout 5 - Game Playing and Game Programming

• MINIMAX-VALUE(n) = UTILITY(n) if n is a terminal node

maxsSuccessors(n) MINIMAX-VALUE(s) if n is a MAX node

minsSuccessors(n) MINIMAX-VALUE(s) if n is a MIN node

BUSI 0088 Handout 5 - Game Playing and Game Programming

• Example with the 2-ply game

• The minimax value at B = min(3, 12, 8) = 3

• The minimax value at C = min(2, 4, 6) = 2

• The minimax value at D = min(14, 5, 2) = 2

• The minimax value at A = max(3, 2, 2) = 3

• Uses a simple recursive computation of the minimax values of each successor state by performing a complete depth-first search of the game tree.

• Chooses the move with the highest minimax value.

BUSI 0088 Handout 5 - Game Playing and Game Programming

• If the maximum depth of the tree is m and there are b legal moves at each node, the time complexity of the minimax algorithm is O(bm)

• For real games, the time cost is impractical, e.g., in chess, b 35 and m  100

• Suppose the computer has 100 seconds to “think”, and it can explore 10,000 nodes per second  106 nodes per move

• bm = 106,b = 35  m = 4

• 4-ply  human novice

• 8-ply  typical PC, human master

• 12-ply  Deep Blue, Kasparov

BUSI 0088 Handout 5 - Game Playing and Game Programming

• To compute the correct minimax decision without looking at every node in the game tree.

• Prunes away branches that cannot possibly influence the final decision.

• Always returns the same move as the standard minimax.

BUSI 0088 Handout 5 - Game Playing and Game Programming

BUSI 0088 Handout 5 - Game Playing and Game Programming

BUSI 0088 Handout 5 - Game Playing and Game Programming

BUSI 0088 Handout 5 - Game Playing and Game Programming

BUSI 0088 Handout 5 - Game Playing and Game Programming

BUSI 0088 Handout 5 - Game Playing and Game Programming

• Good ordering of moves can improve effectiveness of pruning.

• With “perfect ordering”, time complexity = O(bm/2)

• Double depth of search

• Can easily reach depth 8 and play good chess

BUSI 0088 Handout 5 - Game Playing and Game Programming

• Why is it called Alpha-Beta Pruning (- pruning)?

•  = the value of the best choice we have found so far for MAX

•  = the value of the best choice we have found so far for MIN

• Alpha-beta search updates the values of  and  as it goes along and prunes the remaining branches at a node as soon as the value of the current node is known to be worse than the current  or  value for MAX or MIN, respectively.

BUSI 0088 Handout 5 - Game Playing and Game Programming

• In general, if m is better than n for the player, we will never get to n in play.

BUSI 0088 Handout 5 - Game Playing and Game Programming

• In many games there are chances introduced by dice, card-shuffling, etc.

• A simple example with coin-flipping:

BUSI 0088 Handout 5 - Game Playing and Game Programming

• Expectiminimax gives optimal play.

• Just like minimax, except that we must also handle chance nodes.

• EXPECTIMINIMAX-VALUE(n) = UTILITY(n) if n is a terminal node

maxsSuccessors(n) EXPECTIMINIMAX-VALUE(s)

if n is a MAX node

minsSuccessors(n) EXPECTIMINIMAX-VALUE(s)

if n is a MIN node

sumsSuccessors(n) P(s)  EXPECTIMINIMAX-VALUE(s)

if n is a chance node

• P(s) = the probability that s will be chosen based on the chance node

BUSI 0088 Handout 5 - Game Playing and Game Programming

• Checkers

• In 1952, Arthur Samuel of IBM developed a checkers program that learned its own evaluation function by playing itself thousands of times. Starting as a novice, it improved very well after a few days and defeated Samuel. 10 years later it defeated Robert Nealy, a human champion. Samuel’s computer had 10,000 words of main memory, magnetic tape for long-term storage, and a 0.001 MHz processor.

• In 1994, the human world champion was defeated by a computer defining perfect play for all possible positions involving 8 or fewer pieces on board (around 443 billion positions).

BUSI 0088 Handout 5 - Game Playing and Game Programming

• Chess: In 1997, Deep Blue, developed at IBM, defeated Kasparov in a six-game exhibition match. Deep Blue searches 200 million positions per seconds, uses very sophisticated evaluation, and undisclosed methods for extending some lines of search up to 40 ply.

• Othello: Defeated human world champion by six games to none. It is generally acknowledged that humans are no match for computers at Othello.

• Go: The board is 19x19, making it too daunting for regular search methods (b > 300). There are some better programs these days.

BUSI 0088 Handout 5 - Game Playing and Game Programming

• We have only considered two-player zero-sum game. Game theory can be applied in non-zero-sum games and multiplayer games. Collaboration (teaming up) may be needed in multiplayer games.

• Risk analysis: how much risk should the computer take?

• High level reasoning or planning can eliminate the search space significantly.

BUSI 0088 Handout 5 - Game Playing and Game Programming

• Finite State machines

• Behaviors of computer characters

• Prediction and Learning

• Artificial Life

BUSI 0088 Handout 5 - Game Playing and Game Programming

• Finite State Machines (FSMs) are the most commonly used techniques in AI game programming.

• FSM is a concise, nonlinear description of how an object can change its state over time.

BUSI 0088 Handout 5 - Game Playing and Game Programming

• Example of a simple FSM for a computer-programmed opponent (CPO)

Enemy in sight

Walk around

Fight

Food in sight

Finish eating

Low energy

Cornered

Flee

Eat Food

Food in sight

BUSI 0088 Handout 5 - Game Playing and Game Programming

• Finite State Machines can be improved by using randomness and probabilities  Fuzzy State Machines

BUSI 0088 Handout 5 - Game Playing and Game Programming

• Pathfinding is often needed in game programming. E.g., asking a CPO (computer-programmed opponent) or NPC (non-player character) to go to a specific position on the map.

• Usually based on A* search

• Need to avoid obstacles

• Need to make sure the entities are not trapped in loop

• Need to control the entities to move together (flocking)

BUSI 0088 Handout 5 - Game Playing and Game Programming

• The characters need to be aware of:

• The player (the most important!)

• Environment (don’t walk through a wall)

• Team management (ensure that the members are working as a team)

BUSI 0088 Handout 5 - Game Playing and Game Programming

• Behavior can be goal-directed (using the agent concept).

• For example, in Age of Empires II, the following rule was used in the AI Script by the CPO of the game:

(defrule

(goal resource-needed WOOD)

(current-age == dark-age)

(civilian-population >= 10)

(not (strategic-number sn-wood-gatherer-percentage = 40) )

=>

(set-strategic-number sn-wood-gatherer-percentage 40)

(set-strategic-number sn-food-gatherer-percentage 60) )

BUSI 0088 Handout 5 - Game Playing and Game Programming

• Sometimes the computer can’t be too tough; otherwise the player will never win.

• Artificial stupidity is often introduced, usually in the easy mode of the game.

• Missing the first attack

• Warning the player before attacking

• Not attacking at the same time

BUSI 0088 Handout 5 - Game Playing and Game Programming

• Needs to predict what the users will do next (e.g., punching in a fighting game, passing or shooting the ball in soccer, running in a certain direction).

• In the past, many computer games cheat by having more resources (e.g., Age of Empires) or allowing illegal moves (e.g., Street Fighter). This is less appreciated by players now and more sophisticated AI techniques are used in games.

BUSI 0088 Handout 5 - Game Playing and Game Programming

• For example, in a street fighting game, the player’s actions can be accumulated into a move history. It is possible to record the player’s movement for statistical prediction. If the following statistics are obtained, then we can better predict the player’s moves.

Player sequence Occurrences Frequency

Low Kick, Low Punch, Uppercut 10 50%

Low Kick, Low Punch, Low Punch 7 35%

Low Kick, Low Punch, Low Kick 3 15%

• Virtua Fighter 4 uses a similar technique.

BUSI 0088 Handout 5 - Game Playing and Game Programming

• In recent years, there are more AI techniques (like neural networks, genetic algorithms, reinforcement learning) being used for learning in game programming.

• Some examples

• Creatures (GA and NN)

• Cloak, Dagger, and DNA (GA, can be freely downloaded at: http://www.gameai.com/src/cddna.zip)

• Black & White (NN)

• Colin McRae Rally 2.0 (NN)

• World Championship Snooker 2003 (NN)

• Fields of Battle (simulated annealing and NN)

BUSI 0088 Handout 5 - Game Playing and Game Programming

• Making artificial lives in computers.

• Examples:

• Princess Maker

• The Sims

• Petz (the Dogz and Catz screensavers).

• Creatures

• Theme Hospital

• Each A-life has his own goals, behaviors, personalities, emotions, and even a family and career.

BUSI 0088 Handout 5 - Game Playing and Game Programming

• In The Sims, even the objects have their own behaviors. For example, a smart microwave oven knows what it can accomplish (cook food) and how it should be used (open door, place food inside, close door, …).

• Agents can use the objects with which they were never programmed to interact.

BUSI 0088 Handout 5 - Game Playing and Game Programming