Turn based games
Download
1 / 35

Game Theory: deterministic games - PowerPoint PPT Presentation


  • 901 Views
  • Updated On :

Turn-Based Games sources: http://www.game-research.com/ www.gamespot.com Wikipedia.org Russell & Norvig AI Book; Chapter 5 (and slides) Jonathan Schaeffer’s AAW 05 presentation My own H é ctor Mu ñ oz-Avila Turn-Based Strategy Games

Related searches for Game Theory: deterministic games

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Game Theory: deterministic games' - Ava


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Turn based games l.jpg

Turn-Based Games

  • sources:

  • http://www.game-research.com/

  • www.gamespot.com

  • Wikipedia.org

  • Russell & Norvig AI Book; Chapter 5 (and slides)

  • Jonathan Schaeffer’s AAW 05 presentation

  • My own

Héctor Muñoz-Avila


Turn based strategy games l.jpg
Turn-Based Strategy Games

  • Early strategy games was dominated by turn-based games

    • Derivate from board games

      • Chess

      • The Battle for Normandy (1982)

      • Nato Division Commanders (1985)

  • Turn-based strategy:

  • game flow is partitioned in turns or rounds.

  • Turns separate analysis by the player from actions

  • “harvest, build, destroy” in turns

  • Two classes:

    • Simultaneous

    • Mini-turns


Turn based games continues to be a popular game genre l.jpg
Turn-Based Games Continues to be A Popular Game Genre

  • At least 3 sub-styles are very popular:

    • “Civilization”-style games

      • Civilization IV came out last week

    • Fantasy-style (RPG)

      • Heroes of Might and Magic series

    • Poker games

      • Poker Academy


Some historical highlights l.jpg
Some Historical Highlights

  • 1952 Turing design a chess algorithm. Around the same time Claude Shannon also develop a chess program

  • 1956 Maniac versus Human

  • 1970 Hamurabi. A game about building an economy for a kingdom

  • The Battle for Normandy (1982)

  • 1987 Pirates!

  • 1990 Civilization

  • 1995 HoMM

  • 1996 Civilization II

    • The best game ever?

  • 2005 Civilization IV

  • 2006 HoMM V


Side tracking game design contradicting principles l.jpg
Side-tracking: Game Design: Contradicting Principles

  • Principle: All actions can be done from a single screen.

  • Classical example: Civ IV

  • But: HoMM uses two interfaces: HoMM IV


Coming back how to construct good ai l.jpg
Coming back: How to Construct Good AI?

  • Idea: Lets just use A* and define a good heuristic for the game

    • Search space: a bipartite tree

    • After all didn’t we use it with the 9-puzzle game?

  • Problems with this idea:

  • Adversarial: we need to consider possible moves of our opponent (s)

  • Time limit: (think Chess)


Types of adversarial tbgs from ai perspective l.jpg
Types of AdversarialTBGs (from AI perspective)

Chance

Deterministic

Chess, Go, rock-paper-scissors

Perfect

information

Backgammon, monopoly

Bridge, Poker

Imperfect information

Battleships, Stratego

Civilization, HoMM


Game tree 2 player deterministic turns l.jpg
Game tree (2-player, deterministic, turns)

  • Concepts:

  • State: node in search space

  • Operator: valid move

  • Terminal test: game over

  • Utility function: value for outcome of the game

  • MAX: 1st player, maximizing its own utility

  • MIN: 2nd player, minimizing Max’s utility


Minimax l.jpg
Minimax

  • Finding perfect play for deterministic games

  • Idea: choose move to position with highest minimax value = best achievable payoff against best play

  • E.g., 2-play game:



Properties of minimax l.jpg
Properties of minimax

  • Complete?

  • Optimal?

  • Time complexity?

    • b: branching factor

    • m: # moves in a game

Yes (if tree is finite)

Yes (against an optimal opponent)

O(bm)

  • For chess, b ≈ 35, m ≈100 for "reasonable" gamesTherefore, exact solution is infeasible


Minimax algorithm with imperfect decisions l.jpg

Cutoff-test(state)

evaluationFunction(state)

Cutoff-test(state)

evaluationFunction(state)

Minimax algorithm with Imperfect Decisions


Evaluation function l.jpg

  • Chess

  • weight: Piece  Number

    • (w1) Pawn  1

    • (w2) Knight  3

    • (w3) Bishop  3

    • (w4) Rook  5

    • (w5) Queen  9

  • Function; state  Number

    • f1 = #(pawns,w)  #(pawns,b)

    • f2 = #(knight,w)  #(knight,b)

    • f3 = #(bishop,w)  #(bishop,b)

    • f4 = #(rook,w)  #(rook,b)

    • f5 = #(knight,w)  #(knight,b)

Evaluation Function

  • Evaluation Function

    • Is an estimate of the actual utility

    • Typically represented as a linear function:

      EF(state) = w1f1(state) + w2f2(state) + … + wnfn(state)

    • Example:


Example evaluation function l.jpg
Example: Evaluation Function

“all things been equal”

White moves,

Who is winning?

Is this consistent with Evaluation function?

Black

Yes!


Evaluation function 2 l.jpg
Evaluation Function (2)

  • Obviously, the quality of the AI player depends on the evaluation function

  • Conditions for evaluation functions:

  • If n is a terminal node,

  • Computing EF should not take long

  • EF should reflect chances of winning

EF(n) = Utility(n)

If EF(state) > 3 then is almost-certain that blacks win


Cutting off search l.jpg
Cutting Off Search

  • When to cutoff minimax expansion?

  • Potential problem with cutting off search: Horizon problem

  • Solution:

  • Fixed depth limit

  • Iterative deepening until times runs out

  • Decision made by opponent is damaging but cannot be “seen” because of cutoff

  • Quiescent: states that are unlikely to exhibit wild swings in the values of the evaluation functions


Example horizon problem l.jpg
Example: Horizon Problem

“all things been equal”

White moves,

Who is winning?

Is this consistent with Evaluation function?

Black

No!


Pruning motivation l.jpg
α-β pruning: Motivation

  • A good program may search 1000 positions per second

  • In a chess tournament, a player gets 150 seconds per move

  • Therefore, the program can explore 150,000 positions per move

  • With a branching factor of 34, this will mean a look ahead of 3 or 4 moves

  • Facts:

  • 4-turns ≈ human novice

    • 8-turns ≈ typical PC, human master

    • 12-turns ≈ Deep Blue, Kasparov

  • How to look ahead more than 4 turns? Use α-β pruning


Example l.jpg
Example:

  • Finding perfect play for deterministic games

  • Idea: choose move to position with highest minimax value = best achievable payoff against best play

  • E.g., 2-play game:







Principle of prunning l.jpg

α is the value of the best (i.e., highest-value) choice found so far at any choice point along the path for max

If v α, max will avoid it

Therefore, prune that branch

β is the lowest-value found so far at any choice point along the path for min

If v α, min will avoid it

Therefore, prune that branch

Principle of α-β Prunning


The algorithm l.jpg
The α-β algorithm found so far at any choice point along the path for


The algorithm27 l.jpg
The α-β algorithm found so far at any choice point along the path for


Properties of l.jpg
Properties of α-β found so far at any choice point along the path for

  • Pruning preserves completeness and optimality of original minimax algorithm

  • Good move ordering improves effectiveness of pruning

  • With "perfect ordering," time complexity = O(bm/2)

    Therefore, doubles depth of search

  • Used in PC games today (9 moves look-ahead, Grand Master level)


Deterministic games in practice l.jpg
Deterministic games in practice found so far at any choice point along the path for

  • Checkers: Chinook ended 40-year-reign of human world champion Marion Tinsley in 1994. Used a precomputed endgame database defining perfect play for all positions involving 8 or fewer pieces on the board, a total of 444 billion positions.

  • Chess: Deep Blue defeated human world champion Garry Kasparov in a six-game match in 1997. Deep Blue searches 200 million positions per second, 24 processors, quiescent identified with help of human grand masters

  • Othello: human champions refuse to compete against computers, who are too good.

  • Go: human champions refuse to compete against computers, who are too bad. In go, b > 300, so most programs use pattern knowledge bases to suggest plausible moves.


Additional notes l.jpg
Additional Notes found so far at any choice point along the path for

  • The next 5 slides are form David W. Aha (NRL) presentation at Lehigh University in Fall’04


Example game freeciv chance adversarial imperfect information game l.jpg
Example Game: FreeCiv found so far at any choice point along the path for (Chance, adversarial, imperfect information game)

Civilization II(MicroProse)

  • Civilization II (1996-): 850K+ copies sold

    • PC Gamer: Game of the Year Award winner

    • Many other awards

  • Civilization series (1991-): Introduced the civilization-based game genre

FreeCiv (Civ II clone)

  • Open source freeware

  • Discrete strategy game

  • Goal: Defeat opponents, or build a spaceship

  • Resource management

    • Economy, diplomacy, science, cities, buildings, world wonders

    • Units (e.g., for combat)

  • Up to 7 opponent civs

  • Partial observability

http://www.freeciv.org


Freeciv scenario l.jpg
FreeCiv Scenario found so far at any choice point along the path for

General description

  • Game initialization: Your only unit, a “settler”, is placed randomly on a random world (see Game Options below). Players cyclically alternate play

  • Objective: Obtain highest score, conquer all opponents, or build first spaceship

  • Scoring: “Basic” goal is to obtain 1000 points. Game options affect the score.

    • Citizens: 2 pts per happy citizen, 1 per content citizen

    • Advances: 20 pts per World Wonder, 5 per “futuristic” advance

    • Peace: 3 pts per turn of world peace (no wars or combat)

    • Pollution: -10pts per square currently polluted

  • Top-level tasks (to achieve a high score):

    • Develop an economy

    • Increase population

    • Pursue research advances

    • Opponent interactions: Diplomacy and defense/combat


Freeciv concepts l.jpg
FreeCiv Concepts found so far at any choice point along the path for

Concepts in an Initial Knowledge Base

  • Resources: Collection and use

    • Food, production, trade (money)

  • Terrain:

    • Resources gained per turn

    • Movement requirements

  • Units:

    • Type (Military, trade, diplomatic, settlers, explorers)

    • Health

    • Combat: Offense & defense

    • Movement constraints (e.g., Land, sea, air)

  • Government Types (e.g., anarchy, despotism, monarchy, democracy)

  • Research network: Identifies constraints on what can be studied at any time

  • Buildings (e.g., cost, capabilities)

  • Cities

    • Population Growth

    • Happiness

    • Pollution

  • Civilizations (e.g., military strength, aggressiveness, finances, cities, units)

  • Diplomatic states & negotiations


Freeciv decisions l.jpg
FreeCiv Decisions found so far at any choice point along the path for

Civilization decisions

  • Choice of government type (e.g., democracy)

  • Distribution of income devoted to research, entertainment, and wealth goals

  • Strategic decisions affecting other decisions (e.g., coordinated unit movement for trade)

City decisions

  • Production choice (i.e., what to create, including city buildings and units)

  • Citizen roles (e.g., laborers, entertainers, or specialists), and laborer placement

    • Note: Locations vary in their terrain, which generate different amounts of food, income, and production capability

Unit decisions

  • Task (e.g., where to build a city, whether/where to engage in combat, espionage)

  • Movement

Diplomacy decisions

  • Whether to sign a proffered peace treaty with another civilization

  • Whether to offer a gift


Freeciv cp decision space l.jpg
FreeCiv CP Decision Space found so far at any choice point along the path for

Variables

  • Civilization-wide variables

    • N: Number of civilizations encountered

    • D: Number of diplomatic states (that you can have with an opponent)

    • G: Number of government types available to you

    • R: Number of research advances that can be pursued

    • I: Number of partitions of income into entertainment, money, & research

  • U: #Units

    • L: Number of locations a unit can move to in a turn

  • C: #Cities

    • Z: Number of citizens per city

    • S: Citizen status (i.e., laborer, entertainer, doctor)

    • B: Number of choices for city production

Decision complexity per turn (for a typical game state)

  • O(DNGRI*LU*(SZB)C) ; this ignores both other variables and domain knowledge

    • This becomes large with the number of units and cities

    • Example: N=3; D=5; G=3; R=4; I=10; U=25; L=4; C=8; Z=10; S=3; B=10

    • Size of decision space (i.e., possible next states): 2.5*1065 (in one turn!)

  • Comparison: Decision space of chess per turn is well below 140 (e.g., 20 at first move)


ad