how computers play games with you l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
How computers play games with you PowerPoint Presentation
Download Presentation
How computers play games with you

Loading in 2 Seconds...

play fullscreen
1 / 51

How computers play games with you - PowerPoint PPT Presentation


  • 294 Views
  • Uploaded on

How computers play games with you CS161, Spring ‘03 Nathan Sturtevant Outline Historic Examples Classes of Games Algorithms Minimax - pruning Other techniques Multi-Player Games Successful Game Programs Checkers Chinook 1992 Tinsley won 40-game match, 4-2-34

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'How computers play games with you' - JasminFlorian


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
how computers play games with you

How computers play games with you

CS161, Spring ‘03

Nathan Sturtevant

outline
Outline
  • Historic Examples
  • Classes of Games
  • Algorithms
    • Minimax
    • - pruning
  • Other techniques
  • Multi-Player Games
successful game programs
Successful Game Programs
  • Checkers
    • Chinook
      • 1992 Tinsley won 40-game match, 4-2-34
      • 1994 Tinsley withdrew due to health reasons
      • 444 billion move end-game database
  • Chess
    • Kasparov is currently the best human
    • 1997 Deep Blue won exhibition match 2-1-3
    • 2003 Deep Junior played to a draw
game programs continued
Game Programs (continued)
  • Othello (Reversi)
    • 1997, Logistello beat Murakami 6-0 (264/120)
  • Scrabble
    • Maven
      • 1998 played Adam Logan, won 9-5
      • Came back from down 98 to win with MOUTHPART
  • Awari (Mancala)
    • Solved in 2002 - draw
    • http://awari.cs.vu.nl/
overview types of games
Overview - Types of Games
  • Single-Agent Search
    • 1 player v. a difficult problem
    • Defined by:
      • Start state
      • Successor function
      • Heuristic function
      • Goal test
overview types of games6
Overview - Types of Games
  • Game Search (Adversary Search)
    • Defined by:
      • Initial State
      • Successor function
      • Terminal Test
      • Utility / payoff function
        • Similar to heuristics in single agent problems
chinese checkers
Chinese Checkers
  • Based on European game Halma
  • Americans called it Chinese Checkers
    • 1 player game?
    • 2 player game?
    • Multi-player game?
classes of games
Classes of Games
  • Deterministic v. Non-deterministic
    • Chess v. Backgammon
  • Perfect Information v. Imperfect information
    • Checkers v. Bridge
  • Zero-sum (strictly competitive)
    • Prisoners dilemna
      • Non-zero sum
how do we simulate games
How do we simulate games?
  • Build a game tree
    • Start state at the root
    • All possible moves as children
how do we choose our move
How do we choose our move?
  • Apply utility function at the leaves of the tree
    • In tic-tac-toe, count how many rows and columns are occupied by each player and subtract
tic tac toe17

Me

Opponent

Tic-Tac-Toe

Utility = 3

x: 2r 2c 2d

o: 2r 2c 0d

x: 2r 2c 2d

o: 2r 1c 1d

x: 2r 3c 2d

o: 2r 1c 1d

x: 3r 3c 2d

o: 2r 2c 0d

Utility = 3

Utility = ∞

Utility = 2

Utility = 3

Utility = ∞

Utility = 2

what is our algorithm
What is our algorithm?
  • Apply utility function at the leaves of the tree
    • In tic-tac-toe, count how many rows and columns are occupied by each player and subtract
    • Back-up values in the tree
  • This calculates the “minimax” value of a tree
minimax

Maximizer

Minimizer

2

3

2

Minimizers strategy

Minimax

3

1 - ply

3

1 - ply

minimax properties
Minimax - Properties
  • Complete?
    • Yes - if tree is finite
  • Optimal?
    • Yes - against an optimal opponent
  • Time Complexity?
    • O(bd)
  • Space Complexity?
    • O(bd)
minimax21
Minimax
  • Assume our computer can expand 105 nodes/sec
  • Assume we have 100 seconds to move
  • 107 nodes/move
  • Tic-tac-toe
    • 9! = 362880 (naïve) ways to play a game (b=4)
    • 39 = 19683 possible states (upper bound) on a board
  • Chess
    • b = 35, d = 100, must search 2154 nodes
minimax issues
Minimax - issues
  • Evaluation function
    • Where does it come from?
      • Expert knowledge
        • Chess: material value
        • Othello (reversi): positional strength
      • Learned information
      • Pre-computed tables
  • Quiescence
minimax issues24
Minimax - issues
  • Quiescence
    • We don’t see the consequences of our bad choices
    • quiescence search
  • Horizon problem
    • We avoid dealing with a bad situation
minimax25
Minimax
  • In Chess
    • b = 35
    • 107 nodes/move
    • Can search 4-ply into tree (human novice)
    • Good humans can search 8-ply
    • Kasparov searches about 12-ply
  • What to do?
    • - pruning
minimax26

Maximizer

Minimizer

2

3

2

Minimizers strategy

Minimax

3

1 - ply

3

1 - ply

pruning
- pruning
  •  = lower bound on Maximizer’s score
    • Start at -∞
  •  = upper bound on Minimizer’s score
    • Start at ∞
slide28

Maximizer

Minimizer

1

 = -∞

 = ∞

 = -∞

 = ∞

 = -∞

 = ∞

≥1

-∞

slide29

Maximizer

Minimizer

1

2

 = -∞

 = ∞

 = -∞

 = ∞

 = 1

 = ∞

≥1

2

-∞

slide30

Maximizer

Minimizer

1

2

 = -∞

 = ∞

 = -∞

 = ∞

 = 2

 = ∞

2

-∞

slide31

Maximizer

Minimizer

1

3

 = -∞

 = ∞

 = -∞

 = 2

≤ 2

 = -∞

 = 2

≥ 3

2

2

-∞

slide32

Maximizer

Minimizer

1

3

 = -∞

 = ∞

≥ 2

 = -∞

 = 2

2

 = 3

 = 2

≥ 3

2

2

-∞

slide33

Maximizer

Minimizer

 = 2

 = ∞

 = 2

 = ∞

5

 = 2

 = ∞

≥ 2

2

≥ 3

≥ 5

2

1

2

3

-∞

slide34

Maximizer

Minimizer

 = 2

 = ∞

 = 5

 = ∞

5

6

 = 2

 = ∞

≥ 2

2

≥ 3

≥ 5

2

6

1

2

3

-∞

slide35

Maximizer

Minimizer

 = 2

 = ∞

≥ 2

 = 2

 = 6

≤ 6

2

 = 2

 = 6

≥ 3

≥7

2

6

1

2

3

5

6

7

-∞

slide36

Maximizer

Minimizer

 = 2

 = ∞

≥ 2

6

 = 2

 = 6

≤ 6

2

6

 = 7

 = 6

≥ 3

≥7

2

6

1

2

3

5

6

7

-∞

pruning37
- pruning
  • Complete?
    • Yes - if tree is finite
  • Optimal?
    • Computes same value as minimax
  • Time Complexity?
    • Best case O(bd/2)
    • Average case O(b3d/4)
pruning38
- pruning
  • Effectiveness depends on order of moves in tree
    • In practice, we can usually get best-case performance
  • Chess
    • Before we could search 4-ply into tree
    • Now we can search 8-ply into tree
other techniques
Other Techniques
  • Transposition tables
  • Opening / Closing book
transposition tables
Transposition Tables
  • Only using linear about of memory
    • Search only takes a few kb of memory
  • Most games aren’t trees but graphs
transposition tables42
Transposition Tables
  • A lot of duplicated effort
  • Transposition tables hash game states into table
    • Store saved minimax value in table
  • Pre-compute & store values
    • Opening book
    • Closing book
multi player games
Multi-Player Games
  • 2-Player game trees have a single minimax value
  • Games with ≥ 2 players use a n-tuple of scores
    • ie (3, 2, 5)
  • The sum of values in every tuple should be constant
max n

1

2

2

2

3

3

3

3

3

3

(7, 3, 0)

(3, 2, 5)

(0, 10, 0)

(4, 2, 4)

(1, 4, 5)

(4, 3, 3)

Maxn

(3 Players)

(7, 3, 0)

(7, 3, 0)

(0, 10, 0)

(1, 4, 5)

can we prune max n trees
Can we prune maxn trees
  • In minimax we bound the game tree value
  • In maxn we bound based on sum of values
    • All scores sum to 10
    • If Player 1 gets 7 points…
    • Player 2-3 will get ≤ 3 points
shallow max n pruning

1

2

2

2

3

3

3

3

(7, 3, 0)

(3, 2, 5)

(0, 10, 0)

(1, 4, 5)

Shallow Maxn Pruning

(3 Players)

(7, 3, 0)

(≥7, ≤3, ≤3)

(7, 3, 0)

(0, 10, 0)

(≤6, ≥4, ≤6)

shallow max n pruning47
Shallow Maxn Pruning
  • Complete?
    • Yes
  • Optimal?
    • Yes*
  • Time Complexity?
    • Best-case**: bd/2
    • Average-case: bd
  • Space Complexity?
    • b•d
max n pruning
Maxn Pruning
  • Why is maxn weak in practice?
    • Only compares 2 scores out of n players
    • Relies on game evaluation properties, not ordering
  • Last-Branch Pruning
  • Speculative Pruning
last branch speculative pruning

(3, 3, 4)

2

2

3

(1, 4, 5)

3

(2, 4, 4)

1

Last-Branch/Speculative Pruning

(3 Players)

(3, 3, 4)

1

2

last branch spec pruning
Last Branch/Spec. Pruning
  • Best case: O(bd·(n-1)/n)
    • As b gets large
    • Dependent only on node ordering in tree
    • http://www.cs.ucla.edu/~nathanst/ for more info
imperfect information
Imperfect Information
  • Most card games have imperfect information
  • We can use monte-carlo simulation
    • Create many consistent samples of possible opponent hands
    • Solve using perfect-information methods
    • Combine results together to make next move