Chapter 7
Download
1 / 58

Computer Chess - PowerPoint PPT Presentation


  • 403 Views
  • Updated On :

Chapter 7 Two Player Perfect-Information Games. Computer Chess A natural domain for studying AI. The game is well structured. Perfect information game. Early programmers and AI researchers were often amateur chess players as well. Brief History of Computer Chess. Maelzel’s Chess Machine

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Computer Chess ' - MartaAdara


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Slide1 l.jpg

Chapter 7

Two Player

Perfect-Information Games


Computer chess a natural domain for studying ai l.jpg
Computer Chess A natural domain for studying AI

  • The game is well structured.

  • Perfect information game.

  • Early programmers and AI researchers were often amateur chess players as well.


Brief history of computer chess l.jpg
Brief History of Computer Chess

  • Maelzel’s Chess Machine

    • 1769 Chess automaton by Baron Wolfgang von Kempelen of Austria

    • Appeared to automatically move the pieces on a board on top of the machine and played excellent chess.

    • Puzzle of the machine playing solved in 1836 by Edgar Allen Poe.


Brief history of computer chess4 l.jpg
Brief History of Computer Chess

Maelzel’s Chess Machine


Brief history of computer chess5 l.jpg
Brief History of Computer Chess

  • Early 1950’s - First serious paper on computer chess was written by Claude Shannon. Described minimax search with a heuristic static evaluation function and anticipated the need for more selective search algorithms.

  • 1956 - Invention of alpha-beta pruning by John McCarthy. Used in early programs such as Samuel’s checkers player and Newell, Shaw and Simon’s chess program.


Brief history of computer chess6 l.jpg
Brief History of Computer Chess

  • 1982- Development of Belle by Condon and Thomson. Belle - first machine whose hardware was specifically designed to play chess, in order to achieve speed and search depth.

  • 1997 - Deep Blue machine was the first machine to defeat the human world champion, Garry Kasparov, in a six-game match.


Checkers l.jpg
Checkers

  • 1952 - Samuel developed a checkers program that learned its own evaluation through self play.

  • 1992 - Chinook (J. Schaeffer) wins the U.S Open. At the world championship, Marion Tinsley beat Chinook.


Othello l.jpg
Othello

  • Othello programs better than the best humans.

  • Large number of pieces change hands in each move.

  • Best Othello program today is Logistello (Michael Buro).


Backgammon l.jpg
Backgammon

  • Unlike the above games backgammon includes a roll of the dice, introducing a random element.

  • Best backgammon program TD -gammon(Gerry Tesauro). Comparable to best human players today.

  • Learns an evaluation function using temporal-difference.


Card games l.jpg
Card games

  • In addition to a random element there is hidden information introduced.

  • Best bridge GIB (M.Ginsberg)

  • Bridge games are not competitive with the best human players.

  • Poker programs are worse relative to their human counterparts.

  • Poker involves a strong psychological element when played by people.


Other games summary l.jpg
Other games - Summary

  • The greater the branching factor the worse the performance.

  • Go - branching factor 361 very poor performance. Checkers - branching factor 4 - very good performance.

  • Backgammon - exception. Large branching factor still gets good results.


Brute force search l.jpg
Brute-Force Search

  • We begin considering a purely brute-force approach to game playing.

  • Clearly, this will only be feasible for small games, but provides a basis for further discussions.

  • Example - 5-stone Nim

    • played with 2 players and pile of stones.

    • Each player removes one or two stones from the pile.

    • player who removes the last stone wins the game.


Example game tree for 5 stone nim l.jpg

5

4

3

2

2

1

3

2

1

1

0

1

0

0

1

0

0

0

0

OR nodes

AND nodes

x

x

0

Example - Game Tree for 5-Stone Nim

+

+

+

+


Minimax l.jpg
Minimax

  • Minimax theorem - Every two-person zero-sum game is a forced win for one player, or a forced draw for either player, in principle these optimal minimax strategies can be computed.

  • Performing this algorithm on tic-tac-toe results in the root being labeled a draw.


A strategy l.jpg
A strategy

  • A strategy is a method which tells the player how to play in each possible scenario.

  • Can be described implicit or explicit

  • An explicit strategy is a subtree of the search tree which branches only at the opponent moves.

  • The size of the subtree b^d/2


Example strategy for 5 stone nim l.jpg

5

4

3

2

2

1

3

2

1

1

0

1

0

0

1

0

0

0

0

OR nodes

AND nodes

x

x

0

Example - strategy for 5-Stone Nim

+

+

+

+


Minmax propgation l.jpg
MinMax propgation

  • Start from the leaves.

  • At each step, evaluate the value of all descendants:

  • take the maximum if it is A’s turn, or the

    minimum if it isB’s turn

  • The result will be the value of the tree.



Heuristic evaluation functions l.jpg
Heuristic Evaluation Functions

  • Problem: How to evaluate positions, where brute force is out of the question?

  • Solution: Use a heuristic static evaluationfunction to estimate the merit of a position when the final outcome has not yet been determined.


Example of heuristic function l.jpg
Example of heuristic Function

  • Chess :

    • Number of pieces on board of each type multiplied by relative value summed up for each color. By subtracting the weighted material of the black player from the weighted material of the white player we receive the relative strength of the position for each player.


Slide23 l.jpg

Heuristic Evaluation Functions

  • A heuristic static evaluation function for a two player game is a function from a state to a number.

  • The goal of a two player game is to reach a winning state, but the number of moves required to get there is unimportant.

  • Other features must be taken into account to get to an overall evaluation function.


Slide24 l.jpg

Heuristic Evaluation Functions

  • Given a heuristic static evaluation function, it is straightforward to write a program to play a game.

  • From any given position, we simply generate all the legal moves, apply our static evaluator to the position resulting from each move, and then move to the position with the largest or smallest evaluation, depending if we are MIN/MAX


Example tic tac toe behavior of evaluation function l.jpg
Example - tic-tac-toeBehavior of Evaluation Function

  • Detect if game over.

  • If Xis the Maximizer, the function should return  if there are three X’s in a row and -  if there are three O’s in a row.

  • Count of the number of different rows, columns, and diagonals occupied by O.


Example first moves of tic tac toe l.jpg
Example: First moves of tic-tac-toe

X

X

X

3-0 = 3

4-0=4

2-0 = 2


Example tic tac toe behavior of evaluation function27 l.jpg
Example - tic-tac-toeBehavior of Evaluation Function

  • This algorithm is extremely efficient, requiring time that is only linear in the number of legal moves.

  • It’s drawback is that it only considers immediate consequences of each move (doesn’t look over the horizon).


Minimax search l.jpg

1

0

1

0

-1

-1

0

-1

0

-2

Minimax Search

Where does X go?

X

X

X

4-3 = 1

4-2 = 2


Minimax search29 l.jpg
Minimax search

  • Search as deeply as possible given the computational resources of the machine and the time constraints on the game.

  • Evaluate the nodes at the search frontier by the heuristic function.

  • Where MIN is to move, save the minimum of it’s children’s values. Where MAX is to move, save the maximum of it’s children’s values.

  • A move is made to a child of the root with the largest or smallest value, depending on whether MAX or MIN is moving.


Minimax search example minimax tree l.jpg

MAX

4

MIN

4

2

4

8

2

14

4

2

6

8

1

2

12

14

4

5

3

2

6

7

8

9

1

10

2

11

12

13

14

14

Minimax searchexample Minimax Tree


Nash equilibrium l.jpg
Nash equilibrium

Nash equilibrium:

  • Once an agreement has been reached, it is not worthwhile for any of the players to deviate from that agreement given that the other players do not deviate.

  • Example: The market place:

  • agreement: no one sells a hot dog for less than 10$

  • Is it worthwhile for me to reduce the price??

  • No, they will burn my stand.


Example prisoners dilemma l.jpg
Example: prisoners dilemma

Should the prisoner “rat” on his friend?

No equilibrium

equilibrium

quite

rat

quite

rat

0

dead

quite

quite

1

1

3

3

1

1

3

3

3

3

rat

rat

0

dead

3

3


Nash equilibrium33 l.jpg

MAX

4

MIN

4

2

4

8

2

14

4

2

6

8

1

2

12

14

4

5

3

2

6

7

8

9

1

10

2

11

12

13

14

14

Nash equilibrium

  • The principal branch values are in Nash equilibrium


Alpha beta pruning l.jpg
Alpha-Beta Pruning

  • By using alpha-beta pruning the minimax value of the root of a game tree can be determined without having to examine all the nodes.


Alpha beta pruning example l.jpg

MAX

MIN

Alpha-Beta Pruning Example

a

4

b

4

m

2

c

i

n

4

>=6

<=2

d

g

<=3

j

4

6

o

<=1

q

<=2

4

5

3

6

7

1

2

e

f

h

k

l

p

r


Alpha beta l.jpg
Alpha-Beta

  • Deep pruning - Right half of tree in example.

  • Next slide code for alpha-beta pruning :

    • MAXIMIN - assumes that its argument node is a maximizing node.

    • MINIMAX - the same.

    • V(N) - Heuristic static evaluation of node N.


Slide37 l.jpg

MAXIMIN ( node: N ,lowerbound : alpha ,upperbound: beta)

IF N is at the search depth, RETURN V(N)

FOR each child Ni of N

value = MINIMAX(Ni,alpha,beta)

IF value > alpha , alpha := value

IF alpha >= beta ,return alpha

RETURN alpha

MINIMAX ( node: N ,lowerbound : alpha ,upperbound: beta)

IF N is at the search depth, RETURN V(N)

FOR each child Ni of N

value = MAXIMIN(Ni,alpha,beta)

IF value < beta , beta := value

IF beta <= alpha, return alpha

RETURN beta


Performance of alpha beta l.jpg
Performance of Alpha-Beta

  • Efficiency depends on the order in which the nodes are encountered at the search frontier.

  • Optimal - b½ - if the largest child of a MAX node is generated first, and the smallest child of a MIN node is generated first.

  • Worst - b.

  • Average b¾ - random ordering.


Games with chance l.jpg
Games with chance

  • chance nodes: nodes where chance events happen (rolling dice, flipping a coin, etc)

  • Evaluate expected value by averaging outcome probabilities:

    • C is a chance node

    • P(di) probability of rolling di (1,2, …, 12)

    • S(C,di) is the set of positions generated by applyingall legal moves for roll dito C


Games with chance41 l.jpg
Games with chance

Backgammon board


Search tree with probabilities l.jpg

MAX

0.5

0.5

0.5

0.5

MIN

Search tree with probabilities

3 -1

2 4 0 -2

2 4 7 4 6 0 5 -2



Additional enhancements l.jpg
Additional Enhancements

  • A number of additional improvements have been developed to improve performance with limited computation.

  • We briefly discuss the most important of these below.


Node ordering l.jpg
Node Ordering

  • By using node ordering we can get close to b½.

  • Node ordering instead of generating the tree left-to-right, we reorder the tree based on the static evaluations of the interior nodes.

  • To save space only the immediate children are reordered after the parent is fully expanded.


Iterative deepening l.jpg
Iterative Deepening

  • Another idea is to use iterative deepening. In two player games using time, when time runs out, the move recommended by the last completed iteration is made.

  • Can be combined with node ordering to improve pruning efficiency. Instead of using the heuristic value we can use the value from previous iteration.


Quiescence l.jpg
Quiescence

  • Quiescence search is to make a secondary search in the case of a position whose values are unstable.

  • This way obtains a stable evaluation.


Transposition tables l.jpg
Transposition Tables

  • For efficiency, it is important to detect when a state has already been searched.

  • In order to detect a searched state, previously generated game states, with their minimax values are saved into a transposition table.


Opening book l.jpg
Opening Book

  • Most board games start with the same initial state.

  • A table of good initial moves is used, based on human expertise, known as an opening book.


Endgame databases l.jpg
Endgame Databases

  • A database of endgame moves, with minimax values, is used.

  • In checkers, endgame for less than eight or fewer pieces on board.

  • A technique for calculating endgame databases, retrograde analysis.


Special purpose hardware l.jpg
Special Purpose Hardware

  • The faster the machine ,the deeper the search in the time available and the better it plays.

  • The best machines today are based on special-purpose hardware designed and built only to play chess.


Selective search l.jpg
Selective Search

  • The fundamental reason that humans are competitive with computers is that they are very selective in their choice of positions to examine, unlike programs which do full-widthfixed depth searches.

  • Selective search: to search only on a “interesting” domain.

  • Example - Best first minimax.


Best first minimax l.jpg
Best First Minimax

  • Given a partially expanded minimax tree, the backed up minimax value of the root is determined by one of the leaf nodes, as is the value of every node on the path from the root to that leaf.

  • This path is known as principal variation, and the leaf is known as principal leaf.

  • In general, the best-first minimax will generate an unbalanced tree, and make different move decisions than full-width-fixed-depth alpha-beta.


Best first minimax search example l.jpg

6

Principal leaf - expand it

4

6

Best First minimax search- Example


Best first minimax search example55 l.jpg

4

4

2

Principal leaf - expand it

5

2

Best First minimax search- Example


Best first minimax search example56 l.jpg

2

1

2

8

1

5

2

Principal leaf - expand it

Best First minimax search- Example


Best first minimax search example57 l.jpg

5

1

5

8

1

5

7

3

7

Best First minimax search- Example


Best first search l.jpg
Best First search

  • Full width search is a good insurance against missing a move (and making a mistake).

  • Most game programs that use selective searches use a combined algorithm that starts with a full-width search to a nominal length, and then searches more selectively below that depth.


ad