GP End-Chess - PowerPoint PPT Presentation

Gp end chess l.jpg
Download
1 / 56

GP End-Chess Evolution of Chess Endgame Players Ami Hauptman & Moshe Sipper Outline Introduction The Game of Chess – a solved problem? Important differences between human and artificial chess players Chess Endgames - features & building blocks GP problem definition Experiment and Results

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

GP End-Chess

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Gp end chess l.jpg

GP End-Chess

Evolution of Chess Endgame Players

Ami Hauptman & Moshe Sipper


Outline l.jpg

Outline

  • Introduction

    • The Game of Chess – a solved problem?

    • Important differences between human and artificial chess players

  • Chess Endgames - features & building blocks

  • GP problem definition

  • Experiment and Results

  • Future work


The game of chess l.jpg

The game of Chess

  • First developed in India and Persia

  • Considered THE complex game of strategy and inventiveness

  • Enormous search space

    • Roughly 50 possible moves at mid-game

    • A typical game consists of a few dozen moves

    • Estimated at 1043 in 40-move game (Shannon, 1950)

Elephants don’t play Chess(?)


The game of chess ai history l.jpg

The game of Chess – AI history

  • First chess AI at 1958 – novice level

  • Machine strength increasing linearly

  • 1997 – defeat of former world champion, Garry Kasparov, by IBM’s deep blue

  • Last years – performance still increasing

    • Mainly Hardware

    • Also Software

  • … The End ?


The game of chess5 l.jpg

The game of Chess

  • …NO!

  • Deep(er) blue use extreme brute-force, traversing several millions of boards ps

  • Very little generalization

  • Virtually no human resemblance

  • Deemed theoretically uninteresting

    • Chomsky: As interesting as a weight lifting competition between machine and man

  • Low A to I ratio; Low return


The game of chess basic concepts l.jpg

The game of Chess – Basic concepts

  • 8x8 board

  • Each player starts with 16 pieces, of 6 different types, and may only move 1 piece per turn

  • A piece can only move into an empty square or into one containing an opponent’s piece (a capture)

  • Win by capturing the opponent’s king


The game of chess pieces l.jpg

The game of Chess – pieces

  • Pawn: may only move forward (or capture diagonally)

  • Bishop: diagonals

  • Knight: L shaped moves. The only “unblockable” piece

  • Rook: Ranks & files

  • Queen: Bishop & Rook combined

  • King: 1 square in any direction. May not move into attacked square

Values :

1 3 3/3.5 5 9 ∞


The game of chess example l.jpg

The game of Chess – example

  • White has over 30 possible moves

  • If black’s turn – can capture pawn at c3 and check (also fork)


The game of chess check and checkmate l.jpg

The game of Chess – Check and Checkmate

  • “Checking” is attacking opponent’s king. Opponent must respond

  • “Mating” (Checkmate)

    is when the opponent can’t avoid losing the king – and thus forfeiting the game


Man vs machine l.jpg

Man vs Machine


Human artificial players ai search l.jpg

Human & Artificial Players – AI search

  • AI uses search to assign a score to a board

  • Traverse the move tree from leaves - up

  • Select the best child using scores found

  • Only partial tree

Computer is O (Max) opponent is X


Human artificial players the machine l.jpg

Human & Artificial Players – The Machine


Human artificial players the machine13 l.jpg

Human & Artificial Players – The Machine

  • Millions of boards (nodes) per second

  • Little time for each board – less knowledge

  • Smart search algorithms –

    • pruning

    • Alpha-beta variants (negascout etc.)

  • Still use heuristics at end – can’t see all tree

  • Most research revolves around search

  • Human resemblance minimal – humans use little search


Human artificial players humans l.jpg

Human & Artificial Players – Humans

  • Humans use problem solving cognition

  • Deeply knowledge based –

    • Extensive “theory” exists

    • Numerous books and institutions

  • Massive use of pattern recognition

  • Also use search but

    • Less deep

    • Only develop “good” positions

  • More efficient – less nodes for “same” result

    • Reminiscent of greedy search

  • Not only in chess


Human artificial players grand masters findings l.jpg

Human & Artificial Players – Grand Masters - Findings

  • Play against several opponents at the same level they play against a single opponent

  • GMs and novices: same level of performance when memorizing a random board; differ when memorizing real game positions

  • GM eye movements show they only scan “correct” parts of board

  • Strong Amateurs use the same meta-search as GMs - equally deep, same nodes, same speed; Differ in knowledge of domain (De Groot)


Endgames example l.jpg

Endgames - example

  • White’s turn: mate in 5, with Qe6+

  • Features include:

    • #moves for black king minimal

    • Attacking, un-attacked

    • Checking

    • Officers same line\row

  • Black’s turn: draw with: Rc1+, then Qg5 – fork & exchange


Endgames 2 features l.jpg

Endgames (2) - features

  • Few pieces remain (typically: king, 0-3 officers and sometimes pawns)

  • Fewer options, but more moves for each piece

  • Trees still extremely large


Endgames building blocks l.jpg

Endgames - Building Blocks

  • Main goals

    • Reduce search by “smart” features of the board

    • Use more game knowledge as humans do

  • Allow more complex features to be built by supplying basic ones (terminals) and building methods (functions)

  • Schemata evolution


Features example fork l.jpg

Features Example - Fork

  • My piece is:

    • Attacking 2 or more pieces

    • Protected or not attacked

  • Opponent pieces:

    • Unprotected

    • OR protected but of greater value

  • Example: black must exchange Q for R because of fork


Fork traditional ai search l.jpg

Fork: Traditional AI search

  • Only 3 legal moves for black

  • Find that one of white’s next moves (out of 23 possible) captures black queen

  • Check all following moves for more piece exchanges

  • Sometimes, still check other moves (non capturing)

  • At end of search – compare remaining pieces

  • No fork “concept”


Features example fork feature search gp l.jpg

Features Example – Fork Feature Search (GP)

  • One of the features is isMyFork function – Checks all previously defined conditions

  • Also, use some smaller building blocks:

    • Is Opponent piece Attacked?

    • Is attacking piece protected?

    • Is opponent in check?

    • Value of attacked piece


Gp problem definition l.jpg

GP Problem Definition

  • Terminals & Functions

    • Numerous “chess terminals” and ERCs

    • Non-chess funtions

  • Fitness

    • Tournament

  • Run parameters

  • Termination

  • We will see each element in the following experiments


Endgame experiments conducted l.jpg

Endgame experiments conducted

  • KRKR – each player has 1 king and 1 rook

  • KRRKR – King with 2 Rook against King and Rook

  • KRRKRR

  • KQKQ – Kings and Queens

  • KQRKQR – Combined


Basic program architecture l.jpg

Basic program architecture

  • Generate all possible moves (depth=1)

  • Evaluate each board with GP individual

  • Select board with best score (or stochastically decide between equal)

  • Perform best move

  • Repeat process with GP opponent until game ends (or until only kings left)


Krkr endgame l.jpg

KRKR Endgame

  • Each player has 1 King, 1 Rook

  • “Toy” problem for chess endgames

  • Theoretical draw (experts never lose this)

  • Some interesting positions exist


Krkr endgame what needs to be learned 1 l.jpg

KRKR Endgame - what needs to be learned (1)

  • Avoid losing rook

    • Don’t move to attacked, unprotected squares

  • Vice versa - capture opponent’s rook if able

Black to move – white loses Rook


Krkr endgame what need to be learned 2 l.jpg

KRKR Endgame - what need to be learned (2)

  • Avoid getting king stuck in edges

  • Again, take advantage if opponent does this

Black to move – mate in 1


Krkr endgames terminals l.jpg

KRKR Endgames - Terminals

  • Used in first runs:

  • Is My Rook Attacked, Is Opp Rook attacked

  • Is {My, Opp} Rook Protected (two as above)

  • Is {My, Opp} Rook In Play

  • Num Moves {My, Opp} king

  • {My, Opp}-King’s proximity to edges

  • Is Mate

  • ERCs: ± {0.25, 0.5, 1} * MAX

    • MAX = 1000 (empirically)


Krkr endgames functions l.jpg

KRKR Endgames - Functions

  • Boolean

    • OR2, OR3, OR4

    • AND2, AND3, AND4

    • NOT

  • Arithmetic - +, -, *

  • Combined - <, =, >, IF

  • STGP

  • For now, no “chess” functions, only terminals


Krkr endgames fitness l.jpg

KRKR Endgames - Fitness

  • Competitive, Random-2-ways

    • Each individual plays against k randomly selected opponents

    • Each game counts for both players

  • For each encounter

    • Several games (typically 4) are played

    • Short games - ~5-8 moves per player

    • Each game starts at a random legal position

    • Safe start - no piece is attacked at the beginning


Krkr endgames fitness 2 l.jpg

KRKR Endgames – Fitness(2)

  • Scoring method:

    • Victory: 1-2 points

    • Piece count advantage (theoretical win) – ¾ point

    • Draw: ½ point

      • After advantage – 0 points

    • Loss: 0 points


Krkr endgames parameters l.jpg

KRKR Endgames – Parameters

  • Population size - 80

  • #Generations - 150..250

  • Operators:

    • Reproduction 0.35

    • Crossover 0.5

    • Mutation 0.15 (including ERC mutation)

  • Termination – ~10-25 hours


Krkr endgames results l.jpg

KRKR Endgames – Results

  • Every 10 generations, best individual played against:

    • Best of generation 0

    • An opponent performing random moves

    • Longer games: ~10-12 moves per player

  • 50-150 games

  • Games were doubled – each player staring from both positions


Krkr endgames results34 l.jpg

KRKR Endgames – Results

  • Bad results – no distinct improvement

  • Several reasons:

    • Arithmetic operations problematic – we get large numbers

    • Mate not distinct enough (traditionally terminates the search)

    • Boolean functions not clear enough

    • Slow Runs due to large trees with repeating functions


Krkr endgames improvements l.jpg

KRKR Endgames –Improvements

  • Boolean functions

    • Divided to good and bad

    • Example: Is-My-King-In-Check changed to Is-My-King-Not-In-Check

    • Mate changed to 1000*Mate

    • Added Not-My-Rook-Attacked-Unprotected and Opp-Rook-Attacked-Unprotected


Krkr endgames results improvements l.jpg

KRKR Endgames – Results - Improvements

  • Also consulted Chess Experts – added more:

  • Is-Opp-King-Behind-Rook

  • Split to

    • Opp-King-Prox-Rook

    • Opp-King-Behind-Rook

  • Is-Stalemate (only kings left)

Black moves and White loses Rook


Krkr endgames results improvements37 l.jpg

KRKR Endgames – Results - Improvements

  • Arithmetic functions canceled

    • Although Still using floats for terminals

    • Also divide to good and bad: NumNotMovesOppKing

    • Theoretically justified – more “logical” search in literature

    • Empirically - need more logical rules, and not : ( > (+ (#moves-k #moves-opp-k) 5.5))

  • Memoization – saves more than ½ the time


Krkr endgames final results l.jpg

KRKR Endgames – Final Results

  • Improvement

    • Above 75% of games against random end in advantage or mate

    • Still, too few mates, even when score for win is increased – difficult to learn move sequence

    • Same against best of generation 0 (almost random)

    • The main thing that was learned was avoiding getting the rook captured


Krrkr endgames l.jpg

KRRKR Endgames

  • Example (right)

  • Very good for white

    • Black king exposed

    • 2 rooks close

    • Next move – captures rook

    • (mate in 5)


Krrkr endgames goals l.jpg

KRRKR Endgames - goals

  • One player has 2 rooks, the other – 1

  • Not theoretically drawn

  • We want one generalized individual for all endgames and positions (Not one for each endgame):

    • Each player needs to play both advantage, draw (KRKR) and disadvantage

    • Terminals need to be more general


Krrkr endgames changes l.jpg

KRRKR Endgames - changes

  • Terminals - changed and added to cope with changing state

    • Material-Count (recall each rook = 5)

    • Num-My-Pieces-Not-Attacked, since now there are more than 1

    • Is-My-King-Protecting-Piece and My-Officers-Same-Line to allow more complex considerations

  • Functions

    • If-Adv-Then-(left child)-Else-(right child)

    • Eventually divided to 3 trees


Krrkr endgames changes42 l.jpg

KRRKR Endgames - changes

  • Also added – comparing differences to parent node

    • Boolean Is-Material-Increase, which compares to the parent node (board)

    • Material decrease is not needed since considering only my move

    • Not-My-King-Moves-Decreaseto further use number of moves for king


Krrkr endgames opponents l.jpg

KRRKR Endgames – Opponents

  • Random forsaken; Best-of-0 still used but less

  • Added new opponent – MASTER

    • a program we wrote based on consultation with experts, highest being InternationalMaster Boris Gutkin, ELO 2400 (only about 3000 of those…)

    • Used ~50 general positions and rules derived from them, together with scores for each

    • Defined a strategy (“Expert”) accordingly

  • Tested evolved programs against it

    • Human competitive?


Krrkr endgames fitness l.jpg

KRRKR Endgames – Fitness

  • Test were conducted by assigning each player both roles for each position

  • Fitness was refined – score effected by:

    • Starting position (advantage or disadvantage)

    • End result – win, loss or draw

    • Adv position ending in draw receives a score of near zero

    • Dis-adv ending in a draw will receive better than 0.5


Krrkr endgames results l.jpg

KRRKR Endgames – Results

  • Expert-defined performed extremely well against Random and Best0

  • Evolved programs performed generally as well as expert defined, sometimes better

Percent of favorable results in game outcomes


Main experiment kqrkqr l.jpg

Main Experiment – KQRKQR

  • Most complex endgame we worked with

  • Still theoretical draw

  • Highly position dependant – “noisy”

  • Larger trees

    • 2 officers

    • Queens

  • Easier to mate


Kqrkqr endgames changes l.jpg

KQRKQR Endgames - changes

  • Added – more “heavy” terminals (and components)

    • Boolean Is-Not-Mate-in-one, most time consuming but necessary

    • Boolean Is-My-King-Not-Trapped

      • Not all king’s moves lead closer to edges

      • Important but vague – usually happens with complex terminals

    • My-Officers-Same-Line


Genome summary l.jpg

Genome Summary


Kqrkqr endgames new opponent l.jpg

KQRKQR Endgames – New Opponent

  • CRAFTY, second in the 2004 Computer Chess Championship (held at Bar-Ilan)

  • Uses brute force methods; State-of-the-art search algorithms

  • Specializes in Blitz games (typically 3 minutes per game)

  • We limited to 5 secs per move, enough to scan ~1.5 Million boards with pruning


Kqrkqr endgames our parameters l.jpg

KQRKQR Endgames – Our parameters

  • Used lookahead of depth 2

    • Typically ~5 secs per move

    • Simple Minimax search, but not Alpha-Beta

  • Played 5-6 moves per game

  • Never cancelled a game, even if it started with mate-in-4 (which CRAFTY easily saw)

    • Played each position 2 times, switching places

    • ~100 games - reduce noises in starting positions


Results master l.jpg

Results: Master


Results crafty l.jpg

Results: CRAFTY


Multiple endgames l.jpg

Multiple Endgames

  • Aim for general-purpose strategies

  • All endgames used during evolution

  • Results:


Sample gp endchess l.jpg

Sample GP-Endchess

Tree 0:

(If3 (Or2 (Not (Or2 (And2 OppPieceAttUnprotected NotMyKingInCheck) (Or2 NotMyPieceAttUnprotected 100*Increase))) (And2 (Or3 (And2 OppKingStuck NotMyPieceAttUnprotected) (And2 OppPieceAttUnprotected OppKingStuck) (And3 -1000*MateInOne OppKingInCheckPieceBehind NotMyKingStuck)) (Or2 (Not NotMyKingStuck) OppKingInCheck))) NumMyPiecesUNATT (If3 (< (If3 (Or2 NotMyPieceAttUnprotected NotMyKingInCheck) (If3 NotMyPieceAttUnprotected #NotMovesOppKing OppKingInCheckPieceBehind) (If3 OppKingStuck OppKingInCheckPieceBehind -1000*MateInOne)) (If3 (And2 100*Increase 1000*Mate?) (If3 (< NumMyPiecesUNATT (If3 NotMyPieceAttUnprotected -1000*MateInOne OppKingProxEdges)) (If3 (< MyKingDistEdges #NotMovesOppKing) (If3 -1000*MateInOne -1000*MateInOne NotMyPieceATT) (If3 100*Increase #MovesMyKing OppKingInCheckPieceBehind)) NumOppPiecesATT) (If3 NotMyKingStuck -100.0 OppKingProxEdges))) (If3 OppKingInCheck (If3 (Or2 NotMyPieceAttUnprotected NotMyKingInCheck) (If3 (< MyKingDistEdges #NotMovesOppKing) (If3 -1000*MateInOne -1000*MateInOne NotMyPieceATT) (If3 100*Increase #MovesMyKing OppKingInCheckPieceBehind)) NumOppPiecesATT) (If3 (And3 -1000*MateInOne NotMyPieceAttUnprotected 100*Increase) (If3 (< NumMyPiecesUNATT (If3 NotMyPieceAttUnprotected -1000*MateInOne OppKingProxEdges)) (If3 (< MyKingDistEdges #NotMovesOppKing) (If3 -1000*MateInOne -1000*MateInOne NotMyPieceATT) (If3 100*Increase #MovesMyKing OppKingInCheckPieceBehind)) NumOppPiecesATT) -1000*MateInOne)) (If3 (< (If3 100*Increase MyKingDistEdges 100*Increase) (If3 OppKingStuck OppKingInCheckPieceBehind -1000*MateInOne)) -100.0 (If3 (And2 NotMyPieceAttUnprotected -1000*MateInOne) (If3 (< NumMyPiecesUNATT (If3 NotMyPieceAttUnprotected -1000*MateInOne OppKingProxEdges)) (If3 (< MyKingDistEdges #NotMovesOppKing) (If3 -1000*MateInOne -1000*MateInOne NotMyPieceATT) (If3 100*Increase #MovesMyKing OppKingInCheckPieceBehind)) NumOppPiecesATT) (If3 OppPieceAttUnprotected NumMyPiecesUNATT MyFork)))))

Tree 1:

(If3 NotMyPieceAttUnprotected #NotMovesOppKing 1000*Mate?)

Tree 2:

(If3 1000*Mate? NumMyPiecesUNATT -1000*MateInOne)


Summary l.jpg

Summary

  • Draw and better against Master-defined

  • Draw against a world class opponent

    • On limited conditions (not a full game, time ,etc.)

  • Shows deep search may have an alternative

  • Fast, pattern-oriented search suggests more human resemblance

  • Search and lookahead are still important


Future work l.jpg

Future Work

  • Add more pieces

  • Improve evolution speed

    • Parallel nets

    • Stronger board representations

  • Develop more cognitive models using evolution

  • Search scheme space as well as game space

  • Tackle beyond endgames

    • Openings and mid-game

  • General game concept schemas (?)


  • Login