GP End-Chess

1. GP End-Chess Evolution of Chess Endgame Players Ami Hauptman & Moshe Sipper

2. Outline • Introduction • The Game of Chess – a solved problem? • Important differences between human and artificial chess players • Chess Endgames - features & building blocks • GP problem definition • Experiment and Results • Future work

3. The game of Chess • First developed in India and Persia • Considered THE complex game of strategy and inventiveness • Enormous search space • Roughly 50 possible moves at mid-game • A typical game consists of a few dozen moves • Estimated at 1043 in 40-move game (Shannon, 1950) Elephants don’t play Chess(?)

4. The game of Chess – AI history • First chess AI at 1958 – novice level • Machine strength increasing linearly • 1997 – defeat of former world champion, Garry Kasparov, by IBM’s deep blue • Last years – performance still increasing • Mainly Hardware • Also Software • … The End ?

5. The game of Chess • …NO! • Deep(er) blue use extreme brute-force, traversing several millions of boards ps • Very little generalization • Virtually no human resemblance • Deemed theoretically uninteresting • Chomsky: As interesting as a weight lifting competition between machine and man • Low A to I ratio; Low return

6. The game of Chess – Basic concepts • 8x8 board • Each player starts with 16 pieces, of 6 different types, and may only move 1 piece per turn • A piece can only move into an empty square or into one containing an opponent’s piece (a capture) • Win by capturing the opponent’s king

7. The game of Chess – pieces • Pawn: may only move forward (or capture diagonally) • Bishop: diagonals • Knight: L shaped moves. The only “unblockable” piece • Rook: Ranks & files • Queen: Bishop & Rook combined • King: 1 square in any direction. May not move into attacked square Values : 1 3 3/3.5 5 9 ∞

8. The game of Chess – example • White has over 30 possible moves • If black’s turn – can capture pawn at c3 and check (also fork)

9. The game of Chess – Check and Checkmate • “Checking” is attacking opponent’s king. Opponent must respond • “Mating” (Checkmate) is when the opponent can’t avoid losing the king – and thus forfeiting the game

10. Man vs Machine

11. Human & Artificial Players – AI search • AI uses search to assign a score to a board • Traverse the move tree from leaves - up • Select the best child using scores found • Only partial tree Computer is O (Max) opponent is X

12. Human & Artificial Players – The Machine

13. Human & Artificial Players – The Machine • Millions of boards (nodes) per second • Little time for each board – less knowledge • Smart search algorithms – • pruning • Alpha-beta variants (negascout etc.) • Still use heuristics at end – can’t see all tree • Most research revolves around search • Human resemblance minimal – humans use little search

14. Human & Artificial Players – Humans • Humans use problem solving cognition • Deeply knowledge based – • Extensive “theory” exists • Numerous books and institutions • Massive use of pattern recognition • Also use search but • Less deep • Only develop “good” positions • More efficient – less nodes for “same” result • Reminiscent of greedy search • Not only in chess

15. Human & Artificial Players – Grand Masters - Findings • Play against several opponents at the same level they play against a single opponent • GMs and novices: same level of performance when memorizing a random board; differ when memorizing real game positions • GM eye movements show they only scan “correct” parts of board • Strong Amateurs use the same meta-search as GMs - equally deep, same nodes, same speed; Differ in knowledge of domain (De Groot)

16. Endgames - example • White’s turn: mate in 5, with Qe6+ • Features include: • #moves for black king minimal • Attacking, un-attacked • Checking • Officers same line\row • Black’s turn: draw with: Rc1+, then Qg5 – fork & exchange

17. Endgames (2) - features • Few pieces remain (typically: king, 0-3 officers and sometimes pawns) • Fewer options, but more moves for each piece • Trees still extremely large

18. Endgames - Building Blocks • Main goals • Reduce search by “smart” features of the board • Use more game knowledge as humans do • Allow more complex features to be built by supplying basic ones (terminals) and building methods (functions) • Schemata evolution

19. Features Example - Fork • My piece is: • Attacking 2 or more pieces • Protected or not attacked • Opponent pieces: • Unprotected • OR protected but of greater value • Example: black must exchange Q for R because of fork

20. Fork: Traditional AI search • Only 3 legal moves for black • Find that one of white’s next moves (out of 23 possible) captures black queen • Check all following moves for more piece exchanges • Sometimes, still check other moves (non capturing) • At end of search – compare remaining pieces • No fork “concept”

21. Features Example – Fork Feature Search (GP) • One of the features is isMyFork function – Checks all previously defined conditions • Also, use some smaller building blocks: • Is Opponent piece Attacked? • Is attacking piece protected? • Is opponent in check? • Value of attacked piece

22. GP Problem Definition • Terminals & Functions • Numerous “chess terminals” and ERCs • Non-chess funtions • Fitness • Tournament • Run parameters • Termination • We will see each element in the following experiments

23. Endgame experiments conducted • KRKR – each player has 1 king and 1 rook • KRRKR – King with 2 Rook against King and Rook • KRRKRR • KQKQ – Kings and Queens • KQRKQR – Combined

24. Basic program architecture • Generate all possible moves (depth=1) • Evaluate each board with GP individual • Select board with best score (or stochastically decide between equal) • Perform best move • Repeat process with GP opponent until game ends (or until only kings left)

25. KRKR Endgame • Each player has 1 King, 1 Rook • “Toy” problem for chess endgames • Theoretical draw (experts never lose this) • Some interesting positions exist

26. KRKR Endgame - what needs to be learned (1) • Avoid losing rook • Don’t move to attacked, unprotected squares • Vice versa - capture opponent’s rook if able Black to move – white loses Rook

27. KRKR Endgame - what need to be learned (2) • Avoid getting king stuck in edges • Again, take advantage if opponent does this Black to move – mate in 1

28. KRKR Endgames - Terminals • Used in first runs: • Is My Rook Attacked, Is Opp Rook attacked • Is {My, Opp} Rook Protected (two as above) • Is {My, Opp} Rook In Play • Num Moves {My, Opp} king • {My, Opp}-King’s proximity to edges • Is Mate • ERCs: ± {0.25, 0.5, 1} * MAX • MAX = 1000 (empirically)

29. KRKR Endgames - Functions • Boolean • OR2, OR3, OR4 • AND2, AND3, AND4 • NOT • Arithmetic - +, -, * • Combined - <, =, >, IF • STGP • For now, no “chess” functions, only terminals

30. KRKR Endgames - Fitness • Competitive, Random-2-ways • Each individual plays against k randomly selected opponents • Each game counts for both players • For each encounter • Several games (typically 4) are played • Short games - ~5-8 moves per player • Each game starts at a random legal position • Safe start - no piece is attacked at the beginning

31. KRKR Endgames – Fitness(2) • Scoring method: • Victory: 1-2 points • Piece count advantage (theoretical win) – ¾ point • Draw: ½ point • After advantage – 0 points • Loss: 0 points

32. KRKR Endgames – Parameters • Population size - 80 • #Generations - 150..250 • Operators: • Reproduction 0.35 • Crossover 0.5 • Mutation 0.15 (including ERC mutation) • Termination – ~10-25 hours

33. KRKR Endgames – Results • Every 10 generations, best individual played against: • Best of generation 0 • An opponent performing random moves • Longer games: ~10-12 moves per player • 50-150 games • Games were doubled – each player staring from both positions

34. KRKR Endgames – Results • Bad results – no distinct improvement • Several reasons: • Arithmetic operations problematic – we get large numbers • Mate not distinct enough (traditionally terminates the search) • Boolean functions not clear enough • Slow Runs due to large trees with repeating functions

35. KRKR Endgames –Improvements • Boolean functions • Divided to good and bad • Example: Is-My-King-In-Check changed to Is-My-King-Not-In-Check • Mate changed to 1000*Mate • Added Not-My-Rook-Attacked-Unprotected and Opp-Rook-Attacked-Unprotected

36. KRKR Endgames – Results - Improvements • Also consulted Chess Experts – added more: • Is-Opp-King-Behind-Rook • Split to • Opp-King-Prox-Rook • Opp-King-Behind-Rook • Is-Stalemate (only kings left) Black moves and White loses Rook

37. KRKR Endgames – Results - Improvements • Arithmetic functions canceled • Although Still using floats for terminals • Also divide to good and bad: NumNotMovesOppKing • Theoretically justified – more “logical” search in literature • Empirically - need more logical rules, and not : ( > (+ (#moves-k #moves-opp-k) 5.5)) • Memoization – saves more than ½ the time

38. KRKR Endgames – Final Results • Improvement • Above 75% of games against random end in advantage or mate • Still, too few mates, even when score for win is increased – difficult to learn move sequence • Same against best of generation 0 (almost random) • The main thing that was learned was avoiding getting the rook captured

39. KRRKR Endgames • Example (right) • Very good for white • Black king exposed • 2 rooks close • Next move – captures rook • (mate in 5)

40. KRRKR Endgames - goals • One player has 2 rooks, the other – 1 • Not theoretically drawn • We want one generalized individual for all endgames and positions (Not one for each endgame): • Each player needs to play both advantage, draw (KRKR) and disadvantage • Terminals need to be more general

41. KRRKR Endgames - changes • Terminals - changed and added to cope with changing state • Material-Count (recall each rook = 5) • Num-My-Pieces-Not-Attacked, since now there are more than 1 • Is-My-King-Protecting-Piece and My-Officers-Same-Line to allow more complex considerations • Functions • If-Adv-Then-(left child)-Else-(right child) • Eventually divided to 3 trees

42. KRRKR Endgames - changes • Also added – comparing differences to parent node • Boolean Is-Material-Increase, which compares to the parent node (board) • Material decrease is not needed since considering only my move • Not-My-King-Moves-Decreaseto further use number of moves for king

43. KRRKR Endgames – Opponents • Random forsaken; Best-of-0 still used but less • Added new opponent – MASTER • a program we wrote based on consultation with experts, highest being InternationalMaster Boris Gutkin, ELO 2400 (only about 3000 of those…) • Used ~50 general positions and rules derived from them, together with scores for each • Defined a strategy (“Expert”) accordingly • Tested evolved programs against it • Human competitive?

44. KRRKR Endgames – Fitness • Test were conducted by assigning each player both roles for each position • Fitness was refined – score effected by: • Starting position (advantage or disadvantage) • End result – win, loss or draw • Adv position ending in draw receives a score of near zero • Dis-adv ending in a draw will receive better than 0.5

45. KRRKR Endgames – Results • Expert-defined performed extremely well against Random and Best0 • Evolved programs performed generally as well as expert defined, sometimes better Percent of favorable results in game outcomes

46. Main Experiment – KQRKQR • Most complex endgame we worked with • Still theoretical draw • Highly position dependant – “noisy” • Larger trees • 2 officers • Queens • Easier to mate

47. KQRKQR Endgames - changes • Added – more “heavy” terminals (and components) • Boolean Is-Not-Mate-in-one, most time consuming but necessary • Boolean Is-My-King-Not-Trapped • Not all king’s moves lead closer to edges • Important but vague – usually happens with complex terminals • My-Officers-Same-Line

48. Genome Summary

49. KQRKQR Endgames – New Opponent • CRAFTY, second in the 2004 Computer Chess Championship (held at Bar-Ilan) • Uses brute force methods; State-of-the-art search algorithms • Specializes in Blitz games (typically 3 minutes per game) • We limited to 5 secs per move, enough to scan ~1.5 Million boards with pruning

50. KQRKQR Endgames – Our parameters • Used lookahead of depth 2 • Typically ~5 secs per move • Simple Minimax search, but not Alpha-Beta • Played 5-6 moves per game • Never cancelled a game, even if it started with mate-in-4 (which CRAFTY easily saw) • Played each position 2 times, switching places • ~100 games - reduce noises in starting positions