550 likes | 768 Views
GP End-Chess. Evolution of Chess Endgame Players Ami Hauptman & Moshe Sipper. Outline. Introduction The Game of Chess – a solved problem? Important differences between human and artificial chess players Chess Endgames - features & building blocks GP problem definition
E N D
GP End-Chess Evolution of Chess Endgame Players Ami Hauptman & Moshe Sipper
Outline • Introduction • The Game of Chess – a solved problem? • Important differences between human and artificial chess players • Chess Endgames - features & building blocks • GP problem definition • Experiment and Results • Future work
The game of Chess • First developed in India and Persia • Considered an extremely complex game of strategy and inventiveness • Enormous search space • Roughly 50 possible moves at mid-game • A typical game consists of a few dozen moves • Estimated at 1043 in 40-move game (Shannon, 1950)
The game of Chess – AI history • First chess AI at 1958 – novice level • Machine strength increasing linearly • 1997 – defeat of former world champion, Garry Kasparov, by IBM’s deep blue • Last years – performance still increasing • Mainly Hardware • Also Software • … The End ?
The game of Chess • …NO! • Deep(er) blue uses extreme brute-force, traversing several millions of boards ps • Very little generalization • Virtually no human resemblance • Deemed theoretically uninteresting • Chomsky: As interesting as a weight lifting competition between machine and man • Low A to I ratio; Low return
The game of Chess – Basic concepts • 8x8 board • Each player starts with 16 pieces, of 6 different types, and may only move 1 piece per turn • A piece can only move into an empty square or into one containing an opponent’s piece (a capture) • Win by capturing the opponent’s king
The game of Chess – pieces • Pawn: may only move forward (or capture diagonally) • Bishop: diagonals • Knight: L shaped moves. The only “unblockable” piece • Rook: Ranks & files • Queen: Bishop & Rook combined • King: 1 square in any direction. May not move into attacked square Values : 1 3 3/3.5 5 9 ∞
The game of Chess – example • White has over 30 possible moves • If black’s turn – can capture pawn at c3 and check (also fork)
The game of Chess – Check and Checkmate • “Checking” is attacking opponent’s king. Opponent must respond • “Mating” (Checkmate) is when the opponent can’t avoid losing the king – and thus forfeiting the game
Human & Artificial Players - AI • AI uses search to assign a score to a board • Traverse the move tree from leaves - up • Select the best child using scores found • Only partial tree Computer is O (Max) opponent is X
Human & Artificial Players – AI Search • Millions of boards (nodes) per second • Little time for each board – less knowledge • Smart search algorithms – • pruning • Alpha-beta variants (negascout etc.) • Still use heuristics at end – can’t see all tree • Most research revolves around search • Human resemblance minimal – humans use little search
Human & Artificial Players – Humans(1) • Humans use problem solving cognition • Deeply knowledge based – extensive “Chess theory” • Massive use of pattern recognition • Also use search but • Less deep • Only develop “good” positions • More efficient – less nodes for “same” result • Reminiscent of greedy search • Not only in chess
Human & Artificial Players – Grand Masters - Findings • Play against several opponents at the same level they play against a single opponent • GMs and novices: same level of performance when memorizing a random board; differ when memorizing real game positions • GM eye movements show they only scan “correct” parts of board • Strong Amateurs use the same meta-search as GMs - equally deep, same nodes, same speed; Differ in knowledge of domain (De Groot)
Endgames - example • White’s turn: mate in 5, with Qe6+ • Features include: • #moves for black king minimal • Attacking, un-attacked • Checking • Officers same line\row • Black’s turn: draw with: Rc1+, then Qg5 – fork & exchange
Endgames (2) - features • Few pieces remain (typically: king, 0-3 officers and sometimes pawns) • Fewer options, but more moves for each piece • Trees still extremely large
Endgames - Building Blocks • Main goals • Reduce search by “smart” features of the board • Use more game knowledge as humans do • Allow more complex features to be built by supplying basic ones (terminals) and building methods (functions) • Schemata evolution
Features Example - Fork • My piece is: • Attacking 2 or more pieces • Protected or not attacked • Opponent pieces: • Unprotected • OR protected but of greater value • Example: black must exchange Q for R because of fork
Fork: Traditional AI search • Only 3 legal moves for black • Find that one of white’s next moves (out of 23 possible) captures black queen • Check all following moves for more piece exchanges • Sometimes, still check other moves (non capturing) • At end of search – compare remaining pieces • No fork “concept”
Features Example – Fork Feature Search (GP) • One of the features is isMyFork function – Checks all previously defined conditions • Also, use some smaller building blocks: • Is Opponent piece Attacked? • Is attacking piece protected? • Is opponent in check? • Value of attacked piece
GP Problem Definition • Terminals & Functions • Fitness • Run parameters • Termination • We will see each element in the following experiments
Endgame experiments conducted • KRKR – each player has 1 king and 1 rook • KRRKR – King with 2 Rook against King and Rook • KRRKRR • KQKQ – Kings and Queens • KQRKQR – Combined
Basic program architecture • Generate all possible moves (depth=1) • Evaluate each board with GP individual • Select board with best score (or stochastically decide between equal) • Perform best move • Repeat process with GP opponent until game ends (or until only kings left)
KRKR Endgame • Each player has 1 King, 1 Rook • “Toy” problem for chess endgames • Theoretical draw (experts never lose this) • Some interesting positions exist
KRKR Endgame - what needs to be learned (1) • Avoid losing rook • Don’t move to attacked, unprotected squares • Vice versa - capture opponent’s rook if able Black to move – white loses Rook
KRKR Endgame - what need to be learned (2) • Avoid getting king stuck in edges • Again, take advantage if opponent does this Black to move – mate in 1
KRKR Endgames - Terminals • Used in first runs: • Is My Rook Attacked, Is Opp Rook attacked • Is {My, Opp} Rook Protected (two as above) • Is {My, Opp} Rook In Play • Num Moves {My, Opp} king • {My, Opp}-King’s proximity to edges • Is Mate • ERCs: ± {0.25, 0.5, 1} * MAX • MAX = 1000 (empirically)
KRKR Endgames - Functions • Boolean • OR2, OR3, OR4 • AND2, AND3, AND4 • NOT • Arithmetic - +, -, * • Combined - <, =, >, IF • STGP • For now, no “chess” functions, only terminals
KRKR Endgames - Fitness • Competitive, Random-2-ways • Each individual plays against k randomly selected opponents • Each game counts for both players • For each encounter • Several games (typically 4) are played • Short games - ~5-8 moves per player • Each game starts at a random legal position • Safe start - no piece is attacked at the beginning
KRKR Endgames – Fitness(2) • Scoring method: • Victory: 1-2 points • Piece count advantage (theoretical win) – ¾ point • Draw: ½ point • After advantage – 0 points • Loss: 0 points
KRKR Endgames – Parameters • Population size - 80 • #Generations - 150..250 • Operators: • Reproduction 0.35 • Crossover 0.5 • Mutation 0.15 (including ERC mutation) • Termination – ~10-20 hours
KRKR Endgames – Results • Every 10 generations, best individual played against: • Best of generation 0 • An opponent performing random moves • Longer games: ~10-12 moves per player • 50-150 games • Games were doubled – each player staring from both positions
KRKR Endgames – Results • Bad results – no distinct improvement • Several reasons: • Arithmetic operations problematic – we get large numbers • Mate not distinct enough (traditionally terminates the search) • Boolean functions not clear enough • Slow Runs due to large trees with repeating functions
KRKR Endgames –Improvements • Boolean functions • Divided to good and bad • Example: Is-My-King-In-Check changed to Is-My-King-Not-In-Check • Mate changed to 1000*Mate • Added Not-My-Rook-Attacked-Unprotected and Opp-Rook-Attacked-Unprotected
KRKR Endgames – Results - Improvements • Also consulted Chess Experts – added more: • Is-Opp-King-Behind-Rook • Split to • Opp-King-Prox-Rook • Opp-King-Behind-Rook • Is-Stalemate (only kings left) Black moves and White loses Rook
KRKR Endgames – Results - Improvements • Arithmetic functions canceled • Although Still using floats for terminals • Also divide to good and bad: NumNotMovesOppKing • Theoretically justified – more “logical” search in literature • Empirically - need more logical rules, and not : ( > (+ (#moves-k #moves-opp-k) 5.5)) • Memoization – saves more than ½ the time
KRKR Endgames – Final Results • Improvement • Above 75% of games against random end in advantage or mate • Still, too few mates, even when score for win is increased – difficult to learn move sequence • Same against best of generation 0 (almost random) • The main thing that was learned was avoiding getting the rook captured
KRRKR Endgames • Example (right) • Very good for white • Black king exposed • 2 rooks close • Next move – captures rook • (mate in 5)
KRRKR Endgames - goals • One player has 2 rooks, the other – 1 • Not theoretically drawn • We want one generalized individual for all endgames and positions (Not one for each endgame): • Each player needs to play both advantage, draw (KRKR) and disadvantage • Terminals need to be more general
KRRKR Endgames - changes • Terminals - changed and added to cope with changing state • Material-Count (recall each rook = 5) • Num-My-Pieces-Not-Attacked, since now there are more than 1 • Is-My-King-Protecting-Piece and My-Officers-Same-Line to allow more complex considerations • Functions • If-Adv-Then-(left child)-Else-(right child) • Eventually divided to 3 trees
KRRKR Endgames - changes • Also added – comparing differences to parent node • Boolean Is-Material-Increase, which compares to the parent node (board) • Material decrease is not needed since considering only my move • Not-My-King-Moves-Decreaseto further use number of moves for king
KRRKR Endgames – Opponents • Random forsaken; Best-of-0 still used but less • Added new opponent – MASTER • a program we wrote based on consultation with experts, highest being InternationalMaster Boris Gutkin, ELO 2400 (only about 3000 of those…) • Used ~50 general positions and rules derived from them, together with scores for each • Defined a strategy (“Expert”) accordingly • Tested evolved programs against it • Human competitive?
KRRKR Endgames – Fitness • Test were conducted by assigning each player both roles for each position • Fitness was refined – score effected by: • Starting position (advantage or disadvantage) • End result – win, loss or draw • Adv position ending in draw receives a score of near zero • Dis-adv ending in a draw will receive better than 0.5
KRRKR Endgames – Results • Expert-defined performed extremely well against Random and Best0 • Evolved programs performed generally as well as expert defined, sometimes better Percent of favorable results in game outcomes
Main Experiment – KQRKQR • Most complex endgame we worked with • Still theoretical draw • Highly position dependant – “noisy” • Larger trees • 2 officers • Queens • Easier to mate
KQRKQR Endgames - changes • Added – more “heavy” terminals (and components) • Boolean Is-Not-Mate-in-one, most time consuming but necessary • Boolean Is-My-King-Not-Trapped • Not all king’s moves lead closer to edges • Important but vague – usually happens with complex terminals • My-Officers-Same-Line
KQRKQR Endgames – New Opponent • CRAFTY, second in the 2004 Computer Chess Championship (held at Bar-Ilan) • Uses brute force methods; State-of-the-art search algorithms • Specializes in Blitz games (typically 3 minutes per game) • We limited to 5 secs per move, enough to scan ~1.5 Million boards with pruning
KQRKQR Endgames – Our parameters • Used lookahead of 2 • Typically ~5 secs per move • Simple Minimax search, but not Alpha-Beta • Played 5-6 moves per game • Never cancelled a game, even if it started with mate-in-4 (which CRAFTY easily saw) • Played each position 2 times, switching places • ~100 games - reduce noises in starting positions