Understanding Search Algorithms in Game Playing: Techniques and Strategies

1. Search CSC 358/458 5.22.2006

2. Outline • Homework #6 • Game search • States and operators • Issues • Search techniques • DFS, BFS • Beam search • A* search • Alpha-beta search

3. Homework #7 • #C • with-list-iterator • doiter

4. Game Playing • How can we automate game playing? • One of the first problems tackled by AI research • Basic idea • represent the "state" of the game • set of cards • board position • moves are changes in game state • winning means reaching a particular state • defined by the rules

5. Game tree • Think of • each possible position as a node • each possible move as an edge • We have a graph structure • starting state • subsequent states • branching for different possible moves • terminates with winning (or losing)

6. How to win? • Find a path through the tree to a winning state • make all the moves along that path • But • what about the opponent? • what about uncertainty? • we'll return to these questions in a minute

7. Graph search • Game tree search is a special case of graph search • lots of other AI problems have been conceptualized the same way • Search domains • Running Prolog programs • each state is an assignment of bindings • links are applying rules to generate new bindings • Planning and scheduling • nodes are states of the world • links are operations that can be performed • Major subfield of AI

8. Planning • States are combinations of predicates • Operators may have conditional effects • Interleaving of planning and execution • replanning

9. What we need • Start State • Goal State • Successors • Search Strategy

10. Tree Search 1 2 3 4 5 6 7

11. Tree Search, cont'd • Main question • How to order the states?

12. Tree Search Cont’d (defun tree-search (states goal-p successors combiner) (cond ((null states) fail) ((funcall goal-p (first states)) (first states)) (t (tree-search (funcall combiner (funcall successors (first states)) (rest states)) goal-p successors combiner))))

13. Tree Search: Depth First Search • Work On The Longest Paths First • Backtrack Only When The Current State Has No More Successors (defun depth-first-search (start goal-p successors) (tree-search (list start) goal-p successors #’append))

14. Tree Search: DFS Summary • Depth-First Search Is OK In Finite Search Spaces • In Infinite Search Spaces, Depth-First Search May Never Terminate

15. Tree Search: Breadth-First Search • Search The Tree Layer By Layer (defun prepend (x y) (append y x)) (defun breadth-first-search (start goal-p successors) (tree-search (list start) goal-p successors #’prepend))

16. Tree Search: BFS Summary • In Finite Search Spaces, BFS Is Identical To DFS • In Infinite Search Spaces, BFS Will Always Find A Solution If It Exists • BFS Requires More Space Than DFS

17. Iterative Deepening • Search depth first to level n • then increase n • Seems wasteful • but actually is the best method for large spaces of unknown charcteristics • the search frontier expands exponentially • so it doesn't matter that you're sometimes searching the same (small number of) nodes multiple times

18. Bi-Directional Search • Work forwards from start • Work backwards from goal • Until the two points meet • Doesn't work for many game problems • How many different checkmate positions are there?

19. Controlling Search • Knowledge • DFS and BFS do not use knowledge of the domain • Distance heuristic • in many domains, possible to estimate how far from the goal • "stronger" board position • choose successor (move) that takes you closest

20. Example Problem: Visit too many nodes, some clearly out of the question

21. Best First Search (defun sorter (cost-fn) #’(lambda (new old) (sort (append new old) #’< :key cost-fn))) (defun best-first-search (start goal-p successors cost-fn) (tree-search (list start) goal-p successors (sorter cost-fn)))

22. Greedy Search • Best = closest to goal • Problem • Isn't guaranteed to find a solution • not complete • Isn't guaranteed to find the best solution • not optimal

23. Greedy example Heuristic: minimize h(n) = “Euclidean distance to destination” Problem: not optimal (through Rimmici Viicea and Pitesti is shorter)

24. A* Search • Best = min (path so far + estimated cost to goal) • Restriction • estimate must never overestimate the cost • If so • complete • optimal

25. Example A*: minimize f(n) = g(n) + h(n)

26. Beam Search • Ever-increasing queue of states under consideration • Can be very large • O(bn) where b is the branch factor and n is the depth • Completeness is required if there is only one solution • we don't want to throw out the state that leads to it • What if there are many good solutions • many possible checkmate positions • discard some unpromising states • Beam search • keep no more than k states of the queue • if too many, discard the ones with highest f(n)

27. Beam Search Cont’d (defun beam-search (start goal-p sucessors cost-fn beam-width) (tree-search (list start) goal-p succecssors #’(lambda (old new) (let ((sorted (funcall (sorter cost-fn) old new))) (if (> beam-width (length sorted)) sorted (subseq sorted 0 beam-width))))))

28. Improving Beam Search • What if the search fails? • try different beam widths (defun iter-wide-search (start goal-p successors &key (width 1) (max 100)) (unless (> width max) (or (beam-search start goal-p successors cost-fn width) (iter-wide-search start goal-p successors cost-fn :width (+ width 1) :max max))))

29. (Practically) Infinite Search • What if the goal state is so far away that search won't find it? • chess = 1043 states • greater than the number of atoms in the universe • Pick a search depth • estimate the "value" of the position at that depth • treat that as the "result" of the search • Search then becomes finding the best board position after k moves • easy enough to store the best node so far • and the path (move) to it

30. What about the opponent? • Obviously, our opponent will not pick moves on the path to our winning game • What move to predict? • Worst case scenario • the opponent will do what's best for him • To win • we need a strategy that will succeed even if the opponent plays his best

31. Mini-max assumption • Assume that the opponent values the game state the opposite from you • Vme(state) = -Vopp(state) • At alternate nodes • choose the state with maximum f • for me • or, choose the state with minimum f • for the opponent

32. Mini-max algorithm • Build tree with two types of nodes • max nodes • my move • min nodes • opp move • Perform depth-first search, with iterative deepening • Evaluate the board position at each node • on a max node, use the max of all children as the value of the parent • on a min node, use the min of all children as the value of the parent • when search is complete • the move that leads to the max child of the current node is the one to take • Anytime • this is an "anytime" algorithm • you can stop the search at any time and you have a best estimate of your move (to some depth)

33. Problem • I may waste time searching nodes that I would never use • A* doesn't help • since a position may be bad in one move but better after 3 • sacrifice

34. Alpha-beta pruning • Alpha-beta pruning • reduces the size of the search space • without changing the answer • Simple idea • don't consider any moves that are worse than ones you already know about

35. Animated example • http://sern.ucalgary.ca/courses/CPSC/533/W99/presentations/L2_5B_Lima_Neitz/abpruning.html

36. What about chance? • In a game of chance • there is a random element in the game process • Backgammon • the player can only make moves that use the outcome of the dice roll • How do I know what my opponent will do? • I don't • but I can have an expectation

37. Expectiminimax • The idea • Game theoretic utility calculation • Expected value = sum of all outcome values * the likelihood of occurrence • The value of a node is not simply copied from the "best" child • but summed over all possible children

38. Algorithm • Tree has three types of nodes • max nodes • min nodes • chance nodes • Chance nodes calculate the expectation associated with all of the children

39. http://sern.ucalgary.ca/courses/cpsc/533/W99/presentations/L2_5B_Lima_Neitz/chance.htmlhttp://sern.ucalgary.ca/courses/cpsc/533/W99/presentations/L2_5B_Lima_Neitz/chance.html

40. Killer heuristic • One additional optimization • works well in chess • Often a move that is really good • or really bad • Will be really good or bad in multiple board positions • Example • a move that captures my queen • if my queen is under attack • the move in which the opponent takes my queen • will be his best move in most board positions • except the positions in which I move the queen out of attack • If a move leads to a really good or really bad position • try it first when searching • more likely to produce an extreme value that helps alpha-beta search

41. No class next week • Progress report due tonight • 1 or 2 pages of text • saying where you are • Class on 6/5 • CLOS