CMSC 100 Heuristic Search and Game Playing

1. CMSC 100Heuristic Search and Game Playing Professor Marie desJardinsTuesday, November 13, 2012 E-Commerce and Databases

2. Summary of Topics • What is heuristic search? • Examples of search problems • Search methods • Uninformed search • Informed search • Local search • Game trees

3. Building Goal-Based Intelligent Agents To build a goal-based agent we need to answer the following questions: • What is the goal to be achieved? • What are the actions? • What relevant information is needed to describe the state of the world, describe the available transitions, and solve the problem? Initial state Goal state Actions

4. Representing States • What information is needed to sufficiently describe all relevant aspects to solving the goal? • That is, what knowledge needs to be represented in a state description to adequately describe the current state or situation of the world? • The size of a problem is usually described in terms of the number of states that are possible. • Tic-Tac-Toe has about 39 states. • Checkers has about 1040 states. • Rubik’s Cube has about 1019 states. • Chess has about 10120 states in a typical game.

5. Real-world Search Problems • Route finding (GPS algorithms, Google Maps) • Touring (traveling salesman) • Logistics • VLSI layout • Robot navigation • Learning

6. 8-Puzzle Given an initial configuration of 8 numbered tiles on a 3 x 3 board, move the tiles into a desired goal configuration of the tiles.

7. 8-Puzzle Encoding • State: 3 x 3 array configuration of the tiles on the board. • 4 Operators: Move Blank Square Left, Right, Up or Down. • This is a more efficient encoding of the operators than one in which each of four possible moves for each of the 8 distinct tiles is used. • Initial State: A particular configuration of the board. • Goal: A particular configuration of the board. • What does the state space look like?

8. Solution Cost • A solution is a sequence of operators that is associated with a path in a state space from a start node to a goal node. • The cost of a solution is the sum of the arc costs on the solution path. • If all arcs have the same (unit) cost, then the solution cost is just the length of the solution (number of steps / state transitions)

9. Evaluating Search Strategies • Completeness • Guarantees finding a solution whenever one exists • Time complexity • How long (worst or average case) does it take to find a solution? Usually measured in terms of the number of nodes expanded • Space complexity • How much space is used by the algorithm? Usually measured in terms of the maximum size of the “nodes” list during the search • Optimality/Admissibility • If a solution is found, is it guaranteed to be an optimal one? That is, is it the one with minimum cost?

10. Search Methods • Uninformed search strategies • Also known as “blind search,” uninformed search strategies use no information about the likely “direction” of the goal node(s) • Variations on “generate and test” or “trial and error” approach • Uninformed search methods: breadth-first, depth-first, uniform-cost • Informed search strategies • Also known as “heuristic search,” informed search strategies use information about the domain to (try to) (usually) head in the general direction of the goal node(s) • Informed search methods: greedy search, (A*) • (Informed search is what your GPS does!!)

11. Search Methods cont. • Local search strategies • Pick a starting solution (that might not be very good) and incrementally try to improve it • Local search methods: hill-climbing, genetic algorithms • Game trees • Search strategies for situations where you have an opponent who gets to make some of the moves • Try to pick moves that will let you win most of the time by “looking ahead” to see what your opponent might do

12. Uninformed Search

13. Maze Example • Reach the goal as cheaply as possible • Start at “S” • Try to get to “G” • Blue area costs ten dollars to move into • Every other area costs one dollar • No point in retracing steps... (treat dead ends as having cost ∞)

14. Maze Search Tree Cost: 3 Cost: 4 Cost: ∞ • Search tree == all possible paths • Try left, then straight, then back • Never reverse direction Cost: 2 Cost: 22 Cost: 23 Cost: 0 Cost: 1 • CHALLENGE: How to explore (generate) only the part of this search tree that actually leads to a solution? • That’s what SEARCH does! Cost: 2 Cost: ∞

15. Depth-First Search (DFS) • Enqueue nodes on nodes in LIFO (last-in, first-out) order. That is, nodes used as a stack data structure to order nodes. • Like walking through a maze, always following the rightmost branch • When search hits a dead end, back up one step at a time • Can find long solutions quickly if lucky (and short solutions slowly if unlucky!)

16. DFS Maze Solution (Step 0) Frontier(revisit later) Current Cost: 0

17. DFS Maze Solution (Step 1) Cost: 0 Cost: 1

18. DFS Maze Solution (Step 2) Cost: 2 Cost: 0 Cost: 1 Cost: 2

19. DFS Maze Solution (Step 3) Cost: 3 Cost: 2 Cost: 22 Cost: 0 Cost: 1 Cost: 2

20. DFS Maze Solution (Step 4) Cost: 3 Cost: 4 Cost: 2 Cost: 22 Cost: 0 Cost: 1 Cost: 2

21. DFS Maze Solution (Step 5) Cost: 3 Cost: 4 Cost: ∞ Cost: 2 Cost: 22 Cost: 0 Cost: 1 Cost: 2

22. DFS Maze Solution (Step 6) Cost: 3 Cost: 4 Cost: ∞ Cost: 2 Cost: 22 Cost: 0 Cost: 1 Cost: 2

23. DFS Maze Solution (Step 7) Cost: 3 Cost: 4 Cost: ∞ Cost: 2 Cost: 22 Cost: 23 Cost: 0 Cost: 1 Cost: 2

24. Breadth-First Search • Enqueue nodes on nodes in FIFO (first-in, first-out) order. • Like having a team of “gremlins” who can keep track of every branching point in a maze • Only one gremlin can move at a time, and breadth-first search moves them each in turn, duplicating the gremlins if they reach a “fork in the road” – one for each fork! • Finds the shortest path, but sometimes very slowly (and at the cost of generating lots of gremlins!)

25. BFS Maze Solution (Step 0) Cost: 0

26. DFS Maze Solution (Step 1) Cost: 0 Cost: 1

27. BFS Maze Solution (Step 2) Cost: 2 Cost: 0 Cost: 1 Cost: 2

28. BFS Maze Solution (Step 3) Cost: 3 Cost: 2 Cost: 22 Cost: 0 Cost: 1 Cost: 2

29. BFS Maze Solution (Step 4) Cost: 3 Cost: 2 Cost: 22 Cost: 0 Cost: 1 Cost: 2 Cost: ∞

30. BFS Maze Solution (Step 5) Cost: 3 Cost: 4 Cost: 2 Cost: 22 Cost: 0 Cost: 1 Cost: 2 Cost: ∞

31. BFS Maze Solution (Step 6) Cost: 3 Cost: 4 Cost: 2 Cost: 22 Cost: 23 Cost: 0 Cost: 1 Cost: 2 Cost: ∞

32. Uniform Cost Search • Enqueue nodes by path cost. That is, let priority = cost of the path from the start node to the current node. Sort nodes by increasing value of cost (try low-cost nodes first) • Called “Dijkstra’s Algorithm” in the algorithms literature; similar to “Branch and Bound Algorithm” from operations research • Finds the very best path, but can take a very long time and use a very large amount of memory!

33. UCS Maze Solution (Step 0) Cost: 0

34. UCS Maze Solution (Step 1) Cost: 0 Cost: 1

35. UCS Maze Solution (Step 2) Cost: 2 Cost: 0 Cost: 1 Cost: 2

36. UCS Maze Solution (Step 3) Cost: 3 Cost: 2 Cost: 22 Cost: 0 Cost: 1 Cost: 2

37. UCS Maze Solution (Step 4) Cost: 3 Cost: 2 Cost: 22 Cost: 0 Cost: 1 Cost: 2 Cost: ∞

38. UCS Maze Solution (Step 5) Cost: 3 Cost: 4 Cost: ∞ Cost: 2 Cost: 22 Cost: 0 Cost: 1 Cost: 2 Cost: ∞

39. UCS Maze Solution (Step 6) Cost: 3 Cost: 4 Cost: ∞ Cost: 2 Cost: 22 Cost: 0 Cost: 1 Cost: 2 Cost: ∞

40. UCS Maze Solution (Step 7) Cost: 3 Cost: 4 Cost: ∞ Cost: 2 Cost: 22 Cost: 23 Cost: 0 Cost: 1 Cost: 2 Cost: ∞

41. Holy Grail Search If only we knew where we were headed…

42. Informed Search

43. What’s a Heuristic? From WordNet (r) 1.6 heuristic adj 1: (computer science) relating to or using a heuristic rule 2: of or relating to a general formulation that serves to guide investigation [ant: algorithmic] n : a commonsense rule (or set of rules) intended to increase the probability of solving some problem [syn: heuristic rule, heuristic program]

44. Informed Search: Use What You Know! • Add domain-specific information to select the best path along which to continue searching • Define a heuristic function that estimates how close a node n is to the goal node • Incorporate the heuristic function into search to choose the most promising branch along which to search: • Greedy search: Go in the most promising direction of any frontier node (ignoring how much it cost you to get to that frontier node) • A* search: Combine the work you’ve done so far to get to a frontier node, plus the heuristic estimate to the goal node, and pick the most promising in terms of total cost

45. Local Search

46. Local Search • Another approach to search involves starting with an initial guess at a solution and gradually improving it until it is a legal solution or the best that can be found. • Also known as “incremental improvement” search • Some examples: • Hill climbing • Genetic algorithms

47. Hill Climbing on a Surface of States Height Defined by Evaluation Function

48. Drawbacks of Hill Climbing • Problems: • Local Maxima: peaks that aren’t the highest point in the space • Plateaus: the space has a broad flat region that gives the search algorithm no direction (random walk) • Ridges: flat like a plateau, but with dropoffs to the sides; steps to the North, East, South and West may go down, but a step to the NW may go up. • Remedies: • Random restart • Problem reformulation • Some problem spaces are great for hill climbing and others are terrible.

49. Genetic Algorithms • Start with k random states (the initial population) • New states are generated by “mutating” a single state or “reproducing” (combining) two parent states (selected according to their fitness) • Encoding used for the “genome” of an individual strongly affects the behavior of the search • Genetic algorithms / genetic programming are a large and active area of research

50. Game Playing