Artificial Intelligence for Games Online and local search

1. Artificial Intelligence for GamesOnline and local search Patrick Olivier p.l.olivier@ncl.ac.uk

2. Local search • In many optimisation problems we are interested in the solution rather than the path to it. • Previously the state space was the paths to goal • Now the state space is "complete" configurations • The approach is to find one or more configurations satisfying constraints then use a local search algorithm and try to improve it • So local search algorithms need only keep a single "current" state rather than a current path, thus being very memory efficient even in very large problems.

3. Example problems • Scheduling • Layout • Evolution • Travelling salesman • N-queens

4. Example – N-queens • Given a n×n chess board, place n queens without any being able to take another in a single chess move. • i.e. Only a single queen in any row, column, or diagonal. • Left example a 4-queens problem (n=4).

5. Objective function • Local search problems need an objective function. • This may be the nearness to goal or simply the negative of the “cost” of a given state. • A high objective function denotes a good solution.

6. Example objective functions • Scheduling: negative time to complete, time spend working/wasted time. • Layout: number of items/amount of space • Evolution: reproductive success of a species • Travelling salesman: negative number of cities visited twice or more • N-queens: number of non-attacking queens

7. State space landscape

8. State space landscape

9. Hill-climbing search • “Climbing Everest in thick fog with amnesia” • Repeatedly “climb” to the neighboring state with highest objective function until no neighbor has higher objective function.

10. Local maxima/minina • Problem: depending on initial state, can get stuck in local maxima/minina 1/ (1+H(n)) = 1/17 1/ (1+H(n)) = 1/2 Local minima

11. Local beam search • keep track of k states rather than just one • start with k randomly generated states • at each iteration, all the successors of all k states are generated • if any one is a goal state, stop; else select k best successors from complete list and repeat.

12. Simulated annealing search • Idea: escape local maxima by allowing some "bad" moves but gradually decrease their frequency and range (VSLI layout, scheduling)

13. Simulated annealing example • Point feature labelling

14. Genetic algorithm search • population of k randomly generated states • a state is represented as a string over a finite alphabet (often a string of 0s and 1s) • evaluation function (fitness function) - higher values for better states. • produce the next generation of k states by selection, crossover, and mutation • rates for each of these configure search • elitism / crossover rate / mutation rate

15. Genetic algorithms in games • computationally expensive so primarily used as an offline form of learning • Cloak, Dagger & DNA (Oidian Systems) • 4 DNA strands defining opponent behaviour • between battles, opponents play each other • Creatures (Millennium Interactive) • genetic algorithms to learning the weights in a neural network that defines behaviour

16. Online search • All previous techniques have focused on offline reasoning (think first then act). • Now we will briefly look at online search (think, act, think, act, ...) • Advantageous in dynamic situations or those with only partial information.

17. “Real-time” search concepts • in A* the whole path is computed off-line, before the agent walks through the path • this solution is only valid for static worlds • if the world changes in the meantime, the initial path is no longer valid: • new obstacles appear • position of goal changes (e.g. moving target)

18. “Real-time” definitions • off-line (non real-time): the solution is computed in a given amount of time before being executed • real-time: one move computed at a time, and that move executed before computing the next • anytime: the algorithm constantly improves its solution through time capable of providing “current best” at any time

19. Agent-based (online) search • for example: • mobile robot • NPC without perfect knowledge • agent that must act now with limited information • planning and execution are interleaved • could apply standard search techniques: • best-first (but we know it is poor) • depth-first (has to physically back-track) • A* (but nodes in the fringe are not accessible)

20. LRTA*: Learning Real-time A* • augment hill-climbing with memory • store “current best estimate” • follow path based on neighbours’ estimates • update estimates based on experience • experience  learning • flatten out local maxima…

21. 1 1 1 1 1 1 1 1 1 1 2 4 1 9 9 3 4 9 1 8 8 8 2 4 2 4 1 3 4 5 5 9 8 1 4 1 5 9 8 4 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 LRTA*: example

22. Learning real-time A*