1 / 44

Local and Stochastic Search

Local and Stochastic Search. Based on Russ Greiner’s notes. A Different Approach. So far: systematic exploration: Explore full search space (possibly) using principled pruning (A*, . . . ) Best such algorithms (IDA*) can handle

mindy
Download Presentation

Local and Stochastic Search

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Local and Stochastic Search Based on Russ Greiner’s notes

  2. A Different Approach • So far: systematic exploration: • Explore full search space (possibly) using principled pruning (A*, . . . ) • Best such algorithms (IDA*) can handle • 10100 states ≈ 500 binary-valued variables (ballpark figures only!) • but. . . some real-world problem have 10,000 to 100,000 variables 1030,000 states • We need a completely different approach: • Local Search Methods or • Iterative Improvement Methods

  3. Local Search Methods • Applicable when seeking Goal State & don't care how to get there. E.g., • N-queens, map coloring, . . . • VLSI layout, planning, scheduling, TSP, time-tabling, . . . • Many (most?) real Operations Research problems are solved using local search!

  4. Random Search • Select (random) initial state (initial guess at solution) • Make local modification to improve current state • Repeat Step 2 until goal state found (or out of time) Requirements: • generate a random (probably-not-optimal) guess • evaluate quality of guess • move to other states (well-defined neighborhood function) . . . and do these operations quickly. . .

  5. Example: 4 Queen • States: 4 queens in 4 columns (256 states) • Operators: move queen in column • Goal test: no attacks • Evaluation: h(n) = number of attacks

  6. Example: Graph Coloring • Start with random coloring of nodes • Change color of one node to reduce # of conflict • Repeat 2

  7. Hill-Climbing

  8. 3-Coloring

  9. Problems with Hill Climbing

  10. Problems with Hill Climbing • Foothills / Local Optimal: No neighbor is better, but not at global optimum. • (Maze: may have to move AWAY from goal to find (best) solution) • Plateaus: All neighbors look the same. • (8-puzzle: perhaps no action will change # of tiles out of place) • Ridges: going through a sequence of local maxima • Ignorance of the peak: Am I done?

  11. Hill-climbing search: 8-queens problem • A local minimum with h = 1

  12. Issues The Goal is to find GLOBAL optimum. • How to avoid LOCAL optima? • How long to do plateau walk? • When to stop? • Climb down hill? When?

  13. Overcoming Local Optimum and Plateau • Random restarts • Simulated annealing • Mixed-in random walk • Tabu search (prevent repeated states) • Others (Genetic algorithms/programming, . . . )

  14. Local Search Example: SAT(Russell & Norvig p221-225) • Many real-world problems can be translated into propositional logic (A v B v C) ^ (¬B v C v D) ^ (A v ¬C v D) . . . solved by finding truth assignment to variables (A, B, C, . . . ) that satisfies the formula • Applications • planning and scheduling • circuit diagnosis and synthesis • deductive reasoning • software testing • . . .

  15. Satisfiability Testing • Best-known systematic method: • Davis-Putnam Procedure (1960) • Backtracking depth-first search (DFS) through space of truth assignments (with unit-propagation)

  16. Greedy Local Search: GSAT • GSAT: 1. Guess a random truth assignment 2. Flip value assigned to the variable that yields the largest increase of the total # of satisfied clauses. (Note: Flip even if the increase is 0, but not negative) 3. Repeat until all clauses satisfied, or have performed “enough” flips 4. If no sat-assign found, repeat entire process, starting from a different initial random assignment.

  17. Systematic vs. Stochastic • Systematic search: • DP systematically checks all possible assignments. • Can determine if the formula is unsatisfiable. • Stochastic search: • Guided random search approach. • Once we find it, we're done. • Can't determine unsatisfiability.

  18. GSAT vs. DP on Hard Random Instances

  19. What Makes a SAT Problem Hard? • Suppose we have n variables to write m clauses with k variables each. • We negate the resulting variables randomly (flip an unbiased coin). • What is the number of possible sentences in terms of n, m, and k ? • What if there are many variables and only a smaller number of clauses? • What if there are many clauses and only a smaller number of variables?

  20. Phase Transition • For 3-SAT • m/n < 4.2, under constrained: nearly all sentences sat. • m/n > 4.3, over constrained: nearly all sentences unsat. • m/n ~ 4.26, critically constrained: need to search

  21. Phase Transition • Under-constrained problems are easy, just guess an assignment. • Over-constrained problems are easy, just say “unsatisfiable” (often easy to verify using Davis-Putnam). • At a m/n ratio of around 4.26, there is a phase transition between these two different types of easy problems. This transition sharpens as n increases. • For large n, hard problems are extremely rare (in some sense).

  22. Improvements to Basic Local Search • Issue: • How to move more quickly to successively better plateaus? • Avoid “getting stuck” / local maxima? • Idea: Introduce uphill moves (“noise”) to escape from plateaus/local maxima • Noise strategies: 1. Simulated Annealing • Kirkpatrick et al. 1982; Metropolis et al. 1953 2. Mixed Random Walk • Selman and Kautz 1993

  23. Simulated Annealing • Pick a random variable If flip improves assignment: do it. Else flip with probability p = e-δ/T = 1/(eδ/T ) • δ = # of additional clauses becoming unsatisfied • T = “temperature” • Higher temperature = greater chance of wrong-way move • Slowly decrease T from high temperature to near 0. • Q: What is p as T tends to infinity? . . . as T tends to 0? For δ = 0? • Thus, bad moves likely to be allowed at the start when T is high.

  24. Simulated Annealing Algorithm

  25. Notes on SA • Noise model based on statistical mechanics • . . . introduced as analogue to physical process of growing crystals • Kirkpatrick et al. 1982; Metropolis et al. 1953 • Convergence: • 1. W/ exponential schedule, will converge to global optimum • 2. No more-precise convergence rate (Recent work on rapidly mixing Markov chains) • Key aspect: upwards / sideways moves • Expensive, but (if have enough time) can be best • Hundreds of papers/ year; • Many applications: VLSI layout, factory scheduling, . . .

  26. Properties of simulated annealing search • One can prove: If T decreases slowly enough, then simulated annealing search will find a global optimum with probability approaching 1 • Widely used in VLSI layout, airline scheduling, etc

  27. Local Beam Search • Keep track of k states rather than just one • Start with k randomly generated states • At each iteration, all the successors of all k states are generated • If any one is a goal state, stop; else select the k best successors from the complete list and repeat.

  28. Pure WalkSat • PureWalkSat( formula ) Guess initial assignment While unsatisfied do Select unsatisfied clause c = ±Xi v ±Xj v ±Xk Select variable v in unsatisfied clause c Flip v

  29. Example:

  30. Mixing Random Walk with Greedy Local Search • Usual issues: • Termination conditions • Multiple restarts • Value of p determined empirically, by finding best setting for problem class

  31. Finding the best value of p • WalkSat[p]: • W/prob p, flip var in unsatisfied clause • W/prob 1-p, make a greedy flip to minimize # of unsatisfied clauses • Q: What value for p? • Let: Q[p, c] be quality of using WalkSat[p] on problem c. • Q[p, c] = Time to return answer, or = 1 if WalkSat[p] return (correct) answer within 5 minutes and 0 otherwise, or = . . . perhaps some combination of both . . . • Then, find p that maximize the average performance of WalkSat[p] on a set of challenge problems.

  32. Experimental Results: Hard Random 3CNF • Time in seconds • Effectiveness: prob. that random initial assignment leads to a solution. • Complete methods, such as DP, up to 400 variables • Mixed Walk better than Simulated Annealing • better than Basic GSAT • better than Davis-Putnam

  33. Overcoming Local Optimum and Plateau • Random restarts • Simulated annealing • Mixed-in random walk • Tabu search (prevent repeated states) • Others (Genetic algorithms/programming, . . . )

  34. Other Techniques • random restarts: restart at new random state after pre-defined # of local steps. • [Done by GSAT] • tabu: prevent returning quickly to same state. • Implement: Keep fixed length queue (tabu list). • Add most recent step to queue; drop oldest step. • Never make step that's on current tabu list. • Example: • without tabu: • flip v1, v2, v4, v2, v10, v11, v1, v10, v3, ... • with tabu (length 5)—possible sequence: • flip v1, v2, v4, v10, v11, v1, v3, ... • Tabu very powerful; competitive w/ simulated annealing or random walk (depending on the domain)

  35. Genetic Algorithms • A class of probabilistic optimization algorithms • A genetic algorithm maintains a population of candidate solutions for the problem at hand, and makes it evolve by iteratively applying a set of stochastic operators • Inspired by the biological evolution process • Uses concepts of “Natural Selection” and “Genetic Inheritance” (Darwin 1859) • Originally developed by John Holland (1975)

  36. The Algorithm • Randomly generate an initial population. • Select parents and “reproduce” the next generation • Evaluate the fitness of the new generation • Replace the old generation with the new generation • Repeat step 2 though 4 till iteration N

  37. Stochastic Operators • Cross-over • decomposes two distinct solutions and then randomly mixes their parts to form novel solutions • Mutation • randomly perturbs a candidate solution

  38. Genetic Algorithm Operators Mutation and Crossover 1 0 1 0 1 1 1 Parent 1 1 1 0 0 0 1 1 Parent 2 1 0 1 0 0 1 1 Child 1 Mutation Child 2 1 1 0 0 1 1 0

  39. Examples: Recipe • To find optimal quantity of three major ingredients (sugar, wine, sesame oil) denoting ounces. • Use an alphabet of 1-9 denoting ounces.. • Solutions might be 1-1-1, 2-1-4, 3-3-1.

  40. Examples • Mutation: The recipe example: 1-2-3 may be changed to 1-3-3 or 3-2-3. • Parameters to adjust • How often? • How many digits change? • How big?

  41. More examples: • Crossover Recipe : Parents 1-3-3 & 3-2-3. Crossover point after the first digit. Generate two offspring: 3-3-3 and 1-2-3. Can have one or two point crossover.

  42. Randomized Algorithms • A randomized algorithm is defined as an algorithm where at least one decision is based on a random choice.

  43. Monte Carlo and Las Vegas There are two kinds of randomized algorithms: • Las Vegas: A Las Vegas algorithm always produces the correct answer, but randomness only influence the resources used. E.g. Randomized Quiksort with pivot randomly chosen. • Monte Carlo: A Monte Carlo algorithm runs for a fixed number of steps for each input and produces an answer that is correct with a bounded probability

  44. Local Search Summary • Surprisingly efficient search technique • Wide range of applications • Formal properties elusive • Intuitive explanation: • Search spaces are too large for systematic search anyway. . . • Area will most likely continue to thrive

More Related