1 / 37

Exhaustive Search and a first approach to avoiding it

Exhaustive Search and a first approach to avoiding it. Brute Force Methods. guarantee best fitness value is found feasible for ‘small’ data sets only. SAT satisfiability problem. set of propositions prop[n] fitness function logical expression based on propositions boolean fitness(prop)

barbra
Download Presentation

Exhaustive Search and a first approach to avoiding it

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Exhaustive Search and a first approach to avoiding it

  2. Brute Force Methods • guarantee best fitness value is found • feasible for ‘small’ data sets only

  3. SAT satisfiability problem • set of propositions prop[n] • fitness function • logical expression based on propositions boolean fitness(prop) • try all 2n combinations of T/F for propositions

  4. boolean fitness(boolean b[]) //fitness function boolean [] prop = new boolean[n]; boolean satisfied = sat(n-1); boolean sat (int index) // find a solution { if (index==0) //base case { prop[0] = true; if fitness(prop) return true; prop[0] = false; return fitness(prop); } prop[index] = true; // recursive if (sat(index-1)) return true; prop[index] = false; return sat(index-1); } ... ... 2 1 0 ..T ..F .TT .FT .TF .FF TTT FTT TFT FFT TTF FTF TFF FFF

  5. efficiency without risk • pruning the tree: • suppose fitness(prop) is (p0 \/ ~p1) /\ (~p2) partial evaluation ... ... p2 p1 p0 nodes can be ignored: fitness can not be true ..T ..F .TT .FT .TF .FF TTT FTT TFT FFT TTF FTF TFF FFF

  6. TSP travelling salesman • directed edges, complete graph: exhaustive search is the permutation problem enumerate all n! permutations of n distinct items (cities) {0,1,2,3,…,n-1} rank in path array 1 0 D[i][j] 0 1 2 3 3 1 4 2 2 3

  7. TSP travelling salesman O(nn) input(V) // number of cities (Vertices) int[V] rIP // rankInPath, init to 0==not ranked visit(1) // generate visiting sequences void visit(int rank) { for (city = 0 to V-1) if (rIP[city] == 0) // not yet visited { rIP[city] = rank; if (rank == V) fitness(rIP) // base case else visit(rank+1) //recursive case rIP[city] = 0 } } fitness(int[] p) calculate pathlength from D[i][j] if bestpath, save p

  8. efficiency without risk • fix first city  O((n-1)n-1) • use sets instead of searching array  O((n-1)!) == O((n-1)n) • keep partial fitness values • reduce cost of fitness evaluation • apply branch and bound (pruning) BUT… still O(en)

  9. ... ... Branch and bound ..T ..F .TT .FT .TF .FF TTT FTT TFT FFT TTF FTF TFF FFF 0 ... 126 549 274 01 03 02 013 032 023 012 031 021 0123 0132 0312 0321 0213 0231 pathlength = 845 592 693 519

  10. Variations on TSP • undirected edges: D[i][j] == D[j][i] • incomplete graph: some D[i][j] = null • Euclidean distances (on an airplane) • cities are located on plane at (xi,yi) • D[i][j] is computed from coordinates: D[i][j] = D[j][i] = sqrt((xi-xj)2 + (yi-yj)2)  other data structures, efficiencies

  11. Continuousproblem spaces Where is height of land? 1. what scale to sample? x∈(0,1], y∈(0,1] interval length: 0.1: 100 data points 0.01: 10,000 0.001: 1,000,000 --- y  --- x 

  12. Continuousproblem spaces Where is height of land? 2. constraints – ignore water fewer data points but constraints must be tested --- y  --- x 

  13. Continuousproblem spaces Where is height of land? 3. where to locate sample --- y  --- x 

  14. Continuousproblem spaces - NLP Non-Linear Programming problems Typical problems are functions of multiple variables over domains of each Maximize f(x1,x2,x3,…,xn) for x1 ∈ D1, x2 ∈ D2, x3 ∈ D3,…, xn ∈ Dn NLP problems are NP complete *Linear Programming problems are polynomial solvable with provable optimum in O(nk)

  15. Linear Programming*-Simplex Method Fitness function and constraints all linear fitness: example constraints: *assuming you are familiar with approximation methods, p.69-76

  16. Linear Programming with these restrictions: feasible solution space is a convex polyhedron -> solution is at a vertex 2D e.g.:

  17. Simplex Method starts at a vertex (usually xi = 0 ∀i) and “walks along edges” to adjacent vertex, until optimum is reached approx O(n4)

  18. Local Search in Space of Complete Solutions

  19. Complete or Partial Solution Space? ... ... space of partial solutions 2 1 0 ..T ..F .TT .FT .TF .FF TTT FTT TFT FFT TTF FTF TFF FFF space of complete solutions TTT FTT TFT FFT TTF FTF TFF FFF

  20. Partial Solution Space ... ... space of partial solutions 2 1 0 ..T ..F .TT .FT .TF .FF TTT FTT TFT FFT TTF FTF TFF FFF • algorithms are based on tree traversals • must have a partial fitness function • performance is improved by tree ‘pruning’ • coming soon to a lecture near you...

  21. Complete Solution Space ... ... ..T ..F .TT .FT .TF .FF space of complete solutions TTT FTT TFT FFT TTF FTF TFF FFF • start with a (random or guided) solution • move to other solutions seeking better fitness • performance is improved by concentrating on good solutions

  22. Local Search -ideal solution for convex functions -partial solution for complex functions (finds local optimum)

  23. Local search algorithm localSearch(point P in domain) evaluate fP = fitness(P) repeat until no improvement in fitness transform P to a neighbouring point P’ evaluate fP’ if fP’ better than fP, P=P’ return P

  24. 1 DimensionalConvex space Example localSearch(point h=19) evaluate fh = 20(19)-192=19 repeat until no improvement in fitness transform h to a neighbouring point h’ evaluate fh’ if fh’ better than fh, h=h’ return h = 10 m = 20h - h2 in this domain, neighbouring points of h are (h+1), (h-1)

  25. 2 Dimensionalspaces 4 local search neighbours --- y  --- x  Would 8 neighbours be better?

  26. Local search with SAT: representation Any logical expression can be written in Conjunctive Normal Form (CNF) a conjunction (/\) of disjunctions (\/) of propositions and negations: (~P1\/ P3) /\ (P2\/~P3) /\ (P4\/ P1) (A \/ B) -> C = ~(A \/ B)\/ C = (~A /\ ~B) \/ C = (~A \/ C) /\ (~B \/ C)

  27. localSearch(point P in domain) evaluate fP = fitness(P) repeat until no better fitness transform P to neighbour pt P’ evaluate fP’ if fP’ better than fP , P=P’ return P Local searchwith SAT What is a “better” point in search space? What is a “neighbouring” point? FITNESS: number of TRUE disjuncts 1 neighbour has one proposition with different boolean value

  28. localSearch(point P in domain) evaluate fP = fitness(P) repeat until no better fitness transform P to neighbour pt P’ evaluate fP’ if fP’ better than fP , P=P’ return P Local searchwith SAT

  29. Local search with SATbut SAT space is not convex… procedure GSAT // Michalewicz & Fogel begin for i = 1 to MAX_TRIES do T  random truth assignment for j = 1 to MAX_FLIPS do if T satisfies fitness return (T) flip a proposition truth value end end return (“no solution found”) end

  30. localSearch(point P in domain) evaluate fP = fitness(P) repeat until no better fitness transform P to neighbour pt P’ evaluate fP’ if fP’ better than fP , P=P’ return P Local searchwith TSP what is a “better” point in search space? EASY – shorter path what is a “neighbouring” point? i.e., what change to transform a path to a similar path? MANY possible definitions of transformation operators.

  31. TSP and local search • some neighbourhood definitions depend on the type of TSP. e.g., 2-Opt: change two non-adjacent edges: ABCDEFG to ABEDCFG pathlength = pathlength –BC –EF + BE + CF fails when? TSP with directed edges: DE ≠ ED, CD ≠ DC A B G F C E D

  32. Neighbourhood/Search tradeoff Neighbourhoods define the connectivity of the complete search space • larger neighbourhood  more chance of better optimum but longer search time e.g. 2-Opt: n edges, n-3 non-adjacent number of neighbours n(n-3)/2 O(n2)

  33. A B G F C E D TSP neighbourhood definitions swap adjacent cities O(n) 2-Opt: change two non-adjacent edges O(n2) 3-Opt: change among three non adjacent edges δ-path (Lin-Kernighan) C E A B D

  34. A B C D E F G H J K 3-Opt O(n3) THREE possible (2-Opt) neighbours: FOUR possible neighbours: C D E H G F A B J K E D C F G H H G F E D C E D C H G F A B J K F G H C D E H G F E D C

  35. δ-path (Lin-Kernighan) based on 2-Opt with 1 city (A) fixed. A B C D E F G H J K A remove KA and consider all δ-paths (e.g., by adding KE) • record best 2-Opt tour if any (e.g. EF FA) • BUT move to next tour according to best δ-path length (e.g. including FA)

  36. A B C D E F G H J K A Lin-Kernighan local search initial tour T, BestT = T start city A, last city L=K added_list =null; // added edges cannot be removed deleted_list=null; // deleted edges can’t be re-added repeat find best 2-Opt neighbour tour (maybe change BestT) move to tour with best δ-path < BestT if any put added edge in added_list (e.g., KE) put deleted edge in deleted_list(e.g., EF) L = F until no moves (all edges added or no short δ-path)

  37. (2) f(p) (1) Variations on local search p, point in search space, fitness f(p) neighbourhood Np = {p1,…,pk} (1) find first pi with f(pi)> f(p) or (2) find best pjsuch that f(pj) ≥ f(pi) 1≤ i ≤ k pk … p1 p2

More Related