1 / 50

Analysis & Design of Algorithms (CSCE 321)

Analysis & Design of Algorithms (CSCE 321). Prof. Amr Goneid Department of Computer Science, AUC Part 11. Backtracking Algorithms. Backtracking Algorithms. Backtracking Algorithms. Problem Statement Brute Force & Backtracking Permutation Tree The 4-Queens Problem Sum of Subsets

keitha
Download Presentation

Analysis & Design of Algorithms (CSCE 321)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Analysis & Design of Algorithms(CSCE 321) Prof. Amr Goneid Department of Computer Science, AUC Part 11. Backtracking Algorithms Prof. Amr Goneid, AUC

  2. Backtracking Algorithms Prof. Amr Goneid, AUC

  3. Backtracking Algorithms • Problem Statement • Brute Force & Backtracking • Permutation Tree • The 4-Queens Problem • Sum of Subsets • Unbound DFG: Permutation Tree • Bound DFG: Backtracking • General Backtracking Algorithm Prof. Amr Goneid, AUC

  4. Backtracking Algorithms • Example: n-Binary Variables Problem • The n-Queens Problem • Another Backtracking Schema • The Hamiltonian Circuit Problem by Backtracking Prof. Amr Goneid, AUC

  5. 1. Problem Statement • Given sets S1, S2, …, Sn of values x with m1, m2, …, mn values in the sets. • We seek a solution vector (n-tuple) X = (x1, x2, …, xn) chosen from the sets out of m = m1 m2 ….mn possible candidates satisfying a criterion function P(X) Prof. Amr Goneid, AUC

  6. 2. Brute Force & Backtracking • Sometimes the best algorithm for a problem is to try all possibilities. • This is always slow, but straightforward. • Brute Force (Exhaustive Search) Methods: Will try all possible m trials and save those satisfying P(X) Prof. Amr Goneid, AUC

  7. Brute Force & Backtracking • Backtracking: • Yields same answer with fewer than m trials. • Builds solution vector one component at a time. • Examines if a partial vector Xi = (x1,x2,..,xi) has a chance of success. • If it does not satisfy constraints, we drop out the k remaining trials, k = mi+1 mi+2 …mn Prof. Amr Goneid, AUC

  8. The 8-Queens Problem • Place 8 Queens on an 8 x 8 chessboard such that no two can attack each other. There are about 4.4 billion trials. • Obvious: Each queen must be on a different row. All solutions are 8-tuples X = (x1,x2,..,x8) , where xi is the column number of the ith queen. • Brute Force Trials:m = 8*8*….*8 = 88 = 224 = 16 M • Constraint(1): Each queen must be on a different column. This reduces number of trials to 8! = 40,320 • Constraint(2): No two queens can be on the same diagonal. Prof. Amr Goneid, AUC

  9. The 8-Queens Problem (continued) • Backtracking: will use only 1625 trials to achieve solution. • Possible Solution: X = (4 , 6 , 8 , 2 , 7 , 1 , 3 , 5) Prof. Amr Goneid, AUC

  10. 3. Permutation Tree • Represents the problem to be solved. • Problem State: A node in the tree. • Layer: between two consecutive levels and represents one variable in the tuple. Edges from level (i) to level (i+1) are labeled with values of variable xi. • State Space: All paths from root to other nodes. • Solution States: Nodes from the root defining a tuple. • Answer States: Solution states satisfying implicit conditions. • Node Generation: Depth First Prof. Amr Goneid, AUC

  11. Example • Problem: Find all sequences of three binary variables such that no two consecutive values are the same. • S1 = S2 = S3 = {0,1} , m1 = m2 = m3 = 2 • n = 3 , xi = 0 or 1 , i = 1 , 2 , 3 • Total number of brute force trials = m1m2m3 = 2 * 2 * 2 = 8 • Implicit Conditions: x1 x2 , x2  x3 Prof. Amr Goneid, AUC

  12. Brute Force Permutation Tree 1 x1 0 1 2 9 1 0 x2 0 1 3 6 10 13 0 1 1 0 0 x3 0 1 1 4 7 8 11 12 15 5 14 {0,1,0) {1,0,1) Prof. Amr Goneid, AUC

  13. Backtracking Method • Generate nodes in Depth First order. • Kill nodes (and their children) not satisfying constraints. • Backtrack to higher level to seek a different path to a leaf node. • If all leaves are killed, the problem has no solution. Prof. Amr Goneid, AUC

  14. Bactracking Permutation Tree 1 x1 0 1 2 9 1 0 x2 0 1 3 6 10 13 0 x3 1 0 0 1 0 1 1 4 7 8 11 12 15 5 14 {0,1,0) {1,0,1) Prof. Amr Goneid, AUC

  15. The Solution • # of solution states = # of leaves = 8 • Total # of nodes in tree = measure of brute force cost = 15 • # of nodes generated = measure of Backtracking cost = 11 • # of surviving nodes = 7 • # of killed parent nodes = 4 • # of answer states (3-tuples) = 2 X1 = {0,1,0} , X2 = {1,0,1} Prof. Amr Goneid, AUC

  16. Conclusion • Backtracking is a Divide & Conquer Brute Force (exhaustive) search with pruning. • Saves time by killing internal nodes that have no useful leaves. • Let Ne and Nb be the number of nodes generated by the exhaustive search and backtracking methods, respectively. • A measure of the Gain obtained by backtracking is G = (1 – Nb / Ne)*100 % • For the previous example, G = (1 – 11/15)*100 = 26.7% Prof. Amr Goneid, AUC

  17. 4. The 4-Queens Problem(a) The Problem • Place 4 Queens on an 4 x 4 chessboard such that no two can attack each other. • With no constraints, there are 16 * 15 * 14 * 13 = 43,680 possible placements. • Constraint (1): Each Queen must be on a different row. Now we seek the 4-tuples X = {x1 , x2 , x3 , x4} representing column numbers. There are 4*4*4*4 = 256 such tuples. Prof. Amr Goneid, AUC

  18. The Problem (cont.) • Constraint(2):Each Queen must be on a different column. The number of possible tuples reduces to 4*3*2*1 = 4! = 24 • Backtracking will be used to impose the final No Attackconstraint so that no two queens can be placed on the same diagonal Prof. Amr Goneid, AUC

  19. (b) The Permutation Tree • The root will have 4 children • In general, each node in level L will have (4-L+1) children. The total # of nodes in the tree will be 1 + 4 + 4*3 + 4*3*2 + 4*3*2*1 = 65 nodes • The # of leaves will be 4! = 24 tuples. Which of these will be answers ? Prof. Amr Goneid, AUC

  20. (c) Portion of the Permutation Tree 1 2 1 18 2 4 2 4 1 3 3 3 19 29 8 13 24 4 2 3 2 1 9 11 14 16 30 3 3 15 31 X = {2,4,1,3} Prof. Amr Goneid, AUC

  21. (d) Solutions Leaf 31 Leaf 39 { 2 , 4 , 1 , 3 } { 3 , 1 , 4 , 2 } Prof. Amr Goneid, AUC

  22. (e) Performance • Brute Force cost = 65 • Backtracking: # nodes generated = 32 # nodes surviving = 18 # parent nodes killed = 14 Gain G = (1-32/65)*100 = 50.8% Prof. Amr Goneid, AUC

  23. 5. Sum of Subsets(a) The Problem • Given a set W of positive integers wi , i = 1,2,…,n and m, find all subsets of W whose sums are equal to m. • Example: n = 4 , W = {11,13,24,7}, m = 31 There are 2 possible subsets: S1 = {11,13,7} , S2 = {24,7} Prof. Amr Goneid, AUC

  24. The Problem (cont.) • Constraints: 1.A member appears only once in the subset. 2. The sum of a subset is exactly m. 3. No multiple instances of the same subset. For example: {1,4,2} and {1,2,4} This is satisfied by requiring wi < w i+1 Prof. Amr Goneid, AUC

  25. (b) Backtracking Solution • Consider the previous example with n = 4 , W = {11,13,24,7}, m = 31 • Let X = {x1,x2,..,xn} be a solution such that xi {0,1} , xi = 1 if wi is selected and xi = 0 otherwise. Hence, we obtain fixed-size tuples. Prof. Amr Goneid, AUC

  26. Brute Force Permutation Tree • The permutation tree will be a complete binary tree with a height of n+1. The # of leaves (possible subsets) will be 2n = 16 and the total # of nodes will be 31. Prof. Amr Goneid, AUC

  27. (c) Portion of Permutation Tree 1 0 1 2 17 1 0 3 10 1 0 0 1 4 7 11 14 1 0 1 5 6 8 9 12 13 15 16 Prof. Amr Goneid, AUC

  28. Performance • # nodes generated = 25 # nodes survived = 14 # killed = 11 • G = 19.4% • Two possible solutions X1 = {1,1,0,1} and X2 = {0,0,1,1} Prof. Amr Goneid, AUC

  29. 6. Unbound DFG: Permutation Tree • The full (Brute Force) permutation tree is generated by an unbound Depth-First Generation (DFG) algorithm • Example: Binary strings of length (n) bits. The DFG algorithm generates a full permutation tree of n+1 levels Prof. Amr Goneid, AUC

  30. Unbound DFG: Algorithm // Assume there is a root node void Unbound_DFG(int k, int n) { for (i = 0; i <= 1; i++) { Generate edge (i) from current parent; Generate child node; if (k == n) then leaf node has been reached; else Unbound_DFG(k+1, n); } } Prof. Amr Goneid, AUC

  31. Example:All 2-bit strings 1 x1 0 1 2 5 x2 1 1 0 0 3 4 6 7 {00} {01} {10} {11} n = 2; Unbound_DFG(1,n) Prof. Amr Goneid, AUC

  32. 7. Bound DFG: Backtracking • A partial permutation tree is generated by a bound Depth-First Generation (DFG) algorithm • Example: n-bit binary strings, with a bounding function B(k,i) == true The DFG algorithm generates a partial permutation tree of n+1 levels Prof. Amr Goneid, AUC

  33. Bound DFG: Algorithm // Assume there is a root node void Bound_DFG(int k, int n) { for (i = 0; i <= 1; i++) { if ( B( k , i ) ){ Generate edge (i) from current parent; Generate child node; if (k == n) then leaf node has been reached; else Bound_DFG(k+1, n); } } } Prof. Amr Goneid, AUC

  34. Example: n-bit binary strings 1 x1 0 1 2 5 x2 0 0 3 6 {00} {10} n = 2; Bound: 2nd bit should not be 1 B’(k,i) = (k==2) && (i==1) Hence B(k,i) = (k == 1) || (i == 0) Invoke as Bound_DFG(1,n) Prof. Amr Goneid, AUC

  35. 8. General Backtracking Algorithm • Consider: n = no. of variables k = index of a variable x1 , x2, .. ,xk, .. xn is a path to a solution state. m = no. of possible values for x vi = a value for a variable, i = 1, 2, .. m B(k,i) = Bounding Function = true if xk can take the value vi Prof. Amr Goneid, AUC

  36. Brute Force Algorithm void Brute_Force (int k, int n) { for all possible values vi of a variable (i = 1..m) { Let variable xk take the value vi ; if (xk is the last variable ) output vector x[1:n]; else Brute_Force (k+1, n); } } Prof. Amr Goneid, AUC

  37. General Brute Force Code void Brute_Force (int k, int n) { for (i = 1; i <= m; i++) { x[k] = v[i] ; if (k == n) output vector x[1:n]; else Brute_Force (k+1, n); } } Prof. Amr Goneid, AUC

  38. Backtracking Algorithm void Backtrack(int k, int n) { for (i = 1; i <= m; i++) { if ( B( k , i )) { // Pruning x[k] = v[i] ; if (k == n) output vector x[1:n]; else Backtrack(k+1, n); } } } Prof. Amr Goneid, AUC

  39. 9. Example:n-Binary bits Problem • Problem: Find all strings of n-binary bits such that no two consecutive bits are the same. • n given , m = 2 , xk = 0 or 1 , k = 1 , 2 , .. , n • Bounding Function: Assume x0 = -1 then x1 x2 , x2  x3 , …… or if (xk-1  i) then xk can take the value (i) Prof. Amr Goneid, AUC

  40. Backtracking Algorithm void Backtrack(int k, int n) { for (i = 0; i <= 1; i++) { if ( x[k-1] != i) { // assume x[0] = -1 x[k] = i ; if (k == n) output vector x[1:n]; else Backtrack(k+1, n); } } } Prof. Amr Goneid, AUC

  41. Permutation Tree for n = 3 1 x1 0 1 2 9 1 0 x2 0 1 3 6 10 13 0 1 1 0 x3 7 8 11 12 {0,1,0) {1,0,1) Prof. Amr Goneid, AUC

  42. 10. The n-Queens Problem(a) The Problem • Find all possible arrangements of n Queens on an n x n chessboard such that no two can attack each other. • Example: 8 queens on an 8x8 board The problem has 92 solutions. Only 12 are unique, others are reflections or rotations. Prof. Amr Goneid, AUC

  43. The n-Queens Problem • Pre-Condition: Each Queen is on a different row. • We seek the n-tuples X = {x1 , x2 , …, xn} representing column numbers satisfying the problem. • Hence, we seek all permutations of X Prof. Amr Goneid, AUC

  44. (b) The Bounding Function • Assume that Queens 1, 2, .. K-1 have already been properly placed. • The bounding function for Queen (k) is that it can be placed in column (i) iff: • It does not share the same column (i) with any of the previous queens (j), j = 1,2,.., k-1, i.e Xj i • It is not on the same diagonal with any of the previous queens: │k-j│  │Xj - i│ , j = 1,2,.., k-1 Qj Row j Row k Qk Col X[ j] Col (i) Prof. Amr Goneid, AUC

  45. The Bounding Function bool can_place (int k, int i) { for (int j = 1; j < k; j++) if ((x[ j ] == i) || abs(x[ j ] - i) == abs(k-j)) return false; return true; } Prof. Amr Goneid, AUC

  46. (c)Backtracking Algorithm void NQueens(int k, int n) { for (i = 1; i <= n; i++) { if ( can_place( k , i ) ){ x[k] = i ; if (k == n) output vector x[1:n]; else NQueens(k+1, n); } } } Prof. Amr Goneid, AUC

  47. The 8-Queens Problem Animation Prof. Amr Goneid, AUC

  48. 11. Another Backtracking Schema • Another schema exists when we replace the bounding function by a function that assigns to x[k] a legal value after x[1: k-1] have already been assigned legal values out of m possible values. If no such value is available, the algorithm backtracks. • Assume the function to be NextValue (k , n) and the vector x[1:n] is initially set to zero. Prof. Amr Goneid, AUC

  49. The Algorithm void Backtrack2(int k, int n) { do { NextValue(k,n); // Assign to x[k] // a legal value if( !x[k] ) break ; // No possible value if (k == n) output vector x[1:n]; else Backtrack2(k+1, n); } while(1); } Prof. Amr Goneid, AUC

  50. 12. Hamiltonian Circuit Problemby Backtracking The Knight’s Tour problem is a Hamiltonian circuit problem Prof. Amr Goneid, AUC

More Related