Generating and Testing Combinations: Optimizing Search for Maximum Values and Solutions

Generate and Test Framework Nattee Niparnan

Optimization Example: Finding Max Value in an Array • There are N possible answers • The first element • The second element • 3rd, 4th … • Try all of them • Remember the best one

Generate and Test Framework • Define the set of admissible solution • Generate all of them (generating) • For each generated solution • Test whether it is the one we want • By the “evaluation” function (testing) • for optimization: remember the best one so far • For decision: report when we found the “correct” one

Combination and Permutation • In many case, the set of the admissible solutions is a set of “combination” or “permutation” of something • We need to knows how to generate all permutations and combinations

Generating all possible answer

Combination • Given N things • Generate all possible selections of K things from N things • Ex. N = 3, k = 2

Combination with replacement • Given N things • Generate all possible selections of K things from N things • When something is selected, we are permit to select that things again (we replace the selected thing in the pool) • Ex. N = 3, k = 2

Breaking the Padlock

Breaking the Padlock • Assuming we have four rings • Assuming each ring has following mark •    • We try •  •  • …. •   •  Backtracking Undone the second step, switch to another value

Key Idea • A problem consists of several similar steps • Choosing a things from the pool • We need to remember the things we done so far

General Framework Initial Step 1. Get a step that is not complete Engine Storage Step that have been done 2. Try any possible next step 3. Store each newly generated next step

Combination • Generate all combinations of lock key • Represent key by int • =0 • =1 • =2 • =3 • Step = sequence of selected symbols • E.g., • ’01’    • ‘0003’     

General Search • Storage s • S  ‘’ • While s is not empty • Curr  Storage.get • If Curr is the last step • evaluate • Else • Generate all next step from Curr • Push them to S If length(curr) == 4 For all symbol i New = curr+I Storage.push(new)

Search Space • Set of all admissible solutions • E.g., Combination Padlock • Search space = 0000  4444 • For a task with distinct steps it is easy to enumerate the search space with “backtracking” • for most of the case, we can do “backtracking”

Backtracking • A tool to enumerate the search space • Usually using the “stack” to implement the storage • Employ the processor stack • i.e., using the “recursive” paradigm

Combination Example by Backtracking void backtracking(intstep,int*sol) { if (step < num_step) { for (inti= 0; i < num_symbol; i++) { sol[step] = i; backtracking(step + 1,sol); } } else { check(sol); } } • The process automatically remember the step

Search Tree • Sols in each step • 0 • 00 • 000 • 0000  check • 000 • 0001  check • 000 • 0002  check • 000 • 0003  check • 000 • 00 • 001 • 0010  check

Search Tree 0 1 2 3 … 01 02 03 04 … … … … … … … … … … … …

8-queen • Given a chess board 8 queens

8-queens problem • Try to place the queens so that they don’t get in the others’ ways

Solving the Problem • Define the search space • What is the search space of this problem? • How large it is? • Choose an appropriate representation

Example • Every possible placement of queens • Size: 648 • Representation: a set of queens position • E.g., (1,1) (1,2) (2,5) (4,1) (1,2) (3,4) (8,8) (7,6) • This includes overlapping placement!!!

Example • Another representation • Try to exclude overlapping • Use combination without replacement • This is a combination • Selecting 8 positions out of 64 positions • Size: (64)! / (64 – 8)! * 8! • Implementation: in the “generating next step”, check for overlapping

Combination without replacement • We go over all position • For each position, we either “choose” or “skip” that position for the queen

Combination without replacement Mark_on_board is a binary string indicate that whether the position is selected void e_queen(intstep,int *mark_on_board) { if (step < 64) { mark_on_board[step] = 0; e_queen(step + 1,mark_on_board); mark_on_board[step] = 1; e_queen(step + 1,mark_on_board); } else { check(mark_on_board); } } Also has to check whether we mark exactly 8 spots

Combination without replacement • The generated mark_on_board includes • 000000000000  select no position (obviously not the answer) • 111111000000  select 6 positions (obviously not the answer) • We must limit our selection to be exactly 4

Combination without replacement Number of possible 0 void e_queen(intstep,int *mark_on_board,int chosen) { if (step < 64) { if ((64 – 8) – (step – chosen) > 0) { mark_on_board[step] = 0; e_queen(step + 1,mark_on_board,chosen); } if (8 - chosen > 0) { mark_on_board[step] = 1; e_queen(step + 1,mark_on_board,chosen+1); } } else { check(mark_on_board); } } Number of possible 1

Example • Any better way? • For each row, there should be only one queen • Size: 88 • Representation: sequence of columns • E.g., (1,2,3,4,5,6,7,8)

Using Backtracking? • The problem consists of 8 step • Placing each queen • We never sure whether the queen we place would lead to a solution • Backtracking is an appropriate way

Solving the Problem: Enumerating Search Space • There are eight possible ways in each step • There are eight steps • Very similar to the combination problem void e_queen(intstep,int *queen_pos) { if (step < 8) { for (inti = 0; i < 8; i++) { queen_pos[step] = i; e_queen(step + 1, queen_pos); } } else { check(queen_pos); } }

8-queen by permutation • Queen should not be in the same column • The solution should never have any column repeated • E.g., • (1,2,3,4,5,6,7,1) is bad (column collision • (1,1,3,4,5,6,7,5) is bad as well…. • (1,2,3,4,5,6,7,8) is good • There should be no duplicate column index!!!

Permutation • Given N symbols • A permutation is the element arrange in any order • E.g., 1 2 3 4 • Shows • 1 2 3 4 • 1 2 4 3 • 1 3 2 4 • 1 3 4 2 • … • 4 3 2 1 • For each step, we have to known which one is used

Permutation by Backtracking • The problem consists of several similar steps • Special condition • Symbols never repeat • How to do? • Easy way: • Generate all combination (as done before) • Check for ones that symbols do not repeat • Better way: • Remember what symbols are used

Permutation void backtracking(intstep,int *sol) { if (step < num_step) { for (inti = 0; i < num_symbol; i++) { if not_used(sol,i,step) { sol[step] = i; backtracking(step,sol); } } } else { check(sol); } } Boolnot_used(int *sol,intvalue,int step) { for (inti = 0;i < step; i++) { if (sol[i] == value) return false; } return true; }

Permutation • More proper ways • void backtracking(intstep,int*sol,bool *used) • { • if (step < num_step) { • for (inti= 0; i < num_symbol; i++) { • if (!used[i]) { • used[i] = true; • sol[step]= i; • backtracking(step,sol,used); • used[i] = false; • } • } • } else { • check(sol); • } • }

Backtracking Problems • Given N • Find any sequence of (a1,a2,a3,…) such that • a1 +a2 + a3 +… + ak = N • ai > 0 • ai<= aj for all i < j • aiis an integer

Example • N = 4 • 1 + 1 + 1 + 1 • 1 + 1 + 2 • 1 + 3 • 2 + 2 • 4

Solving with Backtracking • Representation • Array of ai • Step • Choosing the next value for ai

Branch & Bound Technique to reduce enumeration

Main Idea • We should not enumerate solution that will never produce a solution • We have done that!!! • 8-queens • By naïve combination, we will have to do all 648 • But, by each improvement, we further reduce what we have to do

Permutation 0 1 2 00 01 02 10 11 12 20 21 22 000 001 002 010 011 012 020 021 022 100 101 102 110 111 112 120 121 122 200 201 202 210 211 212 220 221 222

Key • If we know, at any step, that the solution is not feasible • Then, it is futile to further search along that path

Optimization Problem • All previous examples are decision problems • Asking whether the solution satisfy the criteria • Now, we consider broader set of problem, the optimization problem • Example • For all students, find one with maximum height

Evaluation • Representation: x = id of student • “goodness evaluation” evaluate(x) • For all x in the search space, we have to find one with maximum evaluate(x)

Solving Optimization Problem • Enumerate all possible solution • Calculate its value • Remember the max void backtracking(int step,int *sol) { if (step < num_step) { for (int i = 0; i < num_symbol; i++) { sol[step] = i; backtracking(step,sol); } } else { value = evaluate(sol); if (value > max) remember(value,sol); } }

Branch & Bound in Optimization Problem • For many problems, it is possible to assert its goodness even the solution is not complete • If we can predict the best value for the remaining step, then we can use that value to “bound” our search

Example • Assuming that we have 10 steps • At step 7, the goodness of the partial solution is X • Assuming that we know that the remaining step could not produce a solution better than Y • If we have found a solution better than X+Y • We can simply “bound” the search

Keys • We must know the so-called “upper bound” of the remaining step • It should be compute easily

Example Let value at this point be 10 If we know that this path never bet higher than 13 (which make 10 + 13 < 35) We can neglect it 23 35 2

Generating and Testing Combinations: Optimizing Search for Maximum Values and Solutions