Lecture 6

1. Lecture 6 Some Applications. I: The Traveling Salesman Problem. This is a “typical” NP-Complete problem, with no known (expected?) polynomial-time solution. By 1990, problems in VLSI fabrication were asking for good solutions in the case of 1.2 million “cities”. Since the only exact solution known is of the type “generate all permutations on N elements , compute the cost of the tour corresponding to the permutation, update the known smallest tour and repeat”, the computational cost is O(N!). It would appear that, at the moment, exact solutions are feasible for problems around 100 (or slightly more) cities; it would also appear that problems involving 10,000+ cities are still on the “far side” for evolutionary based optimization approaches. Our discussion is based on Michalewicz and Fogel’s How to Solve it, Ch. 8.

2. Lecture 6 Note: one of the reasons for choosing this problem is that it is well-known and is non-trivial. Much work has been done on it, with many clever variants: it should be a good problem to stimulate ideas in other areas. Note: A survey article on combinatorial optimization approaches to the the problem is [JohnsonMcGeoch1997]. A follow-up ([Walshaw2001]) provides some recent variants. Essentially, the best algorithm for obtaining approximate solutions is based on one invented by Lin and Kernighan in 1973. The original Lin-Kernighan algorithm (the 1973 paper does not appear to be available on-line) was the best algorithm until 1989, and has been superseded by some more efficient variants. No genetic algorithm approaches seem to come close to it, at this point.

3. Lecture 6 Test Cases: in trying to determine the efficiency (and accuracy) of a proposed approximation algorithm, it is necessary to set up benchmarks. There are two generally accepted ways of setting up test cases: • The cities are distributed at random in the Euclidean plane, according to some uniform random variable. Euclidean distance is assumed. One of the reasons for this is that an empirical formula for the expected length of the minimal tour exists: L* = k•sqrt(N•R), where N is the number of cities, R is the area of the square box containing them and k is an empirical formula - its suggested (from various considerations) value is k = 0.749. • Publicly available test cases (TSPLIB), with documented optimal or best-known solutions.

4. Lecture 6 Variation Operators. For TSP, the evaluation function is trivial: add up the lengths of the edges in a given tour. Fitness is thus easy to compute, and fitness proportionate reproduction is easy to set up. Several problems arise in the decision of which operators to use to provide the next generation. • Do we use binary or integer representation for the “names” of the cities? Binary representation requires N•ceil(log2(N)) bits and both mutation and crossover applied at the bit level are likely to lead to non-tours or, even worse, to indices that do not correspond to any city. A reasonable choice is to pick an integer representation. Even in this case, an array-of-integers representation for the chromosomes is very likely to introduce non-tours under both mutation and crossover, requiring either re-tries or “patching up” operators.

5. Lecture 6 2. Do we attempt to invent variants of whatever operators we want to introduce so that they repair errors (after all, error repair is a fairly common event in real DNA), or do we look for different representations for our chromosomes, with operators that do not introduce errors? 3. How can we guarantee that our changes in representation and operators will still provide a search over the whole search space (or that our repair techniques will not introduce unacceptable biases in our “search populations”)?

6. Lecture 6 A reasonable choice (it has been the historical choice) attempts to introduce appropriate data structures and appropriate operators, trying to “repair” as little as possible. The operant mantra is: lots of repair = wrong data type. We will use the examples in [M&F], with 9 cities, numbered from 1 to 9. We start with a series of node-based operators; we will look at edge-based ones later.

7. Lecture 6 Adjacency Representation. Encode a tour as a list of cities. City j is listed in position i iff the tour leads from city i to city j. The vector (2 4 8 3 9 7 1 5 6) ˛ 1 ® 2® 4 ® 3 ® 8 ® 5 ® 9 ® 6 ® 7. This may be slightly counterintuitive, but a bit of practice will convince you that it works as an unambiguous way of representing tours in term of ordered lists or arrays. It should be clear that not all such vectors can represent tours: (2 4 8 1 9 3 5 7 6) has the disjoint cycles 1 ® 2 ® 4 ® 1 and 3 ® 8 ® 7 ® 5 ® 9 ® 6 ® 3. This representation does not support simple “cut-and-slice” crossover operators (as well as not supporting point mutations). We can choose to “repair” or we can choose to modify the operators so they leave us with legal members of the population (= tours).

8. Lecture 6 J. Grefenstette published in 1985 (paper apparently not available on-line) several potential operators, and ran some experiments. Alternating Edges Crossover. Randomly choose an edge from the first parent; select an appropriate edge from the second parent; then select from first parent; etc. If the new edge from a parent introduces a cycle into the partial tour being constructed, select a random edge from the remaining ones that does not introduce a cycle. p1 = (2 3 8 7 9 1 4 5 6) p2 = (7 5 1 6 9 2 8 4 3) might lead to the partial offspring: s1 = (2 5 8 7 9 1 ? ? ?) since up to this point the picking of alternate edges gives no cycle.

9. Lecture 6 At this point, the choice of 8 from the second parent would introduce the edge 7 ® 8, but the edge endpoint 8 has already been picked. The still missing vertices are 3, 4 and 6. Choosing the edges 7 ® 3 and 7 ® 6, does not introduce any immediate cycles, while 7 ® 4 introduces the cycle 7 ® 4® 7. A potential partial offspring is: s1 = (2 5 8 7 9 1 3 ? ?) with the parents (p1 as potential donor) p1 = (2 3 8 7 9 1 4 5 6) p2 = (7 5 1 6 9 2 8 4 3) 5 has already been chosen; 4 and 6 remain. The choice 8 ® 4 introduces the cycle 8 ® 4® 7 ® 3® 8, so choose 8 ® 6, which forces 9® 4: s1 = (2 5 8 7 9 1 3 6 4). The offspring (2 5 8 7 9 1 6 4 3) is also possible - as well as others.

10. Lecture 6 Question: what is the worst cost of this crossover scheme? The expected cost? It is clear that we may have to perform several checks on long paths before we can determine whether a proposed increment of a partial tour is still a useful partial tour. Subtour-chunks Crossover. Choose a random length subtour from one parent, another random length subtour from the other. Extend the tour by choosing edges alternating between parents. Pick random non-cycle-producing edges if the next choice would introduce a cycle. This is clearly quite similar to the previous crossover scheme.

11. Lecture 6 Heuristic Crossover. Choose a random city to start the tour. For each of the parents, check which edge emanating from that city is shorter. Take it. That will give you the next city. Repeat with the parents. If both parents provide edges introducing a cycle, pick a random city (and edge) not introducing a cycle. Modification: 1) if the shorter edge from a parent introduces a cycle, check the longer before “going random”; 2) if you need to “go random” select the shortest edge from a pool of randomly selected q (parameter) edges (similar to tournament selection). To improve local optimization, another operator was introduced (this has a long history in its own right): randomly select two edges (i, j) and (k, m) and check if |(i, j)| + |(k, m)| > |(i, m)| + |(k, j)|. If true, replace (i, j) and (k, m) by (i, m) and (k, j).

12. Lecture 6 None of the three crossover operators developed in this context has shown outstanding performance. They all suffer from the creation of too much disruption: it would seem that the Adjacency List representation, although good for thinking in terms of schemata that “fix” good edges and attempt to construct around them, is somehow not very good for the problem, because it does not really maintain good schemata.

13. Lecture 6 Ordinal Representation. Assume the existence of a “reference list” that will be used as a starting point for all other lists. This list could be C = (1 2 3 4 5 6 7 8 9), although it does not have to be. A tour such as 1 ® 2® 4 ® 3 ® 8 ® 5 ® 9 ® 6 ® 7 can be represented by the list l = (1 1 2 1 4 1 3 1 1)… How? The first number on l is 1, so take the first city on C as the first city of the tour, and remove it from C. The resulting partial tour is 1. the second number on l is also a 1, so pick the first element on the current version of C, which is 2. Remove from C; partial tour 1 ® 2. The next item on l is 2, corresponding to the 4 on C. Remove the 4 from C. Partial tour 1 ® 2 ® 4. Continue until all elements of l have been accounted for.

14. Lecture 6 Advantage: splicing works! Example: the splice point is |. Parents: p1 = (1 1 2 1 | 4 1 3 1 1) [1 ® 2® 4 ® 3 ® 8 ® 5 ® 9 ® 6 ® 7] p2 = (5 1 5 5 | 5 3 3 2 1) [5 ® 1® 7 ® 8 ® 9 ® 4 ® 6 ® 3 ® 2] Offspring: s1 = (1 1 2 1 | 5 3 3 2 1) [1 ® 2® 4 ® 3 ® 9 ® 7 ® 8 ® 6 ® 5] s2 = (5 1 5 5 | 4 1 3 1 1) [5 ® 1® 7 ® 8 ® 6 ® 2 ® 9 ® 3 ® 4] Disadvantage: only the first part of the tour survives, the second becoming, essentially, random. There is too little inheritance, and the experimental results bear this out.

15. Lecture 6 Path Representation. This is what one would expect: the path 5 ® 1® 7 ® 8 ® 9 ® 4 ® 6 ® 2 ® 3 is represented as the list (5 1 7 8 9 4 6 2 3). It should be clear that this particular tour can be represented via 9 equivalent lists (rotate left or right). Crossover Operators. • Partially Mapped (PMX) crossover: choose a subsequence of a tour from one parent and preserve the order and position of as many ciries as possible from the other parent. As an example, consider the parents with cut-points indicated by |: p1 = (1 2 3 | 5 4 6 7 | 8 9) p2 = (4 5 2 | 1 8 7 6 | 9 3) The offspring would be generated as follows:

16. Lecture 6 s1 = (x x x | 1 8 7 6 | x x) s2 = (x x x | 4 5 6 7 | x x) This swap also defines a mapping: 1 « 4, 8 « 5, 7 « 6, 6 « 7. Next, fill additional cities from the parents that don’t lead to conflict. p1 = (1 2 3 | 5 4 6 7 | 8 9) p2 = (4 5 2 | 1 8 7 6 | 9 3) The easy ones: s1 = (x 2 3 | 1 8 7 6 | x 9) s2 = (x x 2 | 4 5 6 7 | 9 3) For the harder ones, use the mapping: s1 = (4 2 3 | 1 8 7 6 | 5 9) s2 = (1 8 2 | 4 5 6 7 | 9 3).

17. Lecture 6 2) Order (OX) crossover: choose a a subsequence of a tour from one parent and preserve the relative order of the cities from the other. Example: p1 = (1 2 3 | 5 4 6 7 | 8 9) p2 = (4 5 2 | 1 8 7 6 | 9 3) s1 = (x x x | 5 4 6 7 | x x) s2 = (x x x | 1 8 7 6 | x x) The tour in p2, starting from its second cut point, is 9 ® 3® 4 ® 5 ® 2 ® 1 ® 8 ® 7 ® 6. Remove the cities already in s1, obtaining the partial tour 9 ® 3® 2 ® 1 ® 8. Insert this partial tour after the second cut point of s1, obtaining the tour s1 = (2 1 8 | 5 4 6 7 | 9 3 ). Similarly s2 = (3 4 5 | 1 8 7 6 | 9 2).

18. Lecture 6 3) Cycle (CX) crossover: each city and its position comes from one of the parents. Example: p1 = (1 2 3 4 5 6 7 8 9) p2 = (4 1 2 8 7 6 9 3 5) Start by taking the first city from p1: s1 = (1 x x x x x x x x) The next city must be from p2, and from the same position. This gives city 4, which is in position 4 on p1: s1 = (1 x x 4 x x x x x). In p2, in the same position as 4 in p1, we have city 8: s1 = (1 x x 4 x x x 8 x). We continue with s1 = (1 x 3 4 x x x 8 x), s1 = (1 2 3 4 x x x 8 x). Note that the selection of 2 now forces the selection of 1 and we have a cycle in our scheme. We now use the second parent to fill in: s1 = (1 2 3 4 7 6 9 8 5).

19. Lecture 6 If we now start form p2, p1 = (1 2 3 4 5 6 7 8 9) p2 = (4 1 2 8 7 6 9 3 5) s1 = (4 1 2 8 x x x 3 x) completes the first “cycle”. Filling in from p1: s1 = (4 1 2 8 5 6 7 3 9) . Several other path-based operators have been tried, and one may wish to look in the literature for more.

20. Lecture 6 Other Reordering Operators. • Inversion. Select two points along the permutation, cut it at these points and re-insert the reversed string. (1 2 | 3 4 5 6 | 7 8 9) ® (1 2 | 6 5 4 3 | 7 8 9). 2. Insertion. Select a city and insert it in a random place. 3. Displacement. Select a subtour and insert it in a random place. 4. Reciprocal Exchange. Swap two cities. 5. Heuristic Crossover with multiple (> 2) parents. Just as Heuristic Crossover but with as many choices for the next city as there are parents - pick the shortest edge out…

21. Lecture 6 Edge-based Operators. Various people tried to introduce operators that would make better use of edge information. Grefensteette introduced a class of heuristic operators along the following lines: • Randomly select a city to be the current city c of the offspring. • Select 4 edges (2 from each parent) incident to c. • Define a probability distribution over the selected edges based on their cost. Edges incident on a previously visited city have probability 0. • If at least one edge has positive probability, select probabilistically; otherwise select at random to reach an unvisited city. • City at other end of edge is new c. • If tour is complete, stop; if not goto 2.

22. Lecture 6 A number of experiments indicate that only about 60% of the edges are transferred from the parents; 40% are random. Too much randomness to be effective in building more efficient solutions. Edge Recombination: require that the number of edges inherited from the parents be as large as possible. Start from the idea that, in a tour (3 1 2 8 7 4 6 9 5) the edges are (3 1), (1 2), (2 8), (8 7), (7 4), (4 6), (6 9), (9 5), (5 3). Recall that the edges are undirected: (5 3) = (3 5); direction is not important. Position of a city on a tour is not important: tours are circular. Evaluation Function: minimize the total cost of the edges that constitute a legal tour.

23. Lecture 6 Example. Start with two parents: p1 = (1 2 3 4 5 6 7 8 9), p2 = (4 1 2 8 7 6 9 3 5). Using both parents, collect the edges available: City 1: (1 9), (1 2), (1 4); City 2: (2 1), (2 3), (2 8); City 3: (3 2), (3 4), (3 9), (3 5); City 4: (4 3), (4 5), (4 1); City 5: (5 4), (5 6), (5 3); City 6: (6 5), (6 7), (6 9); City 7: (7 6), (7 8); City 8: (8 7), (8 9), (8 2); City 9: (9 8), ( 1), ( 6), (9 3).

24. Lecture 6 The algorithm. Start with either one of the “start cities” in the edge lists of the parents or with a city with the smallest number of edges - this latter criterion maximizes the probability that you will finish the tour using the parental set of edges. Once you have decided on the first city, add an edge to a city with the smallest number of edges. Continue. If we start with City 1: we can reach 2, 4, 9. 2 and 4 have 3 edges; 9 has 4. Pick, randomly, between 2 and 4. Say you picked 4. You now have (1 4 x x x x x x x). 4 has edges to 1, 3, 5. The edge to 1 has already been used; 5 has fewer edges than 3: (1 4 5 x x x x x x). Continuing in this fashion, we arrive at the offspring (1 4 5 6 7 8 2 3 9) - without needing to introduce a new edge to complete the tour. It appears (experimentally) that failure occurs in less than 1.5% of the cases.

25. Lecture 6 A variant attempts to maintain subtours common to both parents: note that, if a city has only two or three edges associated, one (or both) of the edges must be common to both parents. The algorithm “prefers” to choose edges common to both, before looking at any others. This seems to have led to better results. Finding further operators that improve on the know edge ones is an open question (or was as of 2000).

26. Lecture 6 Matrix Representations and Operators. There have been at least 3 attempts. • Precedence Matrix. A tour (3 1 2 8 7 4 6 9 5) is represented by the matrix: the element mij contains a 1 iff the city i occurs before the city j on the tour. Properties of the matrix: 1. The number of 1s is exactly n•(n - 1)/2. 2. mii = 0 for all 1 ≤ i ≤ n. 3. If mij = 1 and mjk = 1 then mik = 1.

27. Lecture 6 Claim: If the number of 1s is less than n•(n - 1)/2, with the other conditions still satisfied, the cities are partially ordered, which is another way of saying that the matrix can be completed in at least one way to obtain a legal tour. The operators devised in [Fox & McMahon, Genetic Operators for Sequencing Problems, in Foundations of Genetic Algorithms, G. J. E. Rawlins, ed., Morgan Kaufman, 1991] were the operators of intersection and union. Intersection. The intersection operator is based on the observation that the (bitwise) intersection of two tour matrices results in a matrix that satisfies conditions 2 and 3, with a number of 1s no greater than n•(n - 1)/2, and thus extendible to a tour

28. Lecture 6 The two parents p1 = (1 2 3 4 5 6 7 8 9) p2 = (4 1 2 8 7 6 9 3 5) correspond to the matrices below:

29. Lecture 6 The intersection is given by the matrix: And we have a partial order. For example, city 1 must precede cities 2, 3, 5, 6, 7, 8 and 9; city 6 is only required to precede city 9; etc.. How do we complete the tour?

30. Lecture 6 Select one of the parents; add some 1s unique to this parent; complete the matrix into a tour sequence through an analysis of the sums of the rows and columns. A possible completion is given by the matrix on the right below, which gives the tour (1 2 4 8 7 6 3 5 9).

31. Lecrture 6 The second operator is the union operator. It is based on the observation that subsets from two matrices can be safely combined provided the two subsets have empty intersection. It thus partitions the set of cities into two disjoint groups and copies the bits of one matrix for the first group, and the bits of the other for the second group. For example, p1 can lead to the set {1, 2, 3, 4}, p2 to the set {5, 6, 7, 8, 9}, with the “union” matrix at the right. It needs to be completed, using the same kind of techniques used for the completion of the intersection.

32. Lecture 6 2. Binary Tours. The matrix element mij contains a 1 iff the tour goes directly from city i to city j. There is only one nonzero entry in each row and column. Matrix (a) below represents the tour ( 1 2 4 3 8 6 5 7 9) (or any “rotation” left or right). Matrix (b) represents another tour (or does it?).

33. Lecture 6 The requirement that each row and column contain a single 1 allows for non-tours to be represented - matrix (b) on the previous slide represents the two subtours (1 2 4 5 7) and (3 8 6 9). An example of subtours that can then be connected to make up a full tour occurs below. The genetic algorithm will be restricted to subtours of length at least 3 - fixed into full tours after it terminates.

34. Lecture 6 Operators. Two were defined. The first took a matrix, randomly selected several rows and columns, removed the set bits at the intersections of those rows and columns and replaced them randomly. Ex.: matrix (a) corresponds to a tour. Assume rows 4, 6, 7, 9 and columns 1, 3, 5, 8, 9 are selected. The marginal sums are calculated and stored; the bits at the intersections are removed and replaced randomly, agreeing with the marginal sums. We have an example below:

35. Lecture 6 Replacing the old rows and column fragments with the new ones, we have a matrix that represents not a single tour but two subtours: (1 2 4 5 7) and (3 8 6 9) - the full matrix (8.6(b)) introduced earlier. The second operator starts with two parents and a matrix of 0s. It fills the new matrix with the result of the “intersection” of the parents: 1 bits in both result in a 1 bit in the offspring. After this initial phase, the operator copies alternately one set bit from each parent until no bits exist in either parent that can be copied without violating the matrix restrictions. If, at this point, the matrix still has some rows without a 1, they receive a 1 at random - again satisfying the matrix constraints.

36. Lecture 6 We start with matrix (a) below, representing the subtours (1 5 3 7 8) and (2 4 9 6). The second parent matrix (b) represents the full tour (1 5 6 2 7 8 3 4 9).

37. Lecture 6 The first result of the first phase of the operator application appears in (a) below; the second phase leads to (b). Notice that a number of rows (columns) in (b) have no 1: it would not be possible to fill from a parent without violating the conditions.

38. Lecture 6 Random filling (with the constraint of having just one 1 in each row and column) leads to the matrix below. It does represent a tour, (1 5 6 2 3 4 9 7 8). Another solution would lead to the two subtours (1 5 3 4 9) and (2 7 8 6).

39. Lecture 6 Binary Matrices with Crossover (XO) Operators. We define crossover operators by first choosing one or more “between column” positions, swapping the values in the blocks so identified - see the picture below. The resulting matrices are (generally) illegal, although they have the correct total number of 1s.

40. Lecture 6 We see the result in the picture below: some rows (columns) have no 1s, some have two. We need to repair these intermediate matrices. For example, the 1 in position m1,4 in (a) could be moved to position m1,8. After this type of “correction” we may end up with the first offspring providing the legal tour (1 2 8 4 3 6 5 7 9), while the second offspring provides the two subtours (1 6 5 7 2 8 9) and (3 4).

41. Lecture 6 The second step of the repair algorithm needs to be applied only to the second matrix: cut and connect subtours to produce a legal tour. You can use information about which edges exist in the parents to choose where to splice the subtours: for example, (2 4) is present in one of the parents, so we splice (3 4) and (4 3) between 2 and 8. A heuristic “inversion operator” was also introduced. The claim - based on a number of experiments - was that this was reasonable, with one 318-city experiment resulting in a tour within 0.6% of optimal. When the splicing of subtours is required, local considerations can be useful - and some systems incorporate them.

42. Lecture 6 A modified GA scheme would involve some local optimizations applied to all elements of the population.

43. Lecture 6 We will introduce some notions from [Papadimitriou & Steiglitz; Combinatorial Optimization] and then we’ll continue. They refer to the TSP. Definition. Let f and g denote tours. A k-change (also know as k-opt) neighborhhood of f is defined as Nk(f) = {g : g is a tour, obtained from f as follows: remove k edges from f and replace them with k edges}. Example for 2-change: 5 2 5 2 1 1 3 4 3 4 6 6 7 7

44. Lecture 6 It has been found that 2-change and 3-change (especially 3-change) lead to very effective heuristics for TSP. 4-change does not seem to be sufficiently effective to bother: the extra computational cost is not paid back in extra improvements leading to faster convergence to a better approximation [results mostly due to Lin]. The local optimization algorithm can be stated as follows: Let t be a tour. procedure local search begin t¬ some initial tour; while k-improve(t) ≠ “no” do t¬k-improve(t); return t end

45. Lecture 6 An algorithm was proposed: • Use a 2-opt procedure to replace each tour in the current population with a locally optimal tour. • Allow higher quality solutions to generate more offspring. • Use recombination and mutation. • Search for the minimum by using local search within each individual. • Repeat steps 2-4 until a termination condition is met. For reproduction, a variant of the order crossover (OX) was used.

46. Lecture 6 Two parents are given - with the cut points marked by |: p1 = (1 2 3 | 4 5 6 7 | 8 9) p2 = (4 5 2 | 1 8 7 6 | 9 3), and they produce offspring as follows. First, the segments between cut points are copied into the offspring: s1 = (x x x | 4 5 6 7 | x x) s2 = (x x x | 1 8 7 6 | x x). Next, instead of starting from the second cut point of one parent - as was the case for OX - the cities from the other parent are copied in the same order from the beginning of the string, omitting those symbols that are already present. This leads to the descendants: s1 = (2 1 8 | 4 5 6 7 | 9 3) s2 = (2 3 4 | 1 8 7 6 | 5 9).

47. Lecture 6 Edge Assembly Crossover (EAX). Assume two parents have been selected: [Y. Nagata & S. Kobayashi, Edge Assembly Crossover: a High-Power Genetic Algorithm for the TSP, in Proceedings of the Seventh International Conference on GAs, Morgan Kaufman, 1997]

48. Lecture 6 Construct a graph G that contains the edges of both parents (A and B). Then construct a set of AB-cycles: an AB-cycle is an even length subcycle of G with edges that come alternately from A and B. An AB-cycle can repeat cities but not edges. Construction: assume we start at city 6 and that the first edge of the AB-cycle is (6 1). This edge comes from parent A.

49. Lecture 6 We pick the next edge from parent B, originating from city 1. Select edge (1 3). You can further select edges (in order) (3 4) from A, (4 5) from B, (5 6) from A (which gives an odd cycle) and (6 3) from B. The latter choice gives us an even cycle, after dropping the first two edges. Remove the edges of the AB-cycle from G, and repeat. This may well lead to “ineffective” AB-cycles containing just two edges.

50. Lecture 6 The next step involves selecting a subset of AB-cycles (an E-set) to extend into a suitable tour. Two selection mechanisms were introduced: 1) a deterministic heuristic method; 2) a random method (each AB-cycle has probability 0.5). Once the E-set is constructed, we construct an intermediate offspring C: setC¬ A for each edge eÎE-set do ifeÎA then C ¬ C - {e} ifeÎB then C ¬ C  {e} C is a set of disjoint subtours that cover all cities. We now introduce a greedy procedure that incorporates local search to obtain a single legal tour.