1 / 86

A new crossover technique in Genetic Programming

A new crossover technique in Genetic Programming. Janet Clegg Intelligent Systems Group Electronics Department. This presentation. Describe basic evolutionary optimisation Overview of failed attempts at crossover methods Describe the new crossover technique

kioko
Download Presentation

A new crossover technique in Genetic Programming

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A new crossover technique in Genetic Programming Janet Clegg Intelligent Systems Group Electronics Department

  2. This presentation Describe basic evolutionary optimisation Overview of failed attempts at crossover methods Describe the new crossover technique Results from testing on two regression problems

  3. Evolutionary optimisation

  4. Start by choosing a set of random trial solutions (population)

  5. Each trial solution is evaluated (fitness / cost) 1.2 0.9 0.8 1.4 0.2 0.3 2.1 2.3 0.3 0.9 1.3 1.0 0.8 2.4 0.4 1.7 1.5 0.6 1.3 0.8 2.5 0.7 1.4 2.1

  6. Parent selection Select mother 2.1 0.9 2.1 1.3 0.7 Select father 1.7 1.4 0.6 1.7 1.5

  7. Perform crossover child 1 child 2

  8. Mutation Probability of mutation small (say 0.1)

  9. This provides a new population of solutions – the next generation Repeat generation after generation 1 select parents 2 perform crossover 3 mutate until converged

  10. Two types of evolutionary optimisation Genetic Algorithms (GA) – Optimises some quantity by varying parameter values which have numerical values Genetic Programming (GP) – Optimises some quantity by varying parameters which are functions / parts of computer code / circuit components

  11. Representation

  12. Representation Traditional GA’s - binary representation e.g. 1011100001111 Floating point GA – performs better than binary e.g. 7.2674554 Genetic Program (GP) Nodes represent functions whose inputs are below the branches attached to the node

  13. Some crossover methods

  14. Crossover in a binary GA Mother 1 0 0 0 0 0 1 0 = 130 Father 0 1 1 1 1 0 1 0 = 122 Child 1 1 1 1 1 1 0 1 0 = 250 Child 2 0 0 0 0 0 0 1 0 = 2

  15. Crossover in a floating point GA Mother father Min parameter value Max parameter value Offspring chosen as random point between mother and father

  16. Traditional method of crossover in a GP mother father Child 2 Child 1

  17. Motivation for this work Tree crossover in a GP does not always perform well Angeline and Luke and Spencer compared:- (1) performance of tree crossover (2) simple mutation of the branches difference in performance was statistically insignificant Consequently some people implement GP’s with no crossover - mutation only

  18. Motivation for this work In a GP many people do not use crossover so mutation is the more important operator In a GA the crossover operator contributes a great deal to its performance - mutation is a secondary operator AIM:- find a crossover technique in GP which works as well as the crossover in a GA

  19. Cartesian Genetic Programming

  20. Cartesian Genetic Programming (CGP) Julian Miller introduced CGP Replaces tree representation with directed graphs – represented by a string of integers The CGP representation will be explained within the first test problem CGP uses mutation only – no crossover

  21. First simple test problem A simple regression problem:- Finding the function to best fit data taken from The GP should find this exact function as the optimal fit

  22. The traditional GP method for this problem Set of functions and terminals

  23. * x x x 1 - * Initial population created by randomly choosing functions and terminals within the tree structures (1-x) * (x*x)

  24. Crossover by randomly swapping sub-branches of the parent trees mother father Child 2 Child 1

  25. CGP representation Set of functions – each function identified by an integer Set of terminals – each identified by an integer

  26. Creating the initial population 2 0 1 0 1 1 2 3 First integer is random choice of function 0 (+), 1 (-), or 2 (*) Second two integers are random choice of terminals 0 (1) or 1 (x) Next integers are random choice of inputs for the function from the set 0 (1) 1 (x) or node 2

  27. Creating the initial population 2 0 1 0 1 1 1 3 1 0 2 3 2 4 1 5 output 4 5 6 2 3 random choice of inputs from 0 1 2 3 4 5 Terminals nodes random choice for output from 0 1 2 3 4 5 6 Terminals all nodes

  28. + x x x 1 * + 2 0 1 0 1 1 1 3 1 0 2 3 2 4 1 5 2 3 4 5 6 output output 5 2 3 (1*x) +(x+x) = 3x

  29. Run the CGP with test data taken from the function Population size 30 28 offspring created at each generation Mutation only to begin with Fitness is the sum of squared differences between data and function

  30. 5 + x 1 x + * + 1 x 3 2 2 Result 0 0 1 2 2 1 1 2 2 0 3 2 0 5 1 5 2 3 4 5 6 output =

  31. Statistical analysis of GP Any two runs of a GP (or GA) will not be exactly the same To analyse the convergence of the GP we need lots of runs All the following graphs depict the average convergence out of 4000 runs

  32. Introduce crossover

  33. Introducing some Crossover Pick random crossover point Parent 1 0 0 1 2 2 1 1 2 2 0 3 2 0 5 1 5 Parent 2 2 0 1 0 1 1 1 3 1 0 2 3 2 4 1 5 Child 1 0 0 1 2 2 1 1 3 1 0 2 3 2 4 1 5 Child 2 2 0 1 0 1 1 1 2 2 0 3 2 0 5 1 5

  34. GP with and without crossover

  35. GA with and without crossover

  36. Random crossover point but must be between the nodes Parent 1 0 0 1 2 2 1 1 2 2 0 3 2 0 5 1 5 Parent 2 2 0 1 0 1 1 1 3 1 0 2 3 2 4 1 5 Child 1 0 0 1 2 2 1 13 1 0 2 3 2 4 1 5 Child 2 2 0 1 0 1 1 12 2 0 3 2 0 5 1 5

  37. x x * x - * x + + 1 1 x - x + 1 + 0 0 1 1 1 22 3 2 2 4 1 0 3 5 6 2 3 4 5 6 output 6 3 5 4 2 3 2 2

  38. GP crossover at nodes

  39. Pick a random node along the string and swap this single node Parent 1 0 0 1 2 2 1 1 2 2 0 3 2 0 5 1 5 Parent 2 2 0 1 0 1 1 1 3 1 0 2 3 2 4 1 5 Child 1 0 0 1 2 2 1 1 3 1 0 2 3 2 4 1 5 Child 2 2 0 1 0 1 1 1 2 2 0 3 2 0 5 1 5

  40. Crossover only one node

  41. Each integer in child randomly takes value from either mother or father Parent 1 0 0 1 2 2 1 1 2 2 0 3 2 0 5 1 5 Parent 2 2 0 1 0 1 1 1 3 1 0 2 3 2 4 1 5 Child 1 2 0 1 0 2 1 1 21 0 2 2 251 5 Child 2 0 0 1 2 1 1 1 32 0 3 3 041 5

  42. Random swap crossover

  43. Comparison with random search GP with no crossover performs better than any of the trial crossover here How much better than a completely random search is it? The only means it will improve on a random search are by parent selection mutation

  44. Comparison with a random search

  45. Comparison with a random search GP converges in 58 generations Random search 73 generations

  46. GA performance compared with a completely random search GA tested on a large problem – A random search would have involved searching through 150,000,000 data points The GA reached the solution after testing 27,000 data points ( average convergence of 5000 GA runs) Probability of random search reaching solution in 27,000 trials is 0.00018 !!!!

  47. Why does GP tree crossover not always work well?

  48. f1 x6 f2 f3 f5 f4 x8 x7 x4 x3 x2 x1 f6 f7 x5 f1 { f2 [ f4( x1,x2 ), f5( x3,x4 ) ] , f3 [ f6( x5,x6 ), f7( x7,x8 ) ]} g1 { g2 [ g4( y1,y2 ), g5( y3,y4 ) ] , g3 [ g6( y5,y6 ), g7( y7,y8 ) ] } f1 { f2 [ f4( x1,x2 ), f5( x3,x4 ) ] , f3 [ f6( x5,x6 ), g7( y7,y8 )]}

  49. g( x1) = f( x2) g f Good! x1 x2

  50. g( x1) f( x2) g( x2) f( x1) g g( x2) f Good! x1 x2 f( x1)

More Related