Genetic Programming

Genetic Programming Ranga Rodrigo March 17, 2014

Introduction • Viewed by many researchers as a specialization of GA. • Where GAs use string (or vector) representations, GP uses a tree representation. • Originally, GP was developed by Koza to evolve computer programs. • For each generation, each evolved program (individual) is executed to measure its performance within the problem domain. • The result obtained from the evolved computer program quantifies the fitness of that program.

Implications of Tree-Based Representation • Adaptive individuals: • In Gas the size of individuals are fixed, • A GP population will usually have individuals of different size, shape and complexity. • Size: tree depth • Shape: branching factor of nodes in the tree. • The size and shape of a specific individual too may change due to application of the reproduction operators. • Domain-specific grammar • A grammar needs to be defined that accurately reflects the problem to be solved. • It should be possible to represent any possible solution using the defined grammar.

Grammar for Chromosome Representation • A grammar forms an important part of chromosome representation. • As part of the grammar, a terminal set, function set, and semantic rules need to be defined. • The terminal set specifies all the variables and constants • The function set contains all the functions that can be applied to the elements of the terminal set. • These functions may include mathematical, arithmetic and/or Boolean functions. Decision structures such as if-then-else and loops can also be included in the function set.

Grammar in Tree Terminology • Using tree terminology, elements of the terminal set form the leaf • nodes of the evolved tree, and elements of the function set form the non-leaf nodes. • For a specific problem, the search space consists of the set of all possible trees that • can be constructed using the defined grammar.

Example: XOR OR AND AND NOT NOT x1 x2 (x1AND NOT x2) OR (NOTx1ANDx2) x2 x1 Function set: {AND, OR, NOT} Terminal set: {x1, x2}

Example: Mathematical Expression + * - Y = x*ln(a)+sin(z)/exp(-x) – 3.5 / ln x 3.5 Function set: {-, +, *, /, sin, exp, ln} Terminal set: {a, x, z, 3.5} sin exp a z - x

Initial Population • The initial population is generated randomly within the restrictions of • a maximum depth and • semantics as expressed by the given grammar. • The branching factor of the root, and each non-terminal node are determined by the arityof the selected function. • For each non-root node, the initialization algorithm randomly selects an element either from the terminal set or the function set. • When an element from the terminal set is selected, the particular node becomes a leaf node.

Fitness Function • Because individuals usually represent a program, calculation of fitness requires the program to be evaluated against a number of test cases. • E.g.: • For the Boolean expression, fitness is calculated as the number of correctly predicted target outputs. • For the mathematical expression a data set of sample input patterns and associated target output is needed.

Crossover Operators • Generating one offspring: • A random node is selected within each of the parents. Crossover then proceeds by replacing the corresponding subtree in the one parent by that of the other parent. • Generating two offspring: • A random node is selected in each of the two parents. • The corresponding subtrees are swapped to create the two offspring

One Offspring

Two Offspring

Mutation Operators • Function node mutation: • A non-terminal node, or function node, is randomly selected and replaced with a node of the same arity, randomly selected from the function set. • Terminal node mutation: • A leaf node, or terminal node, is randomly selected and replaced with a new terminal node, also randomly selected from the terminal set. • Swap mutation: • A function node is randomly selected and the arguments of that node are swapped

Mutation Operators • Grow mutation: • With grow mutation a node is randomly selected and replaced by a randomly generated subtree. The new subtree is restricted by a predetermined depth. • Gaussian mutation: • A terminal node that represents a constant is randomly selected and mutated by adding a Gaussian random value to that constant. • Trunc mutation: • A function node is randomly selected and replaced by a random terminal node. This mutation operator performs a pruning of the tree.

Functional Node Mutation

Terminal Node Mutation

Swap Mutation

Grow Mutation

Gaussian Mutation

Trunc Mutation

Advantages of GP • GP does not impose any fixed length of the solution. In principle, the maximal length can be extended up to the hardware limits. • GP does not require as much knowledge about the problem and the possible solutions as do GAs. • GPs can theoretically evolve any series of actions a computer can possibly do, provided that we give the GP algorithm a set of commands to choose from that can describe all possible actions. Slide from Erdem KOÇ & Ömer UZEL

Advantages of GA • They efficiently search the model space, so they are more likely to converge toward aglobal minima. • There is no need of linearization of the problem. • There is no need to compute partial derivatives. • More probable models are sampled more frequently than less probable ones. Slide from Erdem KOÇ & Ömer UZEL

Application Fields of GP • Symbolic Regression and Function Synthesis (Performing symbolic regression) • Grammar Induction • Data Mining and Data Analysis(Data mining and classification) • Electrical Engineering and Circuit Design (Designing analog circuits) • Medicine • Economics and Finance • Geometry and Physics • Cellular Automata and Finite State Machines • Automated Programming • Robotics • Networking and Communication • Evolving Behaviors for Agents or Game Players • Pattern Recognition • Biochemistry • Machine Learning • Discovery of quantum algorithms • Image processing • Creating security protocols Slide from Erdem KOÇ & Ömer UZEL

Genetic Programming