1 / 20

CHAPTER 10 E VOLUTIONARY C OMPUTATION II : G ENERAL M ETHODS AND T HEORY

Slides for Introduction to Stochastic Search and Optimization ( ISSO ) by J. C. Spall. CHAPTER 10 E VOLUTIONARY C OMPUTATION II : G ENERAL M ETHODS AND T HEORY. Organization of chapter in ISSO Introduction Evolution strategy and evolutionary programming; comparisons with GAs

paul2
Download Presentation

CHAPTER 10 E VOLUTIONARY C OMPUTATION II : G ENERAL M ETHODS AND T HEORY

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Slides for Introduction to Stochastic Search and Optimization (ISSO)by J. C. Spall CHAPTER 10EVOLUTIONARYCOMPUTATIONII: GENERAL METHODS AND THEORY Organization of chapter in ISSO Introduction Evolution strategy and evolutionary programming; comparisons with GAs Schema theory for GAs What makes a problem hard? Convergence theory No free lunch theorems

  2. Methods of EC • Genetic algorithms (GAs), evolution strategy (ES), and evolutionary programming (EP) are most common EC methods • Many modern EC implementations borrow aspects from one or more EC methods • Generally: ES generally for function optimization; EP for AI applications such as automatic programming

  3. ES Algorithm with Noise-Free Loss Measurements Step 0 (initialization)Randomly or deterministically generate initial population of N values of  and evaluate L for each of the values. Step 1 (offspring)Generate  offspring from current population of N candidate  values such that all  values satisfy direct or indirect constraints on . Step 2 (selection)For (N+)-ES, select N best values from combined population of Noriginal values plus  offspring; for (N,)-ES, select N best values from population of  > N offspring only. Step 3 (repeat or terminate) Repeat steps 1 and 2 or terminate.

  4. Schema Theory for GAs • Key innovation in Holland (1975) is a form of theoretical foundation for GAs based on schemas • Represents first attempt at serious theoretical analysis • But not entirely successful, as “leap of faith” required to relate schema theory to actual convergence of GA • “GAs work by discovering, emphasizing, and recombining good ‘building blocks’ of solutions in a highly parallel fashion.” (Melanie Mitchell, An Introduction to Genetic Algorithms [p. 27], 1996, paraphrasing John Holland) • Statement above more intuitive than formal • Notion of building block is characterized via schemas • Schemas are propagated or destroyed according to the laws of probability

  5. Schema Theory for GAs • Schema is template for chromosomes in GAs • Example: [* 1 0 * * * * 1], where the * symbol represents a don’t care (or free) element • [11001101] is specific instance of this schema • Schemas sometimes called building blocks of GAs • Two fundamental results: Schema theorem and implicit parallelism • Schema theorem says that better templates dominate the population as generations proceed • Implicit parallelism says that GA processes >> N schemas at each iteration • Schema theory is controversial • Not connected to algorithm performance in same direct way as usual convergence theory for iterates of algorithm

  6. Convergence Theory via Markov Chains • Schema theory inadequate • Mathematics behind schema theory not fully rigorous • Unjustified claims about implications of schema theory • More rigorous convergence theory exists • Pertains to noise-free loss (fitness) measurements • Pertains to finite representation (e.g., bit coding or floating point representation on digital computer) • Convergence theory relies on Markov chains • Each state in chain represents possible population • Markov transition matrix P contains all information for Markov chain analysis

  7. GA Markov Chain Model • GAs with binary bit coding can be modeled as (discrete state) Markov chains • Recall states in chain represent possible populations • ith element of probability vector pk represents probability of achieving ith population at iteration k • Transition matrix: The i, j element of P represents the probability of population i producing population j through the selection, crossover and mutation operations • Depends on loss (fitness) function, selection method, and reproduction and mutation parameters • Given transition matrix P, it is known that

  8. Rudolph (1994) and Markov Chain Analysis for Canonical GA • Rudolph (1994, IEEE Trans. Neural Nets.) uses Markov chain analysis to study “canonical GA” (CGA) • CGA includes binary bit coding, crossover, mutation, and “roulette wheel” selection • CGA is focus of seminal book, Holland (1975) • CGA does not include elitismlack of elitism is critical aspect of theoretical analysis • CGA assumes mutation probability 0 < Pm < 1 and single-point crossover probability 0  Pc 1 • Key preliminary result: CGA is ergodic Markov chain: • Exists a unique limiting distribution for the states of chain • Nonzero probability of being in any state regardless of initial condition

  9. Rudolph (1994) and Markov Chain Analysis for CGA (cont’d) • Ergodicity for CGA provides a negative result on convergence in Rudolph (1994) • Let denote lowest of N (= population size) loss values within population at iteration k • represents loss value for  in population k that has maximum fitness value • Main theorem: CGA satisfies (above limit on left-hand side exists by ergodicity) • Implies CGA does not converge to the global optimum

  10. Rudolph (1994) and Markov Chain Analysis for CGA (cont’d) • Fundamental problem with CGA is that optimal solutions are found but then lost • CGA has no mechanism for retaining optimal solution • Rudolph discusses modification to CGA yielding positive convergence results • Appends “super individual” to each population • Super individual represents best chromosome so far • Not eligible for GA operations (selection, crossover, mutation) • Not same as elitism • CGA with added super individual converges in probability

  11. Contrast of Suzuki (1995) and Rudolph (1994) in Markov Chain Analysis for GA • Suzuki (1995, IEEE Trans. Systems, Man, and Cyber.) uses Markov chain analysis to study GA with elitism • Same as CGA of Rudolph (1994) except for elitism • Suzuki (1995) only considers unique states (populations) • Rudolph (1994) includes redundant states • With N = population size and B = no. of bits/chromosome: unique states in Suzuki (1995), 2NB states in Rudolph (1994) (much larger than number of unique states above) • Above affects bookkeeping; does not fundamentally change relative results of Suzuki (1995) and Rudolph (1994)

  12. Convergence Under Elitism • In both CGA case (Rudolph, 1994) and case with elitism (Suzuki, 1995) the limit exists: (dimension of differs according to definition of states, unique or nonunique as on previous slide) • Suzuki (1995) assumes each population includes one elite element and that crossover probability Pc = 1 • Let represent jth element of , and J represent indices j where population j includes chromosome achieving L() • Then from Suzuki (1995): • Implies GA with elitism converges in probability to set of optima

  13. Calculation of Stationary Distribution • Markov chain theory provides useful conceptual device • Practical calculation difficult due to explosive growth of number of possible populations (states) • Growth is in terms of factorials of N and bit string length (B) • Practical calculation of pk usually impossible due to difficulty in getting P • Transition matrix can be very large in practice • E.g., if N = B = 6, P is 108108 matrix! • Real problems have N and Bmuch larger than 6 • Ongoing work attempts to severelyreduce dimension by limiting states to only most important (e.g., Spears, 1999; Moey and Rowe, 2004)

  14. Example 10.2 from ISSO: Markov Chain Calculations for Small-Scale Implementation • Consider L() =  = [0,15] • Function has local and global minimum; plot on next slide • Several GA implementations with very small population sizes (N) and numbers of bits (B) • Small scale implementations imply Markov transition matrices are computable • But still not trivial, as matrix dimensions range from approximately 20002000 to 40004000

  15. Loss Function for Example 10.2 in ISSOMarkov chain theory provides probability of finding solution ( = 15) in given number of iterations

  16. Example 10.2 (cont’d): Probability Calculations for Very Small-Scale GAs

  17. Summary of GA Convergence Theory • Schema theory (Holland, 1975) was most popular method for theoretical analysis until approximately mid-1990s • Schema theory not fully rigorous and not fully connected to actual algorithm performance • Markov chain theory provides more formal means of convergence—and convergence rate—analysis • Rudolph (1994) used Markov chains to provide largely negative result on convergence for canonical GAs • Canonical GA does not converge to optimum • Suzuki (1995) considered GAs with elitism; unlike Rudolph (1994), GA is now convergent • Challenges exist in practical calculation of Markov transition matrix

  18. No Free Lunch Theorems (Reprise, Chap. 1) • No free lunch (NFL) Theorems apply to EC algorithms • Theorems imply there can be no universally efficient EC algorithm • Performance of one algorithm when averaged over all problems is identical to that of any other algorithm • Suppose EC algorithm A applied to loss L • Let denote lowest loss value from most recent N population elements after nN unique function evaluations • Consider the probability that after n unique evaluations of the loss: NFL theorems state that the sum of above probabilities over all loss functions is independent of A

  19. Comparison of Algorithms for Stochastic Optimization in Chaps. 2 – 10 of ISSO • Table next slide is rough summary of relative merits of several algorithms for stochastic optimization • Comparisons based on semi-subjective impressions from numerical experience (author and others) and theoretical or analytical evidence • NFL theorems not generally relevant as only considering “typical” problems of interest, not all possible problems • Table does not consider root-finding per se • Table is for “basic” implementation forms of algorithms • Ratings range fromL(low),ML(medium-low), M(medium), MH(mediumhigh), andH(high) • These scales are for stochastic optimization setting and have no meaning relative to classical deterministic methods

  20. Comparison of Algorithms

More Related