1 / 31

Even More Random Number Generators Using Genetic Programming

Even More Random Number Generators Using Genetic Programming. Joe Barker. Topics. Genetic Programming Random Numbers Previous Efforts Design & Implementation Results Conclusion Future Bibliography. Genetic Programming. Evolve programs for solutions, instead of solutions

teenie
Download Presentation

Even More Random Number Generators Using Genetic Programming

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Even More Random Number Generators Using Genetic Programming Joe Barker

  2. Topics • Genetic Programming • Random Numbers • Previous Efforts • Design & Implementation • Results • Conclusion • Future • Bibliography

  3. Genetic Programming • Evolve programs for solutions, instead of solutions • Difficulty of representation • Higher level than standard EA compounds standard problems

  4. Genetic Programming (2) Gene Expression Programming • Encodes information in a similar way to genes(operation) to DNA(string) • Mutation & Crossover obvious string operations • Care required to avoid gibberish

  5. 2 + 1 * 3 AG-CC-GT-TA-CC Genetic Programming (3) Gene Expression Programming • Example

  6. Genetic Programming (4) Expression Trees • Encodes operations in a natural tree structure • Internal nodes are operations • Leaf-nodes are variables or constants • Mutation & Crossover follow from the structure • Layout of the tree avoids non-sensical results

  7. Genetic Programming (5) Expression Trees • Example

  8. Random Numbers • Why? • Evolutionary Algorithms • Monte-Carlo Simulations • Software Regression Testing • Game Playing

  9. Random Numbers (2) • What? • “Random” is difficult to define • Even statistical definitions necessarily describe what we would consider random • Uniform • 1-2-3-4-5 is Uniformly distributed but not what we would consider random • Tests exist to try and cover the important aspects of random

  10. Random Numbers (3) • Tests • Chi-Squared test for closeness of fit

  11. Random Numbers (4) • Tests • Frequency or Equidistribution test • Break number space into a small number of blocks • Use the counts for these blocks in Chi-Squared test for Uniform distribution • Gap test • Break number space into to classes(Normally upper and lower parts) • Count length of runs of class 2 between class 1 • Use a Chi-Squared test with the following distribution:

  12. Random Numbers (5) • Tests • Entropy • Arrange the numbers as a bitstring and count occurrences with certain lengths • 101111110101010110011001001110 • 10-11-11-11-01-01-01-01-10-01-10-01-00-11-10 • 101-111-110-101-010-110-011-001-001-110 • Use percent occurrences in the following formula:

  13. Random Numbers (6) • How? • Computers are deterministic, so we must approximate • Several classes of pseudo-random number generators(PRNGs)

  14. Random Numbers (7) • PRNGs • Linear congruential randomizers • Some of the earliest known • Common choices • Park-Miller: a=7^5 b=0 c=2^31-1 • URN08/RANDU: a=65539 b=0 c=2^31

  15. Random Numbers (8) • PRNGs • Shift register randomizers • SR[a,b,c] • A common choice is SR[3,28,31]

  16. Random Numbers (9) • PRNGs • Shuffling randomizer • Uses two other PRNGs • The first PRNG re/fills a list of numbers • The second PRNG selects number from the list • Inversive • Mersenne Twister

  17. Previous Efforts • This project is based largely on the work by John R. Koza • Used expression trees as individuals • The tree was executed on numbers 1..16K to obtain a random sequence • Bit entropy (lengths 1..7) • Non-Terminals=+,-,*,/,% • Terminals=J,0,1,2,3

  18. Design & Implementation • Individuals • Expression Tree • Non-Terminals=+,-,*,/,% • Å=XOR • Each non-terminal is equally likely in a random tree • Terminals=J,0,1,2,3 • 2^i = Power of 2 (i=1-31, uniform) • Each terminal is equally likely in a random tree • Output range is 0..2^32-1 • Aged some number of steps before mature

  19. Design & Implementation (2) • Evaluation – Fitness • Bit entropy (lengths 5,6,7,8) • Frequency tests (512 blocks) • Gap test for “runs above the mean” (Up to 10) • The alpha value calculated from the above two tests was adjusted by the formula: • All three values were normalized to a maximum of 1 and summed • These are then averaged over the life of the individual

  20. Design & Implementation (3) • Selection • 2 mature parents selected uniformly • There is a small chance of both crossover and mutation, but most likely only one • Crossover • Subtrees selected from each parent and swapped • Mutation • Replaced subtree with equal or smaller random tree

  21. Design & Implementation (4) • Crossover - Mutation

  22. Design & Implementation (5) • Competition • Replaces the bottom, by fitness, two mature population members • Other • No termination, runs indefinitely • HUP signal causes population to be dumped to file

  23. Results • Population size=100 • Mature Age=30 • Initial Maximum Tree Depth=10 • Crossover only chance=0.8 • Both chance=0.15

  24. Results (2) • After 10 hours on 5 machines, the best candidate was:

  25. Results (3) • Performance - Entropy Test

  26. Results (4) • Performance - Frequency Test

  27. Results (5) • Performance - Gap Test

  28. Results (6) • Performance - Speed Test (random nums/sec)

  29. Conclusion • It appears that employing EAs in this manner has promise • I hesitate to recommend using Stage 2 as a production randomizer as of now, but it does bear more investigation

  30. Future Work • Add prime numbers to the available terminals • Add more tests, such as periodicity, to the fitness function • Some type of runtime compilation instead of interpreting the expression trees

  31. Bibliography • Koza, John R., Evolving a Computer Program to Generate Random Numbers Using the Genetic Programming Paradigm, Proceedings of the Fourth International Conference on Genetic Algorithms, Morgan Kaufmann Publishers, Inc., pages 37-44, 1991. http://citeseer.nj.nec.com/john91evolving.html • Knuth, D. E., The Art of Computer Programming, Volume 2, Second Edition, Addison-Wesley, pages 9-114, Reading, MA, 1981. • Koza, John R., Genetically Breeding Populations of Computer Programs to Solve Problems in Artificial Intelligence, Proceedings of the Second International Conference on Tools for AI. Washington, November, 1990, IEEE Computer Society Press, Los Alamitos, CA 1990. http://citeseer.nj.nec.com/koza90genetically.html • Kinnear, Kenneth E. Jr., Evolving a sort: Lessons in genetic programming. Proceedings of the 1993 International Conference on Neural Networks, volume 2. IEEE Press, 1993. http://citeseer.nj.nec.com/kinnear93evolving.html • Kinnear, Kenneth E. Jr., Generality and Difficulty in Genetic Programming: Evolving a Sort, Proceedings of the Fifth International Conference on Genetic Algorithms, Morgan Kaufmann Publishers, pages 287-294, Inc., 1993. http://citeseer.nj.nec.com/kinnear93generality.html

More Related