Evolvable Random Number Generators: A Schemata-based Approach

1. Evolvable Random Number Generators: A Schemata-based Approach Robert Watkins, Jason Isaacs, Simon Foo FAMU-FSU College of Engineering P5

2. Genetic Algorithm Based Random Number Generator This project seeks to develop Genetic Algorithm-based Random Number Generator on a field programmable gate array (FPGA) P5

3. Genetic Algorithms • Foundations in natural evolution • Development by Holland • Recombination Techniques • Implementation P5

4. Evolutionary Foundations • General features of evolution that are widely accepted: • Evolution is a process that operates on chromosomes rather than on the living beings they encode. • Natural selection is the link between chromosomes and the performance of the decoded structures. Processes of natural selection cause those chromosomes that encode successful structures to reproduce more often than those that do not. • The process of reproduction is the point at which evolution takes place. Mutations may cause the chromosomes of biological children to be different from those of their biological parents and recombination processes may create quite different chromosomes in the children by combining material from the chromosomes of two parents. • Biological evolution has no memory. Whatever it knows about producing individuals that will function well in their environment is contained in the gene pool – the set of chromosomes carried by the current individuals – and in the structure of the chromosome decoder. P5

5. Development by Holland • Intrigued by these features of evolution John Holland first suggested that they be incorporated into a computer algorithm in the early 70’s. • Holland believed they might yield a technique for solving difficult problems the way nature has done – through evolution. • These algorithms, using simple encoding and reproduction mechanisms, displayed complicated behavior, and turned out to solve some extremely difficult problems; all without the knowledge of the decoded world. • Genetic algorithms belong to the class of stochastic search methods, including simulated annealing, threshold acceptance, and breadth-first and depth-first searches. Most of the methods operate on a single solution to a given problem, genetic algorithms operate on a population of solutions. P5

6. Implementation • To implement a genetic algorithm, you must encode the ‘solutions’ in a usable structure that can be manipulated in a computer. These structures are called chromosomes. Our chromosomes will be binary strings. • The population of chromosomes will reproduce according to their respective level of fitness with predetermined selection criteria (proportional selection). This may be replaced with rank-based selection to prevent premature convergence. • Genetic algorithms manipulate schemata – building blocks of good solutions – that are combined through crossover and that spread in the population in proportion to the relative fitness of the chromosomes that contain them. P5

7. Implementation The Genetic Algorithm 1. Initialize the population of chromosomes. 2. Evaluate each chromosome in the population. 3. Create new chromosomes by mating current population; apply mutation, crossover and other recombination techniques. 4. Population replacement; replace x% of n with x% of n+1. 5. Evaluate new population. 6. If stopping criteria is true then return best chromosome, else go to step 3. P5

8. Recombination Techniques Given two 4-bit chromosomes <0010> and <1011>. Crossover: single-point crossover at the third locus. <00|10> <10|11> <00|11> <10|10> Mutation: single-point with random probability ~ 0.02% for our case. <0010> <0110> Inversion: not a widely used technique. <0010> <1101> Crossover operations are an extremely important component of a genetic algorithm. Without it a genetic algorithm is no longer a genetic algorithm. This can not be said of mutation operators. Mutation operators are important only to reintroduce diversity into a population and prevent localization. We can use combinations of these methods to ensure a broader query and prevent premature convergence. P5

9. Random Number Generator • For this project we will focus on cryptographically secure pseudorandom bit generation. A PRBG is cryptographically secure if there is no polynomial-time algorithm which, on input of the first l bits of an output sequence s, can predict the (l + 1)st bit of s with probability significantly greater than ½. Possibly under some plausible but unproven mathematical assumption such as the intractability of factoring integers. • We are looking into and testing three such RNG’s, the RSA, Blum-Blum-Shub, and the ISAAC 64-bit generators. • For testing we are using the DIEHARD package by Marsaglia, which includes most of the FIPS 140-l statistical tests (used for secure unclassified document encryption), a chi-squared test, and other stringent uniformity tests. P5

10. RNGs An ideal Random Number Generator (RNG), produces an infinite string of digits which are uncorrelated and uniformly distributed TrueRandom Number Generators:tap source of natural randomness; digits are often poorly distributed Quasi-Random Number Generators (QRNG):produce uniformly distributed digits; considered appropriate for Monte Carlo simulations Pseudo-Random Number Generators (PRNG): no detectable biases or correlations; suitable for cryptographic purposes P5

11. Some commonRNG: LCGs - Linear Congruential Generators: xn+1 = a xn + b (mod m); LCG( m, a, b, x0 ), 0 < a, b < m Shift & XOR Generators: Any of various generators based on schemes for shifting and XORing input stream(s) or seeded array(s); other operations are often included (e.g. RC4 and ISAAC). ICGs - Inversive Congruential Generators: xn+1 = a (xn )-1 + b (mod p) ICG( p, a, b, x0 ), with p prime, 0 < a, b < p, and (xn)-1 indicates the inverse of xn mod p Lagged Fibonacci Generators: xn = xn -r + xn -s (mod m) LFG( m, r, s, x0, x1, x2,... xs) P5

12. First Approach/Theoretic Considerations: Parameter/seed optimizations governed by group theory m, prime a, b, x0 in Zp LCGs - Linear Congruential Generators: xn+1 = a xn + b (mod m); LCG( m, a, b, x0 ), 0 < a, b < m LCG and its close cousins Multiple Recursive Generators MRG and Combined Linear Congruential Generators cLCG are the workhorses modern random number generation. P5

13. First Approach/Theoretic Considerations: GA-LCG - (Hernandez, et al, 2000) Considered 5 fitness functions: w/ varied performance on each test. (A main conclusion was the independence of the five functions tested.) Consistently failed Chi-square test We have modified the Hernandez genome to increase the range of the modulus m such that the GA evolves generators that will pass the Chi-square test. This method works well to find good LCGs but most LCGs are poor RNGs, therefore we have considered evolving combinations of parameters and functions employed in many RNGs. P5

14. Current Approach • Classify of parameters/functions from known RNGs • Encode these parameters/functions into the chromosome • Evolve combinations to produce RNGs P5

15. Fips Score Schemata based GARNG Gene Given: Chromosomal RNG Structures Encoding/Decoding |0|1|1|5|9|9|12|3|3|6|5|5|7|0|10|14|14|13|13|6|0|4|15|10|7|2|7|20|66| Decoded: P5

16. yn+k= k-1j=0 MRG * ajyn+j mod m c xn+1= xor cLCG yn+1= + ICG * * * * b mod p + + a ūn mod 235 mod 235 d c bxi axi Example Schemata-based GA RNG Structures P5

17. yn+k= yn+k= k-1j=0 k-1j=0 MRG MRG * * ajyn+j ajyn+j mod m mod m c * yn+1= + * * ICG * * b + + mod p a mod 235 mod 235 ūn d d bxi bxi Example Schemata-based GA RNG Structures: reproduction xn+1= xor cLCG * + mod 235 c axi P5

18. Iter.? Genetic Algorithm Seed User Input Stop? Recombinant Method Gene Tag Translator P F In-depth testing Stop Fitness test Random Number High pass Our Algorithm in Outline P5

19. FIPS 140: 200,000 bits, subjected to each of the following tests. If any of the tests fail, then the generator fails the test. Monobit test: number of 0's or 1's is between 99654 and 100346 Poker test: statistic, X, is between 1.03 to 57.4 X depends on the number of occurrences of each four bit word. (for increased stringency: 8-bit words) Runs test: blocks and gaps of length n between 1 and 6 are compared to a look-up table to determine acceptable ranges. (For the purpose of this test, all runs/gaps of length greater than 6 are counted together. Long run test. The long run test is passed if there are no runs of length 34 or more. P5

20. Diehard: Diehard is a statistical test suite, developed by George Marsaglia, and widely recognized as among the most stringent tests of randomness. BIRTHDAY SPACINGS OPERM5 31x31 BINARY RANK 32x32 BINARY RANK 6x8 BINARY RANK BITSTREAM OPSO OQSO DNA COUNT-THE-1's COUNT-THE-1's (specific bytes) PARKING LOT MINIMUM DISTANCE 3D SPHERES SQUEEZE OVERLAPPING SUMS RUNS CRAPS P5

21. Crossover and Selection Method Analysis • Selection • Roulette: random selection • Tournament: top-five reproduce with remaining population • Elitist: top-five reproduce within each other • Crossover – single-site • Midpoint • Random Point P5

22. Average Performance over 100 Generations using a Roulette Selection Mechanism P5

23. Average Performance over 100 Generations using a Tournament Selection Mechanism P5

24. Average Performance over 100 Generations using an Elitist Selection Method P5

25. Average Performance over 100 Generations using a Random-point Crossover Recombination P5

26. Average Performance over 100 Generations using a Fixed-point Crossover Recombination P5

27. P5

28. P5

29. P5

30. Schemata-based GA-RNG • Recent Improvements • 150 Reassignment RNG added • User selected crossover method • Roulette, Elitist, Tournament, and Random-Site Tournament • Lengthened gene and enlarged population • Performance • Better performance with 150 Re-RNG than implementation with C++ RNG • Longer gene improved performance • Larger population allowed faster convergence to “good” solutions • Barriers to full FPGA implementation • Decoder size and complexity • Complex mathematical functions • Large prime number storage and use P5