Utilizing Condor to Support Genetic Algorithm Design Research

1. Utilizing Condor to Support Genetic Algorithm Design Research Mary Beth Kurz, PhD Assistant Professor Department of Industrial Engineering Clemson University

2. Genetic algorithms are metaheuristics for optimization evaluation Solution space Objective space Feasible solutions Infeasible solutions Chromosome 1 My research focuses on what chromosomes should look like and asks whether the “solution representation” impacts the quality of a genetic algorithm decoding Chromosome 2 Chromosome N Chromosome space

3. Let’s make this more concrete: the TSP Each feasible solution’s total travel time is that solution’s objective evaluation Feasible Solutions: tours that visit all cities exactly once Solution space Objective space Infeasible Solutions: anything else Chromosome 1 The Traveling Salesman Problem asks how to route the salesman through his cities so he returns home as quickly as possible decoding Chromosome 2 Chromosome N • How do I represent the tours? • Directly – the city list? • Indirectly – the list of roads taken? Chromosome space

4. My Hypothesis: Solution representation affects GA design significantly No chromosomes map to these solutions Feasible solutions Optimal objective value Objective space Solution space Infeasible solutions This is the optimal solution!!! Chromosome 1 Chromosome 2 Fix or forbid these chromosomes? Chromosome space

5. Genetic algorithms are motivated by an analogy to “real” genetics A chromosome is composed of genes, generally randomly selected initially Genetic Operators Randomness comes here Population(t) Population(t+1) Chromosome 1 Chromosome 1 Selection picks some chromosomes as potential parents in crossover Crossover creates new chromosomes by taking genes from 2 parents Mutation changes a small number of genes in the entire population Chromosome 2 Chromosome 2 Chromosome N Chromosome N

6. This research is empirical and requires immense computational time • Genetic Algorithms are inherently random • Is it possible that some representation consistently finds better solutions for a specific problem? • Most GA research currently uses 50 replications on numerous data files • 180 problem types, 10 files each, 3 representations = 5400 files • Simplest representation – 1800 files would take about 45 hours in my Lab (a few years ago) • 50 replications of 5400 files → at least 241 days of running time! • This is simply not feasible

7. Grid computing is saving me Spring 2007 325,000 hrs Summer – Fall 2007 212,000 hrs Spring 2008 124,000 hrs Total: about 660,000 hrs

8. Since last spring, I’ve had to relearn how to do research! • How do I compile all this data? • VBA and Excel! • What can I actually analyze? • Not pictures like this • Reduce the data to correlations • What statistics do I need to use? • Needed to learn non-parametric statistics • Needed to use SPSS for the analysis • Used VBA to create the input files • Reran to get different output data • Summer – Fall 2007 212,000 hrs

9. I don’t know about random numbers • I started using rand() in C! • I use up to 600,000,000 random numbers in each run • I have 270,000 runs (5400 * 50) • Trying to use Mersenne Twister • Period is 219937 – 1 which is plenty big • How do I make sure I have independent sets of random numbers? • Use the same initial seed, then “burn” through (n-1) numbers until we get to the nth set • Would take over 4000 days to burn through 269,999 sets for the last run • Again … not feasible! • Tried to initialize using run number • Spring 2008 124,000 hrs

10. I still love Condor • But I don’t know about random numbers • Thought about saving the random numbers in an input file of 600,000,000 numbers each • Stopped generating the first file after it got to 3 GB • This would mean 3*270,000 GB of random number files! • Looking at dynamic streams from Mersenne Twister • Just heard about SPRNG from Todd on Tuesday • Gearing up for another set of runs … all I need is this set of runs to get a paper out!