JavaGenes and Condor: Cycle-Scavenging Genetic Algorithms

1. JavaGenes and Condor: Cycle-Scavenging Genetic Algorithms Al Globus, CSC at NASA Ames Research Center Eric Langhirt, AMTI at NASA Ames Research Center Miron Livny, University of Wisconsin Ravishankar Ramamurthy, University of Wisconsin Marvin Solomon, University of Wisconsin Steve Traugott http://www.nas.nasa.gov/~globus/papers/JavaGrande2000/JavaGrandePaper.html

2. Abstract • JavaGenes implements a genetic algorithm using a graph representation • Stochastic • Embarrassingly parallel • Executed hundreds of multi-day jobs on Condor cycle-scavenging batch system using otherwise idle workstations • Pharmaceutical drug molecules and digital circuits have been evolved with reasonable success • Java good, but serialization may be too slow http://www.nas.nasa.gov/~globus/papers/JavaGrande2000/JavaGrandePaper.html

3. Genetic algorithm • Randomly generate a set of molecules • Rate them with a fitness function • Many times: • Select parent molecules at random with bias towards better fitness • Randomly rip copies of each parent in two • Mate opposite halves and rate new molecules • Replace random molecules with bias towards worse fitness • Repeat until satisfied http://www.nas.nasa.gov/~globus/papers/JavaGrande2000/JavaGrandePaper.html

4. Crossover http://www.nas.nasa.gov/~globus/papers/JavaGrande2000/JavaGrandePaper.html

5. JavaGenes in Action Finding with all-pairs-shortest-path and Tanimoto index fitness function (0 is perfect) http://www.nas.nasa.gov/~globus/papers/JavaGrande2000/JavaGrandePaper.html

6. Condor • Cycle-scavenging batch system for single workstation jobs • University of Wisconsin (http://www.cs.wisc.edu/condor) • In production since 1986 • Unix workstations (limited Windows NT) • 200 SGI and 50 Sun workstations at NASA Ames • Good for • parameter studies • stochastic algorithms (e.g., genetic algorithms) http://www.nas.nasa.gov/~globus/papers/JavaGrande2000/JavaGrandePaper.html

7. Condor and Java • Standard universe • link with Condor library • automatic io redirection • automatic checkpointing • no kernel level threads (e.g., no fork()) • impractical for Java jobs • Vanilla universe jobs worked • Disk access via NFS • Periodic application-specific checkpointing • At startup, check for checkpoint file http://www.nas.nasa.gov/~globus/papers/JavaGrande2000/JavaGrandePaper.html

8. Byte Code Checkpointing Problematic • Condor jobs may be stopped at any arbitrary time • Stack format not defined • Heap format undefined and serialization slow • JIT and optimization causes problems • Must modify or write Java Virtual Machine • Kaffe couldn’t run JavaGenes • There are rumors regarding checkpointable JVMs • Pointers welcome http://www.nas.nasa.gov/~globus/papers/JavaGrande2000/JavaGrandePaper.html

9. Java Universe • Application implements Checkpointable and Serializable: • start(String[] arguments); • restart(); • Application calls • Checkpointer.ok() • Checkpointer.checkpoint() • Condor (or checkpointing thread) calls • Checkpointer.checkpointWhenPossible() • Checkpointer.prepareToDie() • Checkpointer.areYouReadyToDie() • Checkpointer.cancelDeath() http://www.nas.nasa.gov/~globus/papers/JavaGrande2000/JavaGrandePaper.html

10. Java Con • Serialization seemed unbelievably slow and big • three hours and 25 Mbytes for 500 graphs • plus memory leak found last month! • two day application-specific checkpoint implementation: < 1Mbyte / 10 seconds - 5 minutes • Restart from checkpoint leads to optimistic performance data • class Double problem: • double foo = Infinity; // or NaN; • Double(foo + “”) throws bad format exception http://www.nas.nasa.gov/~globus/papers/JavaGrande2000/JavaGrandePaper.html

11. Java Pro • Porting trivial • Unix: SGI and Sun JDK 1.1.x • Windows 95: Visual Cafe, Superseed, JBuilder, Code Warrior, JDK 1.1.x • Bugs • significantly less frequent than in C or C++ • mostly logic bugs, easy to find and fix • Memory Management very easy • one clone() bug • one memory leak quickly found by OptimizeIt http://www.nas.nasa.gov/~globus/papers/JavaGrande2000/JavaGrandePaper.html

12. Summary • Except for serialization, performance is adequate though not spectacular • Wrote once ran everywhere we wanted • No GUI • Accomplished non-Java, non-Condor computational science • "Automatic molecular design using evolutionary techniques," Al Globus, John Lawton, and Todd Wipke, Nanotechnology, Volume 10, Number 3, September 1999, pp. 290-299. http://www.nas.nasa.gov/~globus/papers/JavaGrande2000/JavaGrandePaper.html