Tutorial on ParadisEO: Parallel Models for the TSP

1. Tutorial on ParadisEO: Parallel Models for the TSP

2. Contributions EO (A framework of Evolutionary Algorithms) Techniquesrelated tomulti-objective optimization ParadisEO (Parallel and distributedmetaheuristics) http://www.lifl.fr/~cahon/paradisEO/ Solution-basedmetaheuristics Parallelism (Partitioning solutions at several steps) Hill Climbing Simulated Annealing Cooperation(Algorithms exchange solutions)Ex. Island cooperation Tabu search http://www.lifl.fr/~cahon/paradisEO/ls/

3. A case study : the «Traveling Salesman Problem» • From a full graph of vertices find the shortest hamiltonian cycle. • NP-hard • EO Designing a G.A. • ParadisEO Illustrating some forms of parallelism

4. TSP Case-study – Components > cd $HOME/tsp/src/ > ls –al ~/tsp/ src/ main.cpp – main workbench file – GAs, LS etc. route.h – chromosome definition route_init.h – random chr. initialization operator part_route_eval.h – partial fitness eval. Operator merge_route_eval.h – fitness evaluation aggregation op. route_eval.h – full fitness evaluation operator order_xover.h – OrderXover crossover operator edge_xover.h – EdgeXover crossover operator city_swap.h – CitySwap mutation operator {two_opt_init, two_opt_next, two_opt_incr_eval}.h DEFINITION EVALUATION OPERATORS

5. An unifying view of three hierarchical levels • For both the population-based and solution-based metaheuristics, • The deployment of concurrent independent/cooperative metaheuristics • The parallelization of a single step of the metaheuristic (based on distribution of the handled solutions) • The parallelization of the processing of a single solution

6. DEFINITION TSP – The Initial Algorithm eoPop <Route> ox_pop (POP_SIZE, route_init); /* Population and initializer */ eoGenContinue <Route> ox_cont (NUM_GEN); peoSeqPopEval <Route> ox_pop_eval (full_eval); eoStochTournamentSelect <Route> ox_select_one; eoSelectNumber <Route> ox_select (ox_select_one, POP_SIZE); eoSGATransform <Route> ox_transform (order_cross, CROSS_RATE, city_swap_mut, MUT_RATE); peoSeqTransform <Route> ox_seq_transform (ox_transform); eoEPReplacement <Route> ox_replace (2); eoCheckPoint <Route> ox_checkpoint (ox_cont); /* Checkpoint */ peoEA <Route> ox_ea (ox_checkpoint, ox_pop_eval, ox_select, ox_seq_transform, ox_replace); ox_ea (ox_pop); /* Application to the given population */ continuation criterion evaluation method selection strategy ops. transformations replacement strategy

7. The parallel evaluation of the population Recombination,mutation Selection Replacement Computed values arecoming back Partitionning Full Eval. Full Eval. Full Eval. Evaluating nodes

8. TSP – The Parallel Evaluation of the Population eoPop <Route> ox_pop (POP_SIZE, route_init); /* Population and initializer */ eoGenContinue <Route> ox_cont (NUM_GEN); peoParaPopEval <Route> ox_pop_eval (full_eval); eoStochTournamentSelect <Route> ox_select_one; eoSelectNumber <Route> ox_select (ox_select_one, POP_SIZE); eoSGATransform <Route> ox_transform (order_cross, CROSS_RATE, city_swap_mut, MUT_RATE); peoSeqTransform <Route> ox_seq_transform (ox_transform); eoEPReplacement <Route> ox_replace (2); eoCheckPoint <Route> ox_checkpoint (ox_cont); /* Checkpoint */ peoEA <Route> ox_ea (ox_checkpoint, ox_pop_eval, ox_select, ox_seq_transform, ox_replace); ox_ea (ox_pop); /* Application to the given population */

9. Recombination, mutation Selection Computationf(f1,f2, …fn ) Partiallyevaluatedsolutions Replacement Replication Partial Eval.e2 Partial Eval.en Partial Eval. e1 Evaluating nodes The parallel evaluation of a single solution

10. The distributed evaluation step peoPopEval Route - provided () peoParaPopEval eoPop & Route Route Features eoAggEvalFunc<EOT> vector< eoEvalFunc<EOT> > different partial evaluation functions aggregation of fitness values

11. TSP – The Parallel Evaluation of the Fitness Function MergeRouteEval merge_eval; std :: vector <eoEvalFunc <Route> *> part_eval; for (unsigned i = 1 ; i <= NUM_PART_EVALS ; i ++) { part_eval.push_back (new PartRouteEval ((float) (i - 1) / NUM_PART_EVALS, (float) i / NUM_PART_EVALS)); } . . . eoGenContinue <Route> ox_cont (NUM_GEN); peoParaPopEval <Route> ox_pop_eval (part_eval, merge_eval); eoStochTournamentSelect <Route> ox_select_one; eoSelectNumber <Route> ox_select (ox_select_one, POP_SIZE); . . .

12. Migrations Cycle of evolutions Cycle of evolutions Cycle of evolutions Cycle of evolutions The cooperative island model • Several asynchronous GAs (coarse grain) • Different sub-populations • Asynchronous migration of individuals • Different stopping criteria • Convergence • Frequency / generations • Miscellaneous • Heterogeneous parameters, operators, algorithms, …

13. Manager of asynchronous migrations () eoAsyncIslandMig eoPop & Route Route eoContinue Event « Immigration is required ? » eoSelect Selection of individuals to emigrate Features Integration of immigrants into the localpopulation eoReplacement eoTopology Topology (Ring, Full, …) - provided

14. TSP – Synchronous Island Model /* Migration occurs periodically */ eoPeriodicContinue <Route> ox_mig_cont (MIG_FREQ); eoRandomSelect <Route> ox_mig_select_one; eoSelectNumber <Route> ox_mig_select (ox_mig_select_one, MIG_SIZE); /* Immigrants replace the worse individuals */ eoPlusReplacement <Route> ox_mig_replace; peoSyncIslandMig <Route> ox_mig (MIG_FREQ, // migration frequency ox_mig_select, // strategy of selection ox_mig_replace, // strategy of replacement topo, // topology – ring topology ox_pop, // source population of emigrants ox_pop); // destination population for imigrants

15. TSP – Asynchronous Island Model /* Migration occurs periodically */ eoPeriodicContinue <Route> ox_mig_cont (MIG_FREQ); eoRandomSelect <Route> ox_mig_select_one; eoSelectNumber <Route> ox_mig_select (ox_mig_select_one, MIG_SIZE); /* Immigrants replace the worse individuals */ eoPlusReplacement <Route> ox_mig_replace; peoAsyncIslandMig<Route> ox_mig (ox_mig_cont, // migration frequency ox_mig_select, // strategy of selection ox_mig_replace, // strategy of replacement topo, // topology – ring topology ox_pop, // source population of emigrants ox_pop); // destination population for imigrants

16. Design of several levels of parallelization/hybridization Processing of a single solution(Objective / Data partitioning) Independent walks, Multi-start model, Hybridization/Cooperationof metaheuristics Parallel evaluation ofthe neighborhood/population Scalability Heuristic Population / Neighborhood Solution

17. An example of decomposition in tasks (High level) Starting Fork Cooperativeisland model Fork Fork « Active messages » Fork Shared_rw<eoPop <EOT>> Shared_rw<EOT> Fork Fork Fork Fork Local searches Parallel evaluators

18. An example of decomposition in tasks (Low level) • Internal functions embedded in E.As Partitioning the« data » for the steps of • Selection • Crossover • Evaluation • Replacement Population Shared_r<EOT> Shared_w<EOT :: Fitness> Evaluation Shared_rw<EOT> Recombination

19. MPICH2 – Environment An initial test – list the machines running MPDs (MPI daemons) > mpdtrace lxo9 lxo10 … > mpdringtest 100 number of loops time for 100 loops = 0.0880811… seconds > mpiexec –n 8 hostname An initial test – sending a message in the ring of machines An initial test – launching processes on the machines no. of processes process name

20. TSP Compiling and Launching setting the correct directory for compilation: > cd $HOME/tsp/ compiling the application > make launching four processes – the tsp.param specifies the config. Params (there will be one process on each of the machines assigned to you) > mpiexec –n 4 ./tsp @tsp.param cleanning the directory (erasing the obj. and core files, etc) > make clean

21. Real-world applications developed with ParadisEO • Data Mining in Near-Infrared Spectroscopy • Radio Network design • Genomics

22. Conclusion • ParadisEO is a white-box OO framework • Clear conceptual separation of solution methods and problems • Maximum design and code reuse • High flexibility (fine-grained EO objects) • ParadisEO provides a broad range of features • Evolutionary algorithms, local searches and natural hybridization mechanisms • Invariant part provided • Various transparent and portable parallel/distributed models • Experimental evaluation on academic problems and industrial applications • High reuse capabilities, efficiency of the parallel/distributed models