Reactive Agents to Improve a Parallel Genetic Algorithm Solution

1. Reactive Agents to Improve a Parallel Genetic Algorithm Solution Ana Laureano1, José de la Cruz1, Javier Ramírez1 and Julio Solano2 1Universidad Autónoma Metropolitana - Azcapotzalco 2DISCA - IIMAS / UNAM

2. MultiAgentArchitecture • Reactive Agents • 80s Brooks • Exhausitive representation of the real world • The behaviour is emergent and it is dependent on the perception of the environment at a given instant

3. Agents based on behaviour • A constant interaction with the environment • Control mechanism allowing them to work with limited resources and incomplete information • The advange of the use, lies in the speed of adaptation to unforeseen situations

4. Proposal ... • Analysis and design of reactive agents based • Bottom-up design mecahnism • Maes • Laureano, de Arriaga and García-Alegre

5. To improve a genetic algorithm • Parallel genetic algorithm • Efficiency of the classification of a universal set in two independent subsets • The balance between the capacity to converge to an optimum and the ability to explore new regions (pm, pc) • pm, pc are varied adaptively according to the fitness values of the solution

6. To improve a genetic algorithm • This architecture is implmented by establishing a similarity between the desires espressed by MAES • Eating • Exploring • Fleenig • Interchange individuals • Improve the quality of the solution

7. Design of the Reactive Genetic Algorithm • Maes: • Artificial creatures • Beahviours(eating, drinking, exploring, going-towards-food, going-towards-water, sleeping, fighting, fleeing-from-creature, going-towards-creature, and dodging-obstacle) • Motivations (thirst, hunger, sleepiness, fear, tiredness, safety and agressiveness) • Virtulal environment (water, food, another creature, an obstacle)

8. Each behaviour is associated with: • an activation level represented by a real number • a set of conditions • a threshold, represented by a real number

9. Behavior activation mechanisms are represented by: • the creature’s desire to perform a given behaviour • the present situation: the environment influences the creature’s desires • activation through the precedent

10. Parallel Genetic Algorithm with Reactive Characteristics • The purpose of incorporating a reactive behavior in the PGA, is improve the overall performance by up-dating the sub-populations according to the general behaviour of the algorithm avoiding getting stuck in local minima

11. The Reactive Parallel Genetic Algorithm (RPGA) will be able to send and receive individuals from and to any other sub-population. The former, at the time required, by observing the stimuli presented by the environment at a given time.

12. Characteristics and elements that will substitute the traditional migration mechanism: • A sub-population might send individuals to another sub-population in any generation. There is no need for both sub-populations to be synchronized. The sender does not know in what generation, nor to whom will it send individuals, until the very moment they are sent.

13. Due to the reactive migration plan, several cases can arise in the individuals interchange stages. These go from the minimal case in which in a given generation no population sends any individuals, to the maximum one in which, given a generation, all the populations send individuals to the n - 1 remaining ones. All the subsets of ordered pairs of the set of sub-populations must be considered.

14. In each generation, each sub-population (GA) must verify if individuals from other sub-populations have arrived. The former due to the lack of synchronization and migration topology. For that reason, all the sub-populations (GA), must check a special buffer, which will be called the station. If the station is empty, the GA continues on its task; if it is not, the individuals are extracted and added to the sub-population.

15. The sending of individuals from one sub-population to another is determined by certain non-deterministic impulses assigned to each GA. Those impulses would be controlled by a mechanism similar to the one used by MAES to simulate the behavior of simple creatures in a dynamic environment.

16. The new concepts are defined below: • Migration Limit:is a value representing the limit that the migration impulse must surpass • Migration Impulse: is a value representing the GA’s desire of sending individuals to a specific sub-population • Stimulus Range: is an interval comprising the allowed values of the stimulus in order to increment the migration impulse. The generated value will be called stimulus and it will be added to the present value of the migration impulse

17. An important aspect of the GA is the quality of the solution. For that reason, a mechanism is included in which each population requests to the n-1 remaining sub-populations the sending of individuals. Such request will be made based on the time elapsed from the last time the former best solution was improved. The time interval (number of generations) elapsed without improvement will be called improvement limit.

18. When in a GA a number of generations have passed and the improvement limitdid not change, it sends a message to the other n-1 GAs.This message is interpreted as a request for individuals from the sender. The former will allow the rest of the GAs (sub-populations) to send their own individuals.

19. Generate Initial Population Do While Population Evolution Criterion not Fulfilled Find Aptitude Select Individuals Cross Individuals Mutate Individuals Increment Migration Impulse If (Requests Arrived), Then Increment Migration Impulse End If If (Migration_Impulse > Migration_Limit), Then Send Individuals End If If (individuals arrived), Then Add Individuals to Population End If If (solution not Improved in given time) Then Send Individuals Request End If End Do

21. Experiments: • We had available a set of 201 samples: • 49 from group 1 • 152 from group 2 • Comparison between genetic algorithms according to the effectiveness of each one, in order to classify accurately each individual in its proper group. • An objective function was designed for this purpose, determining the number of successes for each individual in the population.

22. where: n is the number of samples from group 1. m is the number of samples from group 2. C a constant, used to assign a greater weight to successes in samples from group 1.

23. Two types of mutation were used: static and dynamic for Aind > Aavg pm = k2 for Aind Aavg • where: • pm = mutation probability. • Amax = aptitude of the best individual of the population in that generation. • Aavg = average aptitude of all individuals of the population in the generation. • Aind = aptitude of the individual for which pm is being estimated. • k1 = 0.5 • k2 = 0.25

24. Parameters and Configuration • The RPGA uses a maximum of 2000 generations. • The universal stochastic selection method of reproduction is applied. • The crossing mechanism is a discrete one with a probability of 1.0. • The mutation mechanism is based on the one proposed by Mühlenbein and Schilerkamp-Voosen. • The mutation probability is dynamic, based on the work of Patniak and Mandavilli.

25. Parallel Genetic AlgorithmReactive Genetic Algorithm Run Time Aptitude Effect Time Aptitude Effect 1 370 0.9813 93.58 409 0.9798 93.08 2 372 0.9829 94.06 407 0.9829 94.06 3 368 0.9844 94.56 410 0.9829 94.06 4 369 0.9829 94.06 412 0.9813 93.58 5 375 0.9782 92.58 413 0.9798 93.08 6 375 0.9813 93.58 411 0.9844 94.56 7 389 0.9798 93.08 408 0.9813 93.58 8 371 0.9782 92.58 407 0.9844 94.56 9 372 0.9798 93.08 410 0.9844 94.56 10 371 0.9844 94.56 415 0.9829 94.06 Avera 373.2 0.981393.57 410.2 0.982493.92 Table 1. Static mutation

26. Parallel Genetic AlgorithmReactive Genetic Algorithm Run Time Aptitude Effect Time Aptitude Effect 1 670 0.9829 94.06 707 0.9860 95.06 2 679 0.9875 95.55 710 0.9891 96.04 3 673 0.9860 95.06 711 0.9829 94.06 4 672 0.9813 93.58 709 0.9844 94.56 5 670 0.9860 95.06 710 0.9891 96.04 6 669 0.9860 95.06 708 0.9813 93.58 7 671 0.9829 94.06 700 0.9891 96.04 8 675 0.9813 93.58 710 0.9860 95.06 9 679 0.9829 94.06 712 0.9875 95.55 10 673 0.9829 95.55 715 0.9891 96.04 Avera 673.1 0.984094.56709.20.986495.20 Table 2. Dynamic mutation

27. The RPGA has available a population of 800 individuals. The best results were obtained when it was divided into 8 sub-populations of 100 each. Members of the population migrate using the reactive mechanism with a migration percentage of 10%. In order to implement a reactive migration scheme, several runs were conducted. The best results were obtained for migration rates of: 10, 15 and 20. To save communications we choose 20 as migration limit. Therefore, migration impulses are set to zero and progressively increased in each generation randomly

28. Conclusions • A migration mechanism for genetic algorithms controlled by reactive agents is presented. • The results obtained show that both algorithms present their best results when dynamic mutation is used, the RPGA being the best for classifying, with a mean effectiveness percentage of 95.20%, while the PGA reaches 94.55%.

29. Conclusions • In that way the RPGA can be considered as an efficient method for the solution of complex problems. • Even if the computing time of the RPGA is larger, it has however a practical advantage, a higher classification average.