In the name of God shiraz university C omputer Science & Engineering Department

1. In the name of Godshiraz universityComputer Science & Engineering Department Presenter : M.EftekhariAdvisor : DR.S.D.Katebi Ant Colony Optimization

2. Introduction • History & Applications • Static problems • Dynamic problems • Ant Systems ( AS ) • Ant Colony Systems ( ACS ) and ACO pseudo code • MMAS and ASRANK • The Other Applications of ACO

3. Swarm Intelligence: • Aco is a branch of a larger field referred to as Swarm Intelligence . • Swarm intelligence is the behavioral simulation of social insects such as bees ,ants wasps and termites .Also behavioral simulation of social animals such as birds and fishes . • Examples : Robot-bees , Army ant robots , PSO ( Particle Swarm Optimization ) , Miniature Robots.

4. Real Ants: • Aco simulates the collective foraging habits of ants. • Ants have poor vision and poor communication skills . • A large group or swarm of ants can collectively perform complex tasks with proven effectiveness : gathering food , sorting corpses , performing division labor. • The key to such group effectiveness is pheromone. • Pheromone is a chemical substance deposited by ants as they travel . Pheromone provides ants with the ability to communicate with each other ( stigmergy ).

5. Self-Organization : single insect functions autonomously. There is no “supervisor” insect. • Ant essentially move randomly . • Positive Feed Back : If they decide to follow one path , they deposit pheromone on the trail which reinforces the path. The probabilitythat an ant chooses one path over another is governed by the amount of pheromone on the potential path of interest. • Pheromone Evaporation ( is called Negative Feed Back in Artificial Ants ).

6. Stigmergy is : Stimulation of workers by the performance they have achieved. Or on the other hand: indirect communication mediated by modifications of the environment . • consequently Reinforcing and Evaporating of pheromone. • Two types of stigmergy • Qualitative. • Quantitative.

7. If we assume that the only way around the obstacle is either right or left, we can safely assume that at first approximately half of the ants will go right and the other half left at randomly, as illustrated in the next slide. • The ants that happen to pick the shorter path will obviously create a strong trail of pheromone a lot faster than the ones choosing a longer path(see next slide). .

8. Food Nest Food Nest

10. Ant algorithms were first proposed by Dorigo and colleagues as a multi-agent approach to difficult combinatorial optimization problems like the traveling salesman problem (TSP) and the quadratic assignment problem (QAP). • Ant System (AS) was the first (1991) ACO algorithm. Its importance resides mainly in being the prototype of a number of ant algorithms which have found many interesting and successful applications.

11. In AS artifcial ants build solutions (tours) of the TSP by moving on the problem graph from one city to another. • Three AS algorithms have been defined , which differ by the way pheromone trails are updated. These algorithms are called ant-density, ant-quantity,and ant-cycle. • In ant-density and ant-quantity ants deposit pheromone while building a solution, while in ant-cycle ants deposit pheromone after they have built a complete tour.

12. Consequently, research on AS was directed towards a better understanding of the characteristics of ant-cycle, which is now known as Ant System, while the other two algorithms were abandoned. • The main role of pheromone evaporation is to avoid stagnation, that is, the situation in which all ants end up doing the same tour.

13. The memory (or internal state) of each ant contains the already visited cities and is called tabu list . • The memory is used to define, for each ant k, the set of cities that an ant located on city i still has to visit. By exploiting the memory therefore an ant k can build feasible solutions by an implicit state-space graph generation (in the TSP this corresponds to visiting a city exactly once).

14. The ant-decision table Ai = [aij(t)]|Ni|of node i is obtained by the composition of the local pheromone trail values with the local heuristic values as follows:

15. where τij(t) is the amount of pheromone trail on arc (i, j) at time t, ηij = 1/dijis the heuristic value of moving from node i to node j. • Ni is the set of neighbors of node i . • α and β are two parameters that control the relative weight of pheromone trail and heuristic value. The probability with which an ant k chooses to go from city i to city while building its tour at the t-th algorithm iteration is:

16. where is the set of nodes in the neighborhood of node i that ant k has not visited yet (nodes in are selected from those in Ni by using the ant private memory Mk). • The role of the parameters α and β: If α= 0, the closest cities are more likely to be selected: ( greedy algorithm) . If on the contrary β=0, only pheromone amplification is at work: (stagnation situation). • A trade-off between heuristic value and trail intensity therefore appears to be necessary.

17. After all ants have completed their tour, pheromone evaporation on all arcs is triggered ,and then each ant k deposits a quantity of pheromone ∆τkij(t) on each arc that it has used: • Where is the tour done by ant k at iteration t , is its length.

18. In practice, the addition of new pheromone by ants and pheromone evaporation are as following : τij(t+1) (1 - ρ)τij(t) +∆τij(t,t+1) • Where , m is the number of ants at each iteration (maintained constant), and ρ є(0, 1] is the pheromone trail decay coefficient.

19. The initial amount of pheromone τij(0) is set to a same small positive constant value τ0 on all arcs. • The total number of ants is set to m = n . • while α, β and ρ are respectively set to 1, 5 and 0.5; these values were experimentally found to be good by Dorigo . • Dorigo introduced also elitist ants, that is, a daemon action by which the arcs used by the ant that generated the best tour from the beginning of the trial get extra pheromone.

20. Problems • Unfortunately, for problems of growing dimensions AS never reached the best known solutions within the allowed 3,000 iterations, although it exhibited quick convergence to good solutions. • AS was able to find and improve the best solution found by a genetic algorithm for Oliver30 , a 30-city problem. • It had a performance similar or better than that of some general purpose heuristics with which it was compared . • These encouraging, although not exceptional, results stimulated a number of researchers to further study the ACO approach to optimization.

21. Let bi(t) (i=1, ..., n) be the number of ants in town i at time t. • Let m = ∑bi(t) be the total number of ants. • The length of pathij is the Euclidean distance dij between i and j (i.e. dij=[(x1i -x1j )2 + (x2i -x2j )2]1/2). • Let τij(t+1) be the intensity of trail on path ij at time t+1, given by the formula : • τij(t+1) (1 - ρ)τij(t) + ∆τij(t,t+1) • In the ANT-quantity model a constant quantity Q1 of pheromone is left on pathij every time an ant goes from i to j; in the ANT-density model an ant going from i to j leaves Q2 units of pheromone for every unit of length.

22. Therefore, in the ANT-quantity model • and in the ANT-density model we have • Finally in ANT-cycle model we have, after a complete tour ( n step ).

23. The ANT-density and ANT-quantity. • Initialize: Set t:=0 , Set an initial value τij(t) for trail intensity on every pathij Place bi(t) ants on every node I Set τij(t,t+1):= 0 for every i and j • Repeat until tabu list is full {this step will be repeated n times} 2.1 For i:=1 to n do {for every town} For k:=1 to bi(t) do {for every ant on town i at time t} Choose the town to move to, with probability pij .move the k-th ant to the chosen location. Insert the chosen town in the tabu list of ant k Set τij(t,t+1):= τij(t,t+1) + ∆τijk (t,t+1) computing ∆τijk (t,t+1) 2.2 Compute τij(t+1) and pij(t+1) according to equations (1) and (2) • Memorize the shortest path found up to now and empty all tabu lists • If not(End_Test) then set t:=t+1 set τij(t,t+1):=0 for every i and j goto step 2 else print shortest path and Stop {End_test is currently defined just as a test on the number of cycles}. Let bi(t) (i=1, ..., n) be the number of ants in town i at time t and let m = ∑ni=1bi(t)

24. The ANT-cycle algorithm • Initialize: Set t:=0 , Set an initial value τij(t) for trail intensity on every pathij , Place bi(t) ants on every node I , Set ∆τij(t,t+n):= 0 for every i and j • Repeat until tabu list is full {this step will be repeated n times} 2.1 For i:=1 to n do {for every town} For k:=1 to bi(t) do {for every ant on town i at time t} Choose the town to move to, with probability pij given by equation (2), and move the k-th ant to the chosen location Insert the chosen town in the tabu list of ant k 2.2 Compute ∆τijk (t,t+n) as defined in (5) 2.3 Compute ∆τij(t,t+n) = ∑mk=1∆τijk (t,t+n) 2.4 Compute the new values for τ ij(t+n) and pij(t+n) according to equations(1')and(2) • Memorize the shortest path found up to now and empty all tabu lists • If not(End_Test) then set t:=t+n set ∆τij(t,t+n):=0 for every i and j goto step 2 else print shortest path and Stop {End_test is currently defined just as a test on the number of cycles}.

25. (ACS) algorithm has been introduced by Dorigo and Gambardella (1996) to improve the performance of AS. • ACS is based on AS but presents some important differences. • First, the daemon updates pheromone trails offine : at the end of an iteration (global updating for exploitation). τij(t) (1 - ρ)τij(t) + ρ ∆τij(t) , where 0< ρ <1 , ∆τij(t)=1/L +.

26. Second : ACS ants perform only online step-by-step pheromone updates. τij(t) (1 -Ψ)τij(t) + Ψτ0 , where 0<Ψ<1. τ0 = (nLnn)-1, where n is the number of cities and Lnn is the length of a tour produced by the nearest neighbor heuristic. • ACS’s local update rule has the effect of making the visited arcs less and less attractive. (exploration)

27. third :Using a different decision rule. • Let q be a random variable uniformly distributed over [0,1], and q0Є[0,1] be a tunable parameter.Ant k located in node i choose the next node j Є Nki as the following: if q ≤ q0 then • otherwise, when q > q0 , where aij(t) is as As with α=1 • Exploration vs Exploitation ( with tuning the q0 ) If q ≤ q0 we Exploit some solutions , otherwise we Explore

28. procedure ACO Meta heuristic() while (termination criterion not satisfied) schedule activities ants generation and activity(); pheromone evaporation(); daemon actions(); end schedule activities end while end procedure

29. St¨utzle and Hoos [98, 97] have introduced Max-Min AS(MMAS), which is the same as AS,but: • (a) pheromone trails are only updated offlineby the daemon (the arcs that were used by the best ant in the current iteration receiveadditional pheromone) • (b) pheromone trail values are restricted to an interval [τmin;τmax] • (c) trails are initialized to their maximum value τmax.

30. To avoid stagnation, which may occur when some pheromone trails are close toτmax. We use the following update rule: • Δτijα ) τmax - τij)t( ( • That favor exploration instead of exploitation (like proportional selection in GA).

31. In ASrank, as was the case in MMAS, the only pheromone updates are performedby the daemon, which implements the following activities: • the m antsare ranked by tour length )L1)t( ;L2)t(;....... Lm )t((and the arcs that were visited byone of the first σ-1ants in the ranking receive an amount of pheromone proportionalto the visiting ant rank. • the arcs used by the ant that generated thebest tourfrom the beginning of the trial also receive additionalpheromone.

32. Data mining ( Discovery of Classification Rules). • The rule sets discovered by ant Miner is simpler • Smaller than C4.5 • Learning Fuzzy Control Rules. • ASGA ( Evolving ant colony parameters ). • Fitness F=∑ci . Fi • F1=1/(L+-1); • F2=e(-m/10n); • F3=e(-v/5n); • Ant Farm ( using neural networks to simulate ants behavior ant ).

33. Thanks

36. following the scent trail.

37. Smelling operation

38. Inspired from Bird flocking and fish schooling. • Interesting in discover: • changing direction suddenly. • Scattering and regrouping. • Population members are mass-less and volume-less thus terms acceleration and velocities are appropriate. • Updating the speed of particles (as mutation in EP). • Nearest-neighbor velocity matching and Craziness. • Inexpensive in terms of speed and memory.

39. Testing and early applications: • Train NN weights as effectively as back propagation error. • training weights found by PSO generalize from a training set to a test set better than gradient decent method. (e.g. spike wave forms NN 89% correct - PSO 92% correct on the test data) • Comparing to a benchmark of Genetic.( The extremely nonlinear schaffer f6 function). • Placing between GA and Ep.

40. Because of their behavioral resemblance to their insect counterparts, they have been named "army-ant" robots. • Small ,flexible systems.

41. Applications • Nuclear and hazardous waste cleanup with robotic "swarms." • Mining (including material removal and search-and-rescue). • Mine sweeping (both land and water). • Surveillance and sentry. • Planetary surface exploration and excavation.