Ameisenalgorithmen – Ant Colony Optimization

1. Ameisenalgorithmen –Ant Colony Optimization Lehrprobe zur Habilation, Barbara Hammer, AG LNM, Universität Osnabrück Ant Colony Optimization

2. Optimization • Given a graph with two specified vertices A and B, find a shortest path from A to B. • Given a set of cities and pairwise distances, find a shortest tour. • Given a sequence of amino acids of a protein, find the structure of the protein. • ‘Where is my manuscript for the talk, I put it on this pile of papers...’ General optimization problem: given f:Xℝ, find xεX such that f(x) is minimum  shortest path problem, polynomial  traveling salesperson problem, NP-hard  protein structure prediction problem, NP-hard  needle in a haystack, hopeless Ant Colony Optimization

3. Ant colony food nest Ant Colony Optimization

4. Ant Colony Optimization (ACO): a heuristic optimization method for shortest path and other optimization problems which borrows ideas from biological ants Ant Colony Optimization

5. Ant Colony Optimization Outline • History: ACO for shortest paths • Traveling salesperson and ACO-metaheuristic • Protein folding - state of the art ACO • General comments - where is my manuscript? Ant Colony Optimization

6. History: ACO for shortest paths … Ant Colony Optimization

7. History: ACO for shortest paths Goss et al. 1989, Deneuborg et al. 1990 experiments with Argentine ants: • ants go from the nest to the food source and backwards • after a while, the ants prefer the shortest path from the nest to the food source • stigmercy: • the ants communicate indirectly laying pheromone trails and following trails with higher pheromone • length gradient  pheromone will accumulate on the shortest path Dorigo et al. 1991 applications to shortest path problems food nest Ant Colony Optimization

8. History: ACO for shortest paths I:directed A first ACO for a simple shortest path problem: directed acyclic graph (V={0,...,N}, E={ij}), ant hill: 0, food source: N for all i: pi:=0; /*ant position init*/ si:=hungry; /*ant state init*/ for all i j: τij:=const; /*pheromone init*/ repeat for all i: ant_step(i); /*ant step*/ for all i j: τij := (1-ρ) τij ; /*evaporate pheromone*/ Ant Colony Optimization

9. History: ACO for shortest paths I:directed ant_step(i): if pi=N: si:=satisfied; if pi=0: si:=hungry; /*collect food/deliver food*/ if si=hungry: choose j with pij with probability τpi j/Σpij’τpij’ /*choose next step*/ update Δτpi j := ε; pi:=j; /*update pheromone*/ if si=satisfied: choose j with jpi with probability τjpi/Σj’piτj’pi update Δτjpi:= ε; pi:=j; /* reversed directions*/ Ant Colony Optimization

10. WC4 Barbara Marc WC5 Anja Dagmar Espresso 449a 339 WC3 Friedhelm 322 Fachschaft WC2 Rechner Astrid Zeitschriften WC Bibo RZ-Sekretariat Mensa Cafete Getraenke-automat RZ Toiletten History: ACO for shortest paths II:general ...a more complex undirected cyclic graph ... Ant Colony Optimization

11. History: ACO for shortest paths II:general ... Marc was not so happy with the result ... 449a 449a Ant Colony Optimization

12. History: ACO for shortest paths II:general for all i: pi:=0; /*ant position init*/ si:=( ); /*ant brain is empty*/ for all i-j: τi-j:=const; /*pheromone init*/ repeat for all i: construct_solution(i); for all i: global_pheromone_update(i); for all i-j: τi-j := (1-ρ) τi-j; /*evaporate*/ si:=hungry minibrain repeat for all i: ant_step(i); construct_solution(i): while pi≠N /*no solution*/ choose j with pi-j with probability τpi-j / Σpi-j’τpi-j’; pi:=j; append j to si; /*remember the trail*/ minibrain global_pheromone_update(i): for all j-j’ in si: Δτj-j’:= 1/length of the path stored in si; update according to the quality Ant Colony Optimization

13. WC4 Barbara Marc WC5 Dagmar Anja Espresso 449a 339 WC3 Friedhelm 322 Fachschaft WC2 Rechner Astrid Zeitschriften WC Bibo RZ-Sekretariat Mensa Cafete Getraenke RZ Toiletten History: ACO for shortest paths II:general Ant Colony Optimization

14. History: ACO for shortest paths init pheromone ti-j ; repeat for all ants i: construct_solution(i); for all ants i: global_pheromone_update(i); for all edges: evaporate pheromone; construct_solution(i): init ant; while not yet a solution: expand the solution by one edge probabilistically according to the pheromone; global_pheromone_update(i): for all edges in the solution: increase the pheromone according to the quality; Ant Colony Optimization

15. Traveling salesperson and ACO-metaheuristic … Ant Colony Optimization

16. Traveling salesperson Traveling salesperson problem (TSP): given n cities {1,...,N} and distances dij ≥0 between the cities, find a tour with shortest length, i.e. a permutation π:{1,…,N}{1,…,N} such that the length = Σidπ(i)π((i+1)mod N) is minimum classical NP-hard benchmark problem   A simple greedy heuristic: start somewhere and always add the closest not yet visited city to the tour Ant Colony Optimization

17. Traveling salesperson A init pheromone; repeat for all ants i: construct_solution(i); for all ants i: global_pheromone_update(i); for all edges: evaporate pheromone; C B D construct_solution(i): init ant; while not yet a solution expand the solution by one edge probabilistically according to the pheromone; key observation: a tour (ACDBA) decomposes into edges AC, CD, DB pheromone on the edges global_pheromone_update(i): for all edge in the solution: increase the pheromone according to the quality; Ant Colony Optimization

18. Traveling salesperson init: set τij:=const for all cities i≠j; repeat for all ants i: construct_solution(i); for all ants i: global_pheromone_update(i); for all edges i-j: evaporate pheromone; Ant Colony Optimization

19. Traveling salesperson construct_solution(i): set ant to a randomly chosen city; while not yet a solution: j=current city, expand by jk with probability = only valid tours are constructed α, β >0 control the mixture of the greedy heuristic and the pheromone following close cities are preferred global_pheromone_update(i); for all jk in the solution: Δτjk := const / length of the constructed tour short tours yield to most pheromone Ant Colony Optimization

20. Traveling salesperson Results for a 30 cities instance (10 runs, one hour) Results for larger instances (25000 constructed tours, best tour documented) Ant Colony Optimization

21. ACO-metaheuristic Optimization problem for ACO: • over a set of basic components C = {c1,...,cn} • partial solutions are subsets s in C • feasible (partial) solutions F in C • solutions S in C • cost function f for solutions Goal: • iteratively expand feasible partial solutions by components to reach a solution s with minimum f(s), • pheromone attached to each component ci guides the search edges ij partial tours tours which visit each city at most once and in consecutive order valid tours length of the tour Ant Colony Optimization

22. ACO-metaheuristic general ACO algorithm init pheromone τi=const for each component ci; repeat for all ants i: construct_solution(i); for all ants i: global_pheromone_update(i); for all pheromones i: evaporate: τi=(1-ρ)∙τi; construct_solution(i); init s={ }; while s is not a solution: choose cj with probability = expand s by cj; η is a heuristic value, α,β balance the heuristic/pheromone global_pheromone_update(i); for all cj in the solution s: increase pheromone: τj=τj+ const / f(s); Ant Colony Optimization

23. Protein folding - state of the art ACO … Ant Colony Optimization

24. Protein folding • Protein folding • given a sequence of amino acids s1…sn • where si in {1,0}, i.e. hydrophobic/polar • determine the structure of the protein • i.e. coordinates in a 2D rectangular lattice, such that • neighbored sequence entries are at neighbored positions • each position is occupied at most once • the number of 1-1 contacts in the 2D structure is maximized in the 2D-HP-model (Dill) Ant Colony Optimization

25. Protein folding Dill, 1985: the HP model preserves important information of the biological conformation Crescenzi et al. and Berger/Leighton, 1998: the problem is NP hard Shmygelska/Hoss, 2003: ACO 9 additional 1-1 contacts Ant Colony Optimization

26. S L R Protein folding start at the left end and iteratively fold one amino acid into a relative direction • R,S,R,R,L,L,R,S,R,R,L,R,L,L,R,R,S,R in {R,S,L}length-2 basic components: an element in {i-R,i-S,i-L} represents the local structural motif at position (i-1,i,i+1) pheromone values τi-D , i = 2.. length-1, D = L,S,R ... R S R Ant Colony Optimization

27. Protein folding components: local structural motifs i-D partial solutions: subsets of local structural motifs feasible partial solutions: sequences of consecutive structural motifs without overlap of the amino acids in the 2D lattice solutions: final folds cost function to be maximized: number of 1-1 contacts in the 2D lattice init pheromone τi-D=const for each tuple i-D; repeat for all ants i: construct_solution(i); for the best ants i: optimize_solution(i); for the best ants i: global_pheromone_update(i); for all pheromones i-D: evaporate: τi-D=(1-ρ)∙τi-D; daemon action: local optimization elitism Ant Colony Optimization

28. Protein folding • construct_solution(i); • init s = { }; • while s is not a solution: position j • choose a local structural motif j-D with probability proportional to • 0 if the position is already occupied or the sequence gets trapped • proportional to τj-Dα∙ηj-Dβ • expand s by the chosen motif; feasibility η is related to the number of 1-1 contacts of this motif optimize_solution(i); perform a fixed number of feasible and improving substitutions of local structural motifs at random global_pheromone_update(i); for all local structural motifs in a solution: τj-D := τj-D + number of 1-1 contacts in the solution / const; Ant Colony Optimization

29. Protein folding Best reported results for different size instances GA: genetic algorithm EMC: evolutionary algorithm + Monte Carlo methods MSOE: Monte Carlo including overlapping conformations PERM: iterated heuristic growing method Ant Colony Optimization

30. General comments - where is my manuscript … Ant Colony Optimization

31. General comments ACOs: nice, powerful, and robust metaheuristic for NP hard, possibly non -static optimization problems, the solutions of which decompose into single components Applications for (* = state of the art results for some settings): • quadratic assignment problems * • vehicle routing * • sequential ordering * • shortest common supersequence * • scheduling * • graph coloring and partitioning • telecommunication networks and routing * • ... more info on ACOs: • Swarm Intelligence, From Natural to Artificial Systems, E.Bonabeau, M.Dorigo, G.Theraulaz, Santa Fe, 1999 • http://iridia.ulb.ac.be/~mdorigo/ACO/ • Duft der Daten, Der Spiegel, November 13, 2000 • Swarm smarts, Scientific American, March 2000 Ant Colony Optimization

32. General comments Are ACO’s better than other metaheuristics for general optimization problems? No free lunch theorem (Macready/Wolpert) In the mean, no optimization algorithm is to be preferred! Precise: Assume A and B are finite, B is totally ordered, F is a set of functions from A to B which is closed under permutation, H is a (randomized) search heuristic. Then the expected time to reach the first optimum is independent of H. ... so it might take a while until the ants find my manuscript, but they’ll find it. Ant Colony Optimization

33. Jawoll! Gegen Stellenkürzungen im Hochschulbereich! Rettet die Bildung! Ant Colony Optimization

34. Ant Colony Optimization

35. ACO-metaheuristic ACO-metaheuristic • applicable to general optimization problems as stated above • robust and tolerant to changes, e.g. in non-static problems • yields state of the art solvers for some problems with modifications • daemon actions: improve the found solutions using local search (e.g. k-opt for TSP) • elitism: update pheromone only for the (local or global) best ants • ... Ant Colony Optimization

36. Protein folding – state of the art ACO • optimize_solution(i); • perform a fixed number of feasible and improving search moves randomly chosen from the following possibilities • substitution of a single motif: substitute one motif i-D by a different one i-D’ • substitution of a sequence of motifs: substitute all motifs within randomly chosen positions by different motifs • long-range moves: substitute one local motif and refold the two ends to feasible settings, whereby the respective original motifs are preserved if possible LRSSRLS LRSLRLS LRSSRLS LSRLRLS LRSSRLS LRSLRLS SLSLRLR Ant Colony Optimization