Nature-Inspired Algorithms

1. Nature-Inspired Algorithms Artificial Intelligence Department of Industrial Engineering and Management Cheng Shiu University

2. Outline Estimation of Distribution Algorithms Univariate Marginal Distribution Algorithm Compact GA Population-based Incremental Learning Swarm Intelligence Ant Colony System Particle swarm optimization Artificial Immune Systems

3. Estimation of Distribution Algorithm (EDA) Use probabilistic models for recombination Learn and sample that probabilistic models to generate new solutions Selection and replacement strategy of GA can be used Allow adaptation and improve expressiveness

4. EDA (Estimation of Distribution Algorithm) Univate Univariate Marginal Distribution Algorithm(UMDA) (Mühlenbein et.al, 1996) Population Based Incremental Learning (PBIL) (Baluja, 1994) Compact Genetic Algorithm (CGA) (Harik et.al,1998) Multivariate Extended Compact Genetic Algorithm (ECGA) (Harik 1999) Bayesian Optimization Algorithm (BOA) (Pelikan et al.,2000) Estimation of Bayesian Networks Algorithm (EBNA) (Etxeberria et al., 1999) Factorized Distribution Algorithm (FDA) (Mühlenbein et al.,1999) Learning Factorized Distribution Algorithm (LFDA) (Mühlenbein et al.,1999)

5. Different EDA approaches Independent variables The easiest way to calculate an estimate of the required probability distribution is to consider all the variables in a problem as 1D and independent from each other. Then the joint probability distribution becomes the product of the marginal probabilities of the n genes (variables), i.e. where i indexes genes, n is the # genes in a chromosome; xi is the actual value of the gene and l is the generation index

6. Different EDA approaches Independent variables ‘Univariate Marginal Distribution Algorithm (UMDA)’ ‘Population Based Incremental Learning (PBIL)’ ‘Compact Genetic Algorithm (CGA)’ Bivariate Dependencies … Multiple Dependencies … + Discrete vs Continuous versions We only talk about the discrete case with independent variables

8. General Notation EDA represents a solution as a set of value taken by a set of random variable.

9. Estimation of Probability distribution

10. Simple Univariate Estimation of Distribution Algorithm

11. Simple univariate EDA (UMDA)

12. Note It is not guaranteed that the above algorithm will give optimum solution for the graph colouring problem. The reason is obvious. The chromosome representation of GCP has dependency. i.e. node 1 taking black colour depends upon the colour of node 2. But univariate EDAs do not assume any dependency so it may fail. However, one could try

13. Univariate EDA Model

14. Bivariate EDA Model

15. Multivariate EDA Model

16. GA to EDA

17. UMDA Maintaining a population of individuals. Then it applies a selection method to create a new population. Based on the new population, the frequencies of each gene are computed and are used to generate a new population of individuals. This generation step is a kind of population-wise crossover operator and replaces pairwise crossover operator of the traditional GA.

18. Uses a probability vector p=(p1,p2,…,pn), pi is the probability of 1 in i th position Compute p from selected individuals Generate 1 in the position i with prob. pi Approximates the behavior of uniform crossover of GA Problem: Joint probability may be 0 due to some pi=0 and may converge to local optimal

19. Compact genetic algorithm (cGA) Representing the population as a probability distribution over the set of solutions Operationally equivalent to the order-one behavior of the simple GA with uniform crossover.

21. PBIL The population-based incremental learning (PBIL) method was originally proposed by Baluja [Baluja, 1994]. This approach is a combination of genetic algorithms and the hill-climbing or gradient ascent method. The object of the PBIL is to create a probability vector which, when sampled, reveals high quality solution vectors with high probability with respect to the available knowledge of the search space. The vector of probabilities is updated at each generation. The change in the probability vector is towards the better individuals of the population and away from the worst individuals.

22. PBIL (Cont.) Parameters population size (number of samples generated) learning rate (LR) Number of winners To update probabilities vector The fittest chromosomes (Winners) are adopted: all population in the next approaches winners Dpi = h [Ebest(chromosomesji)-pi] Dpi:increment to vector’s ith element Ebest(chromosomesji):the mean value of the ith chromosome bit of all winners h:Learning rate

23. PBIL Workflow Vector of Probabilities, V Generate sample chromosomes from V Find fittest chromosome(s) Increase probability of getting better chromosomes by moving V closer to current fittest chromosome(s). Exploration & Exploitation

24. PBIL (Cont.) Create a vector whose length is the same as the required chromosome and whose elements are the probabilities of a “1” in the corresponding bit position of the chromosome. This vector is intialised with 0.5 in all positions. Generate a number of samples from the vector where the probability of a 1 in each bit position of each vector is determined by the current probability vector. Find the fittest chromosomes from this population. Amend the probability vector’s elements so that the probability of a “1” is increased in the positions in which the fittest chromosomes have a 1.

25. PBIL Pseudo code

26. Pseudocode of PBIL for (j = 1 : l) P(j) = 0.5 while not done Do for (i = 1: n ) for (j = 1:l ) if (randm(0,1) < P[j] ) Ii(j) = 1 else Iii(j) = 0 evaluate Ii with respect fitness (Ii) Find winner1:m = best I 1:m up to fitness(Ii) for (j = 1 : l) for (k =1….m) P(j) = p(j) (1-h)+h (mean(I1:m)) end loop

27. Limitation of Applying PBIL Evolution stops while Probabilities = 0/1 Winners adoption via features of problems Open Question:winners adoption more:interaction of winners’ bits, worse exploitation few:Local fitness optimum Parameter: Learning rate High:worse exploration Low:worse exploitation

28. The Cooperative PBIL Algorithm Genomic changes when meeting paradoxes (Ben Jacob). Uses environmental input to direct changes in V. Winner plays against others and we note which bits are actually used in the competition. Change only these bits in V

29. The cooperative PBIL Uses environmental inputs to direct changes in Probabilities vector V. Randomly producing Vector of Probabilities, V Generate sample chromosomes from V Find fittest chromosome(s): Rank selection Winners play against others (randomly) Bits actually used are noted in the game Change only these bits in V

30. Pseudocode of Cooperative PBIL for (j = 1 : l) P(j) = 0.5 while not done Do for (i = 1 : n ) for (j = 1 : l ) if (randm(0,1) < P[j] ) Ii(j) = 1 else Iii(j) = 0 evaluate Ii with respect fitness (Ii) find winner1:m = best I 1:m up to fitness(Ii) choose Ii as opponents1:q randomly simulate winners compete with opponents for (j = 1 : l) Use(j)=0 if winner1:m(j) is used Use(j)=1 for (j = 1 : l) if Use(j)=1 P(j) = p(j) (1-h)+h (mean(I1:m)) else P(j) = p(j) end loop

31. Swarm Swarms can achieve things no single individual could. Why do animals swarm? Defense against predators Enhanced predator detection Minimizing chance of capture Enhanced foraging success Better chances to find a mate Decrease of energy consumption Natural swarm suggests: No central control Only simple rules for each individual Emergent phenomena Self-organization

32. Swarm Intelligence Swarm intelligence (SI) is an artificial intelligence technique based around the study of collective behavior in decentralized, self-organized systems. The expression "swarm intelligence" was introduced by Beni & Wang in 1989, in the context of cellular robotic systems SI systems are typically made up of a population of simple agents interacting locally with one another and with their environment. Although there is normally no centralized control structure dictating how individual agents should behave, local interactions between such agents often lead to the emergence of global behavior. Examples of systems like this can be found in nature, including ant colonies, bird flocking, animal herding, bacteria molding and fish schooling.

33. An example: Bird flocking “Boids” model created by Rynolds in 1987 Boids = “bird-oid” objects (also schooling fish) Model: biologically and physically sound Individual has only local knowledge Has certain cognitive capabilities Is bound by the laws of physic Only three simple rules …

34. Rule 1: Collision Avoidance Avoid collisions with neighboring boids

35. Rule 2: Velocity Matching Match the velocity of neighboring boids

36. Rule 3: Flock Centering Stay near neighboring boids

37. Define the neighborhood Model the view of a bird Only local knowledge, only local interaction Affects the swarm behavior (fish vs. birds)

38. Application of Swarm Intelligence The U.S. military is investigating swarm techniques for controlling unmanned vehicles. NASA is investigating the use of swarm technology for planetary mapping. A 1992 paper by M. Anthony Lewis and George A. Bekey discusses the possibility of using swarm intelligence to control nanobots within the body for the purpose of killing cancer tumors. Artists are using swarm technology as a means of creating complex interactive environments. Disney's The Lion King was the first movie to make use of swarm technology (the stampede of the wildebeests scene). The movie "Lord of the Rings" has also made use of similar technology during battle scenes. Swarm technology is particularly attractive because it is cheap, robust, and simple.

39. Particle Swarm Optimization Particle swarm optimization or PSO is a global optimization algorithm for dealing with problems in which a best solution can be represented as a point or surface in an n-dimensional space. Hypotheses are plotted in this space and seeded with an initial velocity, as well as a communication channel between the particles. Particles then move through the solution space, and are evaluated according to some fitness criterion after each time step. Over time, particles are accelerated towards those particles within their communication grouping which have better fitness values. The main advantage of such an approach over other global minimization strategies such as simulated annealing is that the large number of members that make up the particle swarm make the technique impressively resilient to the problem of local minima.

40. PSO Idea: Use this to optimize continuous functions Function is evaluated at each time step for the agent’s current position Each agent “remembers” personal best value of the function (pbest) Globally best personal value is known (gbest)

41. PSO (Cont.) Both points are attracting the agent

42. PSO (Cont.) Agents overshoot the target Balance of exploration and convergence Metaheuristic Formula for one agent in one dimension

43. Steps of Particle Swarm Optimization

44. Pseudo code of PSO

45. Ant Colony Optimization Ant colony optimization or ACO is a metaheuristic optimization algorithm that can be used to find approximate solutions to difficult combinatorial optimization problems. Artificial ants build solutions by moving on the problem graph and they, mimicking real ants, deposit artificial pheromone on the graph in such a way that future artificial ants can build better solutions.

46. Ant Colony System (ACS) An ACO algorithm is based on the result of low level interaction among many cooperating simple agents that are not explicitly aware of their cooperative behavior. Each simple agent is called ant and the ACO algorithm (a distributed algorithm) is based on a set of ants working independently and cooperating in a common problem solving activity. The ant colony optimization is a population-based approach in which a set of artificial ants cooperate to build solutions using an indirect form of communication mediated by deposited pheromone.

47. The ants Can explore vast areas without global view of the ground. Can find the food and bring it back to the nest. Will converge to the shortest path. How can they manage such great tasks ? By leaving pheromones behind them. Wherever they go, they let pheromones behind here, marking the area as explored and communicating to the other ants that the way is known. Double Bridge experiment

48. Double Bridge experiment

49. Pheromone Trails Individual ants lay pheromone trails while travelling from the nest, to the nest or possibly in both directions. The pheromone trail gradually evaporates over time. But pheromone trail strength accumulate with multiple ants using path.

50. Pheromone Trails continued Distances are 1 or 0.5 in graph. Consider discretized intervals t=0, 1, 2. Suppose that 30 new ants come from A to B every time unit and 30 from E to D. Each ant walks at a speed of 1 per time unit, and that an ant lays down at time t a pheromone of intensity 1 while walking and that this evaporates completely and instantaneously in the middle of time intervals (t+1, t+2). At t=0, there is no trail yet but 30 ants are in B and 30 in D. Their choice of direction is random. Therefore, on average 15 will go to H and 15 to C. At t=1, the 30 new ants come to B from A and find a trail of intensity 15 on the path that leads to H, laid by the 15 ants that went that way from B, and a trail of intensity 30 on the path to C obtained as the sum of the trail laid by the 15 ants that went that way from B and by the 15 ants that reached B coming from D via C. The probability of choosing a path is therefore biased, so that the expected number of ants going towards C will be double that of those going towards H. The same is true for the new 30 ants in D which came from E. Process continues until all ants choose the shortest path. The idea is that if at a given point an ant has to choose among different paths, those which were heavily chosen by preceding ants are chosen with higher probability. This is an example of an autocatalytic process.Distances are 1 or 0.5 in graph. Consider discretized intervals t=0, 1, 2. Suppose that 30 new ants come from A to B every time unit and 30 from E to D. Each ant walks at a speed of 1 per time unit, and that an ant lays down at time t a pheromone of intensity 1 while walking and that this evaporates completely and instantaneously in the middle of time intervals (t+1, t+2). At t=0, there is no trail yet but 30 ants are in B and 30 in D. Their choice of direction is random. Therefore, on average 15 will go to H and 15 to C. At t=1, the 30 new ants come to B from A and find a trail of intensity 15 on the path that leads to H, laid by the 15 ants that went that way from B, and a trail of intensity 30 on the path to C obtained as the sum of the trail laid by the 15 ants that went that way from B and by the 15 ants that reached B coming from D via C. The probability of choosing a path is therefore biased, so that the expected number of ants going towards C will be double that of those going towards H. The same is true for the new 30 ants in D which came from E. Process continues until all ants choose the shortest path. The idea is that if at a given point an ant has to choose among different paths, those which were heavily chosen by preceding ants are chosen with higher probability. This is an example of an autocatalytic process.

51. From biological ants to ant-agent Distributed process: local decision-taking Autonomous Simultaneous Macroscopic development from microscopic probabilistic decisions Problem: adaptation to reality Solution: Pheromone upgrade: evaporation. Ant aging: after a given time, ants are tired and have to come back to the nest. 2 different pheromones : away (from nest) and back (from source of food).

52. Algorithm of Ant System

53. Algorithm of Ant System

54. Ant Colony Optimisation Algoirithms: Basic Ideas Ants are agents that: Move along between nodes in a graph. They choose where to go based on pheromone strength (and maybe other things) An ant’s path represents a specific candidate solution. When an ant has finished a solution, pheromone is laid on its path, according to quality of solution. This affects behaviour of other ants by `stigmergy’ …

55. E.g. A 4-city TSP

56. E.g. A 4-city TSP

57. E.g. A 4-city TSP

58. E.g. A 4-city TSP

59. E.g. A 4-city TSP

60. E.g. A 4-city TSP

61. E.g. A 4-city TSP

62. E.g. A 4-city TSP

63. The ACO algorithm for the TSP We have a TSP, with n cities. 1. We place some ants at each city. Each ant then does this: It makes a complete tour of the cities, coming back to its starting city, using a transition rule to decide which links to follow. By this rule, it chooses each next-city at random, but biased partly by the pheromone levels existing at each path, and biased partly by heuristic information. 2. When all ants have completed their tours. Global Pheromone Updating occurs. The current pheromone levels on all links are reduced (I.e. pheromone levels decay over time). Pheromone is lain (belatedly) by each ant as follows: it places pheromone on all links of its tour, with strength depending on how good the tour was. Then we go back to 1 and repeat the whole process many times, until we reach a termination criterion.

64. The transition rule

65. Global pheromone update

66. Definition of Artificial Immune System (AIS) AIS are adaptive systems inspired by theoretical immunology and observed immune functions, principles and models, which are applied to complex problem domains [De Castro and Timmis,2002]

67. Scope of AIS ·        Pattern recognition; ·        Fault and anomaly detection; ·        Data analysis (data mining, classification, etc.); ·        Agent-based systems; ·        Scheduling; ·        Machine-learning; ·        Self-organization; ·        Autonomous navigation; ·        Autonomous control; ·        Search and optimization methods; ·        Artificial life; and ·        Security of information systems. ·        Pattern recognition; ·        Fault and anomaly detection; ·        Data analysis (data mining, classification, etc.); ·        Agent-based systems; ·        Scheduling; ·        Machine-learning; ·        Self-organization; ·        Autonomous navigation; ·        Autonomous control; ·        Search and optimization methods; ·        Artificial life; and ·        Security of information systems.

68. From a computational perspective: Unique to individuals Distributed Imperfect Detection Anomaly Detection Learning/Adaptation Memory Feature Extraction Diverse ..and more Robust Scalable Flexible Exhibit graceful degradation Homeostatic

69. Immunology Immune System a complex system of cellular and molecular components having the primary function of distinguishing self from not self and defense against foreign organisms or substances (Dorland's Illustrated Medical Dictionary) Immune system was evolutionary selected as a consequence of its first and primordial function to provide an ideal inter-cellular communication pathway (Stewart) The immune system is a cognitive system whose primary role is to provide body maintenance (Cohen)

70. Classical Immunity A complex system that enables a mechanism: certain dangers recognized. The purpose of the immune system is defence The innate immune system Innate is the first line of defence. Germ line encoded (passed from parents) and is quite ‘static’ (but not totally static) To destroy may pathogens on first encounter. The adaptive immune system Adaptive (acquired). Somatic (cellular) and is acquired by the host over the life time. Very dynamic. To learn to respond appropriately to attacks. These two interact and affect each other

71. Multiple layers of the immune system

72. Innate Immunity May take days to remove an infection, if it fails, then the adaptive response may take over Macrophages and neurophils are actors Bind to common (known) things. This knowledge has been evolved and passed from generation to generation. Other actors such as TLR’s and dendritic cells (next lecture) are essential for recognition

73. A Lymph Node Goldsby et al., Immunology, 5th edition, 2003

74. Adaptive Immunity Lymphocytes Carry antigen receptors that are specific They are produced in the bone marrow through random re-arrangement B and T Cells are the main actors of the adaptive immune system

75. B Cell Pattern Recognition B cells have receptors called antibodies The immune recognition is based on the complementarity between the binding region of the receptor and a portion of the antigen called the epitope. Recognition is not just by a single antibody, but a collection of them Learn not through a single agent, but multiple ones

76. B Cell Receptor V-region: antigenic recognition and binding C-region: effector functions Heavy and light chains VDJC immunoglobulin (encoded by genes) Genes direct the development of an individual. Encode proteins, which are made up of amino acids

77. Processes within the Immune System (very basically) Negative Selection Censoring of T-cells in the thymus gland of T-cells that recognise self Defining normal system behavior Clonal Selection Proliferation and differentiation of cells when they have recognised something Generalise and learn Self vs Non-Self

78. Clonal Selection

79. Clonal Selection

80. Clonal Selection Each lymphocyte bears a single type of receptor with a unique specificity Interaction between a foreign molecule and a lymphocyte receptor capable of binding that molecule with high affinity leads to lymphocyte activation Effector cells derived from an activated lymphocyte bear receptors identical to those of parent cells Lymphocytes bearing self molecules are deleted at an early stage

81. Affinity Maturation Responses mediated by T cells improve with experience Mutation on receptors (hypermutation and receptor editing) During the clonal expansion, mutation can lead to increased affinity, these new ones are selected to enter a ‘pool’ of memory cells Can also lead to bad ones and these are deleted

82. Artificial Immune Systems Interest Features of IS Pattern Recognition Memory Immune Network Learning (via previous attacks) Diversity (with high mutation) Artificial Immune System A computational systems based on metaphors of biological immune system.

83. Dynamics of Immune System Immune Network: Creation of antibody relies on the others interactive antibodies. Clone the better:

84. Clonal Selection Algorithm Randomly initialize a population with the binary-coded antibodies; Evaluate the affinity of each antibody; Determine the winners based on an affinity rank; Clone these winners - high affinity winner clones more than the lower one; Submit the clones to a hyper-mutation scheme, mutation rate is proportional to the affinity; Replace the antibodies of the population randomly by the clones;

85. AIS VS. GA Initializing population randomly. Sufficient information is unnecessary Identity can act with/against environment Memory vs. Exploitation Exploration Individuals in population are not isolated to one another AIS maintain much more diversity in population (Phenotype and Genotype)

86. Parameters of AIS Numbers of winners (m) Clone regimes Number of clones: wj=a*(m-j)+b Mutation rate (m): Hypermutation Mutation rate for clone: (1+(f1-fj)*m Population size (P) Parameters influence search quality/ performance, and diversity of population