CI Technologies

1. CITS7212 Computational Intelligence CI Technologies

2. Particle swarm optimisation • A population-based stochastic optimisation technique • Eberhart and Kennedy, 1995 • Inspired by bird-flocking • Imagine a flock of birds searching a landscape for food • Each bird is currently at some point in the landscape • Each bird flies continually over the landscape • Each bird remembers where it has been and how much food was there • Each bird is influenced by the findings of the other birds • Collectively the birds explore the landscape and share the resulting food

3. PSO • For our purposes • The landscape represents the possible solutions to a problem (i.e. the search space) • Time moves in discrete steps called generations • At a given generation, each bird has a position in the landscape and a velocity • Each bird knows • Which point it has visited that scored the best (its personal best pbest) • Which point visited by any bird that scored the best (the global best gbest) • At each generation, for each bird • Update (stochastically) its velocity v, favouring pbest and gbest • Use v to update its position • Update pbest and gbest as appropriate

4. PSO • Initialisation can be by many means, but often is just done randomly • Termination criteria also vary, but often termination is either • After a fixed number of generations, or • After convergence is “achieved”, e.g. if gbest doesn’t improve for a while • After a solution is discovered that is better than a given standard • Performance-wise • A large population usually gives better results • A large number of generations gives better results • But both obviously have computational costs • Clearly an evolutionary searching algorithm, but co-operation is via gbest, rather than via crossover and survival as in EAs

5. Ant colony optimisation • Another population-based stochastic optimisation technique • Dorigo et al., 1996 • Inspired by colonies of ants communicating via pheromones • Imagine a colony of ants with a choice of two paths around an obstacle • A shorter path ABXCD vs. a longer path ABYCD • Each ant chooses a path probabilistically wrt the amount of pheromone on each • Each ant lays pheromone as it moves along its chosen path • Initially 50% of ants go each way, but the ants going via X take a shorter time, therefore more pheromone is laid on that path • Later ants are biased towards ABXCD by this pheromone, which reinforces the process • Eventually almost all ants will choose ABXCD • Pheromone evaporates over time to allow adaptation to changing situations

6. ACO • The key points are that • Paths with more pheromone are more likely to be chosen by later ants • Shorter/better paths are likely to have more pheromone • Therefore shorter/better paths are likely to be favoured over time • But the stochastic routing and the evaporation means that new paths can be explored

7. ACO • Consider the application of ACO to the Traveling Salesman Problem • Given n cities, find the shortest tour that visits each city exactly once • Given m ants, each starting from a random city • In each iteration, each ant chooses a city it hasn’t visited yet • Ants choose cities probabilistically, favouring links with more pheromone • After n iterations (i.e. one cycle), all ants have done a complete tour, and they all lay pheromone on each link they used • The shorter an ant’s tour, the more pheromone it lays on each link • In subsequent cycles, ants tend to favour links that contributed to short tours in earlier cycles • The shortest tour found so far is recorded and updated appropriately • Initialisation and termination are performed similarly to PSO

8. Learning Classifier Systems Reading: M. Butz and S. Wilson, “An algorithmic description of XCS”, Advances in Learning Classifier Systems, 2001 O. Sigaud and S. Wilson, “Learning classifier systems: a survey”, Soft Computing – A Fusion of Foundations, Methodologies and Applications 11(11), 2007 R. Urbanomwicz and J. Moore, “Learning classifier systems: a complete introduction, review, and roadmap”, Journal of Artificial Evolution and Applications, 2009

9. LCSs • Inspired by a model of human learning: • frequent update of the efficacy of existing rules • occasional modification of governing rules • ability to create, remove, and generalise rules • LCSs simulate adaptive expert systems – adapting both the value of individual rules and the structural composition of rules in the rule set • LCSs are hybrid machine learning techniques, combining reinforcement learning and EAs • reinforcement learning used to update rule quality • an EA used to update the composition of the rule set

10. Algorithm Structure • An LCS maintains a population of condition-action-prediction rules called classifiers • the condition defines when the rule matches • the action defines what action the system should take • the prediction indicates the expected reward of the action • At each step (input), the LCS: • forms a match set of classifiers whose conditions are satisfied by the input • chooses the action from the match set with the highest average reward, weighted by classifier fitness (reliability) • forms the action set – the subset of classifiers from the match set who suggest the chosen action • executes the action and observes the returned payoff

11. Algorithm Structure • Simple reinforcement learning is used to update prediction and fitness values for each classifier in the action set • A steady-state EA is used to evolve the composition of the classifiers in the LCS • the EA executes at regular intervals to replace the weakest members of the population • the EA operates on the condition and action parts of classifiers • Extra phases for rule subsumption (generalisation) and rule creation (covering) are used to ensure a minimal covering set of classifiers is maintained

12. An Example Diagram taken from a seminar on using LCSs for fraud detection, by M. Behdad

13. LCS Variants • There are two main styles of LCS algorithms: • Pittsburgh-style: each population member represents a separate rule set, each forming a permanent “team” • Michigan-style: a single population of rules is maintained; rules form ad-hoc “teams” as required • LCS variants differ on the definition of fitness: • strength-based (ZCS): classifier fitness is based on the predicted reward of the classifier and not its accuracy • accuracy-based (XCS): classifier fitness is based on the accuracy of the classifier and not its predicted reward, thus promoting the evolution of accurate classifiers • XCS generally has better performance, although understanding when remains an open question

14. Fuzzy systems • Fuzzy logic facilitates the definition of control systems that can make good decisions from noisy, imprecise, or partial information • Zadeh, 1973 • Two key concepts • Graduation: everything is a matter of degree e.g. it can be “not cold”, or “a bit cold”, or “a lot cold”, or … • Granulation: everything is “clumped”, e.g. age is young, middle-aged, or old young 1 old middle-aged 0 age

15. Fuzzy Logic • The syntax of Fuzzy logic typically includes propositions ("It is raining", "CITS7212 is difficult", etc.), and Boolean connectives (and, not, etc.) • The semantics of Fuzzy logic differs from propositional logic; rather than assigning a True/False value to a proposition, we assign a degree of truth between 0 and 1, (e.g. v("CITS7212 is difficult") = 0.8) • Typical interpretations of the operators and and not are • v(not p) = 1 – v(p) • v(p and q) = min { v(p), v(q) } (Godel-Dummett norm) • Different semantics may be given by varying the interpretation of and(the T-norm). Anything commutative, associative, monotonic, continuous, and with 1 as an identity is a T-norm. Other common T-norms are: • v(p and q) = v(p)*v(q) (product norm) and • v(p and q) = max{v(p) + v(q) -1, 0} (Lukasiewicz norm)

16. Vagueness and Uncertainty • The product norm captures our understanding of probability or uncertaintywith a strong independence assumption • prob(Rain and Wind) = prob(Rain) * prob(Wind) • The Godel-Dummett norm is a fair representation of Vagueness: • If it’s a bit windy and very rainy, it’s a bit windy and rainy • Fuzzy logic provides a unifying logical framework for all CI Techniques, as CI techniques are inherently vague • Whether or not it is actually implemented is another question

17. Fuzzy Controllers • A fuzzy control system is a collection of rules • IF X [AND Y] THEN Z • e.g. IF cold AND ¬warming-up THEN open heating valve slightly • Such rulesare usually derived empirically from experience, rather than from the system itself • Attempt to mimic human-style logic • Granulation means that the exact values of any constants (e.g. where does cold start/end?) are less important • The fuzzy rules typically take observations, and according to these observations’ membership of fuzzy sets, we get a fuzzy action • The fuzzy action then needs to be defuzzified to become a precise output

18. Fuzzy Control • Applying Fuzzy Rules temperature Cold Right Hot no change heat heat -ve no change heat cool d(temperature) / dt zero no change cool cool +ve Image from http://www.faqs.org/docs/fuzzy/