Dynamically adapting parasite virulence to combat coevolutionary disengagement

1. Dynamically adapting parasite virulence to combat coevolutionary disengagement John Cartlidge: ALife XI, Winchester, UK

2. Synopsis • Disengagement in coevolutionary systems • Review Reduced Virulence (RV) • Analysis of RV in Counting Ones domain • Present Dynamic Virulence (DV), a novel method for adapting Virulence online • Summary/Conclusions John Cartlidge: ALife XI, Winchester, UK

3. Disengagement • Competitive Coevolutionary Systems • Relative fitness assessment through self-play • Fitness varies as opponents vary in ability • Relativity leads to Disengagement • Occurs when one population gets the “upper hand” • Can’t discriminate individuals  no selection pressure • Occurs when competitors are badly matched • Suits of armour and nuclear weapons • There must be no outright winner John Cartlidge: ALife XI, Winchester, UK

4. Reduced Virulence (RV) • Cartlidge, J. & Bullock, S. (2002, 2004) • Reward competitors that sometimes lose RV Fitness Transform f(x,v) = 2x ∕ v – x2∕ v2 virulence: 0.5 ≤ v ≤ 1.0 relative score: x John Cartlidge: ALife XI, Winchester, UK

5. RV: An illustrative Example • Selection only. No mutation. Linear fitness ranking • Population B has an innovation (20) not found in A • Trade-off between engagement and innovation loss V = 1 (standard) V = 0.75 V = 0.5 Selection drives pop B to 20 causing disengagement Pop B drops genotype 20 and remains engaged at 19 Lots of innovation loss as populations move to 12 John Cartlidge: ALife XI, Winchester, UK

6. Symmetry • Mutation introduces genetic novelty • Symmetric system with unbiased mutation profile • Populations have equal chance of +/– mutation • Neither population has an advantage John Cartlidge: ALife XI, Winchester, UK

7. Asymmetry • Here population B has a favourable mutation bias • A finds it harder to discover +ve/beneficial genetic innovations • Disengagement is exacerbated by asymmetry • In genetic representations, genotype-phenotype mappings, genetic operators, interaction rules, location in genotype space, etc. John Cartlidge: ALife XI, Winchester, UK

8. Couting Ones • Watson & Pollack, GECCO 2001 • Two populations of binary strings • Goal: evolve as many 1s as possible • Asymmetrical bias controlled by varying mutation bias of one population (parasites) • When is it best to reduce virulence? John Cartlidge: ALife XI, Winchester, UK

9. Parasite Bias / Asymmetry 0.5 0.7 0.8 0.9 1.0 0.6 Maximums Engagement Parasite virulence ‘Sweet-Spot’ Host virulence Virulence ‘Sweet-Spot’ • Low bias requires high virulence for both populations • As bias increases, want progressively lower parasite V John Cartlidge: ALife XI, Winchester, UK

10. Choosing RV Value • Problem: • How do we know a priori what the asymmetry is likely to be? • Is asymmetry is likely to remain fixed? • Solution: • Adapt virulence dynamically during runtime John Cartlidge: ALife XI, Winchester, UK

11. Dynamic Virulence (DV) • Reinforcement learning approach: • Value(t+1)  Value(t) + LearningRate [Target(t) – Value(t)] • Each generation, t, update virulence, Vt • ∆Vt = ρ(1 − Xt ∕φ) (1) • Xt: Mean relative score of population at time t • φ: Target mean relative score of population • ρ: Acceleration (rate of change of virulence) • Μt = μΜt-1 + (1−μ)∆Vt (2) • μ: Momentum, Μ0 = V0 • if μ = 0, then t, Μt = ∆Vt no momentum • if μ = 1, then t, Μt = V0 fixed virulence • Vt+1 = Vt+Μt (3) • 0 ≤ φ, ρ, μ ≤ 1 John Cartlidge: ALife XI, Winchester, UK

12. 15 runs. Mean value of parameter in population each generation. Bias varying during each evaluation Evolving φ, ρ, μ 30 runs. Mean value of parameter in population each generation. Bias fixed for each evaluation John Cartlidge: ALife XI, Winchester, UK

13. 0.5 0.6 0.7 0.8 0.9 1.0 Parasite Bias / Asymmetry DV Performance • Performance of DV in the Counting Ones domain • DV Parameters: φ = 0.56; ρ = 0.0125; μ = 0.3 • 180/180 successful runs. 31/135,000 disengaged generations • Compare with maximum virulence • 79/180 successful runs. 68,900 disengaged generations Dynamic Virulence Fixed Virulence Fixed Virulence Successful runs using fixed virulence (total 180 runs) 0.5 0.6 0.7 0.8 0.9 1.0 Parasite Bias / Asymmetry John Cartlidge: ALife XI, Winchester, UK

14. DV in Action John Cartlidge: ALife XI, Winchester, UK

15. Lessons for epidemiology? • Can we use DV for modelling virulence in natural systems? • Can we translate ideas of RV to the natural world for control of infectious diseases? • Rather than attack parasites and encourage an arms-race, creating ‘super-bugs’, can we take a reduced virulence approach? • E.g.: ‘Scientists create GM mosquitoes to fight malaria and save thousands of lives’ (Guardian 2005) • ‘Plan to breed and sterilize millions of male insects’ • Project ‘almost ready for testing in wild’ John Cartlidge: ALife XI, Winchester, UK

16. Summary / Conclusions • Disengagement is problematic and is exacerbated by asymmetry • Reducing virulence helps to promote engagement • As asymmetry increases, virulence should fall • Its hard to know a priori what virulence level to set • DV is able to adapt virulence during evolution to find the best value • DV has been shown to vastly outperform fixed virulence (and standard virulence) in the Counting Ones domain John Cartlidge: ALife XI, Winchester, UK

17. Further Reading • Cartlidge & Bullock (2002) CEC, p.1420, IEEE Press • Cartlidge & Bullock (2003) ECAL, p.299, Springer Verlag • Cartlidge & Bullock (2004) Evolutionary Comp., 12, p.193 • Cartlidge (2004) PhD Thesis, University of Leeds Dr John Cartlidge, Research Associate University of Central Lancashire jpcartlidge@uclan.ac.uk John Cartlidge: ALife XI, Winchester, UK