A . Davydov , D . Sokolov , F . Tsarev Scientific advisor – prof. A . Shalyto 2008

1. Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem A. Davydov, D. Sokolov, F. Tsarev Scientific advisor – prof. A. Shalyto 2008

2. Automata based programming • Proposed in Russia in 1991 • Software systems are developed as well as automationof engineering (and other) process • Control system is a system of interacting finite state machines • States • Eventsand input variables • Output events • Finite state machine • System of finite state machines A. Davydov, D. Sokolov, F. Tsarev Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem

3. Method advantages • Effective for systems with complicated behavior • Formal and understandable behavior description • Automatic code generation from transition diagrams • Possibility of program verification • Project documentation A. Davydov, D. Sokolov, F. Tsarev Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem

4. Problem • The general difficulty in automata based programming is automata construction • In a majority of cases automata are designed manually • Heuristic automata construction is often very hard and time-consuming • The solution isto use genetic programming forautomaticfinite state machines generation A. Davydov, D. Sokolov, F. Tsarev Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem

5. “Artificial ant” problem • Torus – 32x32 • 89 cellswith food • 200 steps • The locations of food are fixed • Goal is to create an ant which can eat all food in 200 steps A. Davydov, D. Sokolov, F. Tsarev Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem

6. What ant can do? • Determine is there food in front of it • Do one of these actions in one step: • step forward and eat food if it is in the next cell • turn left • turn right • nothing A. Davydov, D. Sokolov, F. Tsarev Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem

7. Finite state machine – method of ant description • Automaton with transition’s action (Mealy). Automaton with transition’s action (Moore). System (couple) of automata which interact by nesting • It is easy to create automata with many states • There is Mealy automaton with 5 states which allow to eat 81 units of food A. Davydov, D. Sokolov, F. Tsarev Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem

8. Genetic Algorithms – 1 • Jefferson D., Collins R., Cooper C., Dyer M., Flowers M., Korf R., Taylor C., Wang A. The Genesys System. 1992. • Coding automata to bit string • Automaton with 13 states which solves this problem A. Davydov, D. Sokolov, F. Tsarev Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem

9. Genetic Algorithms – 2 • Angeline P. J., Pollack J. Evolutionary Module Acquisition // Proceedings of the Second Annual Conference on Evolutionary Programming. 1993. • Coding automata to bit string + “freezing” • Automaton with 11 states which solves this problem in 193 steps A. Davydov, D. Sokolov, F. Tsarev Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem

10. Genetic Algorithms – 3 • Chambers L. D. Practical Handbook of Genetic Algorithms, Volume 3, Chapter 26, 6 – Algorithms to Improve the Convergence of a Genetic Algorithm with a Finite State Machine Genome. CRC Press, 1999. • Coding automata to bit string + canonical representation • Automaton with 8 states which solves this problem A. Davydov, D. Sokolov, F. Tsarev Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem

11. Genetic Algorithms – 4 • Tsarev F., Shalyto A.About construction of automata with minimal number of states for “Artificial ant” problem /Proceedings of X international conference of soft compitation and measurement. SPbETU “LETI”. Vol.2, 2007, pp.88–91. (in Russian) • Genetic programming A. Davydov, D. Sokolov, F. Tsarev Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem

12. Mealy automaton with 7 states • Two automata with 7 seven were created after exploring over 160 and 230 millions of automata A. Davydov, D. Sokolov, F. Tsarev Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem

13. Problem • Only Mealy automata are discussed in all papers • Goal of this work – creating Moore automata and systems of interacting Mealy automata A. Davydov, D. Sokolov, F. Tsarev Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem

14. Approach proposed – 1 • Island genetic algorithm • Automaton = start state + description of states + internal automaton • State = two transitions + action in state (only for Moore automata) • Transitions = number of state to which this transition leads + action (only for Mealy automaton) A. Davydov, D. Sokolov, F. Tsarev Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem

15. Approach proposed – 2 class Automaton { Transition[][]transitions; int initialState; Automaton nestedAutomaton; char[] stateAction; // for Moore automata } A. Davydov, D. Sokolov, F. Tsarev Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem

16. Initial generation creation • A predefined number of automata (systems of automata) is randomly generated • All automata consists of the same number of states A. Davydov, D. Sokolov, F. Tsarev Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem

17. Next generation creation – 1 • Elitism • Fixed part of individuals move to next generation • For other individuals – tournament strategy. Choose two pairs of individual, mutation or crossover with some probability is applied to best individuals of each pair A. Davydov, D. Sokolov, F. Tsarev Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem

18. Next generation creation – 2 • Evolution on island is independent from other islands • After fixed number of generation islands exchange fixed part of elite individuals A. Davydov, D. Sokolov, F. Tsarev Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem

19. Moore automaton mutation • Change start state • Change one of state actions • Change the state to which one of the transitions leads • Change transition condition A. Davydov, D. Sokolov, F. Tsarev Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem

20. Mealy automata system mutation • Change start state • Delete (insert) transition (only for external automaton) • Change the state to which one of transitions leads • Change transition action • Internal automaton mutation A. Davydov, D. Sokolov, F. Tsarev Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem

21. Moore automata crossover • Input: two individuals • Output: two individuals • Parents – P1 andP2 • Child – S1 andS2 • Start state: S1.is = P1.isandS2.is = P2.is, orS1.is = P2.isandS2.is = P1.is A. Davydov, D. Sokolov, F. Tsarev Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem

22. Transitions crossover • State i • «There is food in front of ant» ↔P1(i, 0) • «There is no food in front of ant»↔ P1(i, 1) • Similarly: P2(i, 0), P2(i, 1), S1(i, 0), S1(i,1), S2(i, 0), S2(i, 1) • There are 4 variants forS1(i, 0), S1(i,1), S2(i, 0), S2(i, 1) A. Davydov, D. Sokolov, F. Tsarev Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem

23. Four variants A. Davydov, D. Sokolov, F. Tsarev Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem

24. Systems of Mealy automata crossover • External automata – the same way as for Moore automata • Internal automaton – crossover is done with some probability A. Davydov, D. Sokolov, F. Tsarev Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem

25. Fitness function for Moore automata • F – number of units of food which ant eats in 200 steps • T – number of step when ant eats last unit of food A. Davydov, D. Sokolov, F. Tsarev Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem

26. Fitness function for systems of Moore automata • F – number of units of food which ant eats in 200 steps • T – number of step when ant eats last unit of food • Z – number of visited states in external automaton • C – coefficient A. Davydov, D. Sokolov, F. Tsarev Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem

27. Generation mutation • After fixed number of generations there is “big” mutation • “Big”mutation – all individuals either change to mutants or change to randomly generated individuals A. Davydov, D. Sokolov, F. Tsarev Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem

28. Adjustable parameters of genetic algorithm – 1 • Number of islands • Population size on an island • Time between “big” mutations • Part of islands which is destroyed in “big” mutation • Part of individuals which moves to next generation • Time to exchange of individual between islands A. Davydov, D. Sokolov, F. Tsarev Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem

29. Adjustable parameters of genetic algorithm – 2 • Number of exchangeable individuals • Parameters of small mutation • Parameters of crossover • Ratio of “mutants”, randomly generatedindividuals and “children” • С – external automaton influence coefficient A. Davydov, D. Sokolov, F. Tsarev Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem

30. Moore automaton • This automaton allows ant to eat all food in 198 steps A. Davydov, D. Sokolov, F. Tsarev Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem

31. System of pair nested automata • This automata system allows ant to eat 87 units of food in 198 steps A. Davydov, D. Sokolov, F. Tsarev Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem

32. Thank you! Thank you! A. Davydov, D. Sokolov, F. Tsarev Application of Genetic Algorithms for Construction of Moore Automaton and Systems of Interacting Mealy Automata in “Artificial Ant” Problem