SMA: AEIO model

1. La simulation agent et des applications Cour d’introduction généralJuliette Rouchier, GREQAM- CNRS, 2 rue de la Charité, 13236 Marseille Cedex 02, Francerouchier@ehess.univ-mrs.fr

3. SMA: AEIO model • agents Real or virtual entity, autonomous, local percreption of environment, capable of acting. Shared perception, image of self and others, memory, goals, beliefs. • environnement Objects : caracteristics, dynamic evolution laws Formalised by a grid (automata) or a network • interactions communication groups, language, communication protocols (ie : offer competencies, answer) Interpretation of messages • organisation Correlation concerning the evolution of certain entities, temporal organisation Groups or predefined networks with tasks, shared norms, links for communication

4. Complex system (Simon, 61) Self-organisation, emergence : what wasn’t defined in the individual entities’ behaviour • Traffic jam, adaptation,... System limits interacting entities dynamics control feedback OBSERVED System Points of view expectations emergence Artificial: human made, with a goal, imitating, imperative Black box of a system

5. Cellular automata Output State Inputs • Intelligence artificielle • Système expert et multi-experts • Logique formelle • Théorie de l’information Automata • S = set of states • I = set of inputs • O = set of outputs • Transition function • Cybernétique - contrôle • Minsky • Bateson Network of automata with special architecture : inputs of some are outputs for others Example : Game of life

6. MANTA : interactions through a resource and specialisation / learning through simple feedbacks • Reactive agents • Ants, egg, larva, cocoon • Don’t perceive the others, but stimuli in the environment • Can choose an action, according to activity levels (thresholds) • Competing tasks: cure, feed, carry gather food Goal: To represent labour division with very simple agents Inspiration: ants societies Build a framework that is useful to computer scientists and ethologists A. Drogoul

7. MANTA : interactions through a resource and specialisation / learning through simple feedbacks • Success of sociogenesis in about 20% cases, without any centralised decision • Collaboration without any knowledge of the others (intelligence of the programmer) • Specialisation without loosing adaptativity when thresholds of reactions are well adapted • Organisation more or less complex and division of labour (different age > different tasks) • Progressive complexification > diverse learning processes – reinforcement engendered by competition A. Drogoul

8. Emergence of hierarchies (Doran, Palmer) • Goal: to produce a hierarchical society from an egalitarian society • Hypothesis: the resource characteristic is the explanation Resources requires many hunters (complexity) defined by: location, instances, energy, type, complexity, distance for agents to get it, sime be4 renewal Agents seek to “stay alive” (energy) cognitive and action “if then” rules (1 action per time-step) memory: resource model, social model, message buffer, miscellanous (hunger, behaviour mode, perception range) location, speed, sensory range, skill, energy, hunger limit modes: autonomous, recruiting, biding, executing

9. Emergence of hierarchies (Doran, Palmer) Recruitment process: leaders and followers an agent calls up -> the other proposes a bid > accepts the bid >> group when leader, the agent involves its own group when accepting (groups get to be parts of groups) Simulation Variables: • recruitment rules, • Fidelity (how long it stays in the group after activity stops) • conditions of agreement to follow the leader Observation indicators: depth of hierarchy (individual and global) Results: Groups appear and last, autonomous actions, if resources are gathered in a place, groups migrate in this area Need of low complexity resource to have step by step hierarchy building Decrease of productivity with rigid social structures (fidelity).

10. Growing artificial societies. Social science from the bottom-up (sugarscape) • Environment – resources • Agents : layers building • Needs / satisfaction / perception / movement > migrations, differenciations • Reproduction / death > s₫lection • Inheritage > less inequalities but less selection • Culture : gene dissociating two groups • Fight for access to resource > elimination or assimilation • Exchanges : two resources and different needs • With exchanges > reduction of mortality and increasing of inequalities • Economic hypotheses test • Equilibrium appear • Pollution, preference evolution, disease transmission in migration

11. Agents Reactive behavior Cognitive behavior Perception Action Perception Decision Action No learning Perception Action Assessment Perception Decision Action Assessment Learning

12. Autonomy vs independence Separation agent - environment / ability to adapt in evolving environment Actions ordering, interpretation, choice, behavioural change NOT INDEPENDENT !!!

13. Interactions Modification of representations / beliefs Modification of goals • Direct • Indirect Directe Communication : give informations / distribute tasks / solve conflicts / learn • Representation of others : how, with whom to communicate, what knowledge • Representation of the relationship: familiarity, trust • Type of language, interpretation of a message COGNITIVE Indirecte Communication: évolve without consciousness of others’ presence but react to the transformation of environment signals left in environment (stimuli - externalities) REACTIVE

14. Memory • No memory • Conservation of thresholds • Conservation of messages received • Conservation of messages received and sent • Conservation of large amount of information about the context of the messages

15. Organisation Built in organisational elements Networks who to communicate with delegation commitment dependence – authority knowledge of abilities Repartition of tasks – roles - abilities Temporal organisation of the system Who has access to ressource / control over Norms of behaviour that exist in the system – interpretation for agents Emerging patterns that retroact on the system by constraining agents Competition Emerging norms (regularities of behaviour in the group)

16. Diverse representations of social behaviour • From goals to intentions (commitment) • Blinded : Until the agent believes it has accomplished its intention • Single-minded : As long as it thinks it is possible • Open minded : As soon as it has the goal • Some strategies for negotiation • Always concede • Be competitive • Be cooperative : look for a mutually acceptable solution • Inaction • Break • Reactive agents often coordinate through the environment • Two approaches for cognitive agents • define mutual beliefs, joint desires and joint intentions • define norms and conventions.

17. Methods to have agents evolve • Reinforcement learning • Utility function > evaluate results and classify them • Learning / memory capacity / Change of behavior rules • Comparison and copy of others’ methods • Information diffusion or behaviour diffusion • Choice of relevant agents to copy (trust, network) • Mecanisms to adopt behaviours • Genetic algorithms (population level – social learning) • « fitness » function, reproduction, meeting, mutation

18. Memetics • Inspiration of genetics in building of representation the noosphère (Morin), « memes » (Dawkins), epidemiology of representations (Sperber). • Hales (following Bura): « Memes » are on animats using an environment and subject to selection. Each meme : propensity to mute and to reproduce; fights with neighbors and strenghthening. 3 stages : calculation of satisfaction, mutation, replication. • If satisfied è increased aggressiveness and decreased mutation • If not satisfiedè the reverse is true Existence of a meta-meme: open-mindedness that suppresses that phenomena Results Scenario : Just enough food, too much food, predators. Stabilisation of size of animat population able to occupy an area (carrying capacity). killing memes population can grow but is a sign of instability in the system open-mindedness meme help global survival of the population

19. En ce qui concerne les apprentissages individuels, Bourgine [1993] distingue plusieurs niveaux de rationalité des agents selon leur relation à leur environnement et leur capacité à modéliser le réel. • Les agents réactifs réagissent de manière fixe à l’information provenant de leur environnement, sur le mode stimulus-réponse (réponse sensori-motrice ou " pavlovienne " héritée génétiquement) : il y a absence d’apprentissage. • Les agents hédoniques apprennent (par auto-renforcement) à modifier leur comportement afin d’augmenter leur " plaisir ". Ils sont capables d’anticipations " hédoniques " et d’adaptation lente à partir de leur expérience historique, ce qui suppose un niveau de conscience plus élevé que l’agent réactif (consciousness). • Les agents éductifs sont dotés d’une capacité de modélisation de leur environnement, ce qui suppose la capacité de former des représentations symboliques, de simuler les conséquences d’une action sur leur environnement, et donc un niveau de conscience plus élevé (awareness).

20. Selon une perspective plus proche des catégories de l’économiste, Walliser [1997] propose une typologie des processus qui permettent de converger vers un équilibre en théorie des jeux. Il en distingue quatre, soit par ordre décroissant des capacités cognitives attribuées aux agents : • Dans un processus EDUCTIF, chaque joueur dispose d’assez d’information pour simuler parfaitement le comportement des autres joueurs, ce qui conduit immédiatement à l’équilibre : il n’y a pas d’apprentissage. • Dans un apprentissage EPISTEMIQUE, chaque joueur révise ses croyances relatives aux stratégies des autres adversaires à partir des informations qu’il a pu observer (Fudenberg, Levine [1998]). • Dans un apprentissage COMPORTEMENTAL, chaque joueur modifie sa stratégie compte tenu des résultats observés de ses propres actions dans le passé (agent hédonique). • Dans un apprentissage ÉVOLUTIONNAIRE, chaque joueur joue une stratégie fixe qui se reproduit proportionnellement au gain obtenu lors de confrontations aléatoires (agent réactif).

21. Model (agents, environment, interactions, organisation) Parameters : number of agents, learning principle, costs Simulation : temporal evolution Initial setting Environment, number of agents, dotations, ... A time-step environment evolves agents perceive agents make choice agents act agents communicate Final setting

22. Simuler = chercher les « causes » de l’auto-organisation • Observer l’auto-organisation : critère global, critère individuel, aggrégation de données individuels, liens / rencontre des agents, représentations individuelles and collectives. INDICATEURS PERTINENTS POUR DECRIRE DES PHENOMENES «EMERGENTS » • Analyse est possible à travers la comparaison (= « sensibilité aux changement des valeurs de paramètres ») : changement de la situation initiale ou des règles de l’univers – observation des résultats finaux et des processus intermédiaires CHERCHER LES IMPLICATIONS DES REGLES ET DE LA SUCCESSION D’EVENEMENTS