Emergent Adaptive Lexicons

1. Emergent Adaptive Lexicons Luc Steels 1996 Artificial Intelligence Laboratory Vrije Universiteit Brussel Presented by Achim RuoppUniversity of Washington

2. Agenda • Introduction to Emergence and Self-Organization • Steel’s Experiments on the emergence of the lexicon • Discussion

3. Excursion: Emergence andSelf-Organization • A system exhibits emergence when • There are coherent emergents at the macro-level • That dynamically arise from interactions between parts a the micro-level • Emergents are novel w.r.t. individual parts of the system

4. Excursion: Emergence and Self-Organization • Definition • A dynamical and adaptive process • Where systems acquire and maintain structure themselves • Without external control • Combining emergence and self-organization

5. Origins of Language • Still unknown • Chomsky’s hypothesis • Species-specific innate language ability • Refinement by parameter setting process • Some support by experimental simulation via neural networks • Alternative hypothesis • Language as an emergent phenomenon • As a mass-phenomenon • Spontaneously forming/becoming more complex

6. Steel’s Experiments • Motivated by symbol grounding problem • Experiments on robotic and software agents • Grounded meaning creation • Lexicon formation ← Focus of paper • Syntax • Emergent phonology

7. Experimental Model DefinitionsFeatures • Set of agents A • ∀a ∈ A ∃ set of features Fa = {f0,…,fn} • A feature fi consists of attribute-value pairs • Examples: (weight light) (size tall) • Distinctive feature set • Subset of Fa distiguishing agent a from all agents in a group B • Filtered subset CK,M = {a|K ⊂ Fa} • K is a set of features • M is a set of agents

8. Experimental Model DefinitionsLexicon • Word • sequence of letters drawn from a shared alphabet • Utterance • Set of words • Word order does not play a role • Lexicon L • A relation between feature sets and words • A single word can have several associated feature sets • A feature set can have several associated words

9. Experimental Model DefinitionsLexicon • Each agent a has lexicon La • Initially empty • Feature set of a word: Fw,L • Cover functions • cover(F,L): set of utterances U: ∀u ∈ U: {f|f ∈ Fw,Land w ∈ u} • uncover(u,L): set of features F:F = {f|f ∈ Fw,Land w ∈ u}

10. Coherence through Self-Organization • Agents can • Create new words and associate them with a feature set • Form new associations between a word and a feature set • Key to self-organized coherence of the lexicon • Agents participate in communication • Record the success of particular word-meaning pairs • Agents (re-)use words that led to high communication success in the past

11. Language Game • Dialog between two agents – Speaker and Hearer • Dialog topic • Other Agent • Chosen by extra-linguistic means (“pointing”) • Speaker and hearer identify possible distinctive feature sets of topic • Speaker • Selects distinctive feature set • Translates to words using cover function • Hearer • Interprets utterance using uncover function • Compares interpretation to expectation • Uses game to • Learn part of the language • Check if right meaning is associated with the right words

12. Language GamePossible Outcomes • No differentiation possible • Speaker does not have a word • May create new word • Hearer does not have a word • Can associate word • Cannot disambiguate when multiple distinctive feature sets

13. Language GamePossible Outcomes • Speaker and hearer know word • Meanings are compatible with situation • Sense-ambiguity possible • Meanings not compatible with situation • No communicative success

15. Experimental ResultsOne-word Utterances • Typical experimental setup (5 agents, 10 meanings, 4000 language games) • Leads to communicative success soon Average communicative success Number of language games(scale 1/20)

16. Experimental ResultsOne-word Utterances • Single meanings soon converge on one word form (10 agents, 5 possible words, 1 meaning) Average communicative success Time

17. Experimental ResultsMultiple Word Utterances • In case the distinctive feature set of the topic contains multiple features • Can be used by hearer to “guess” meaning of unknown words

18. Conclusions • Self-organization is effective mechanism for achieving coherence • Side-effects of lexicon formation • Synonymy • Ambiguity • Multiple-word sentences

19. Discussion • Supports the notion that absolute synonymy does not exist • “After about 4000 language games the lexicon stabilizes as all distinctions that need to be made have been lexicalized” • Are the presented “linguistic” results an artifact of the experimental setup? I.e. in how far does this experiment reflect the real world? • E.g. multi-word sentences • Can the results just be explained by basic communication theory? • Language as an emergent phenomenon • Zipf’s law regarding multiple meanings • Self-organized criticality/highly optimized tolerance

20. References Brighton, H., Selina, H.; Introducing Artificial Intelligence; 2003; Icon Books Ltd.; ISBN 1-84046-463-1 De Wolf, Tom; Holvoet, Tom; Emergence Versus Self-Organisation: Different Concepts but Promising When Combined; 2005; In Engineering Self Organising Systems: Methodologies and Applications, Lecture Notes in Computer Science, volume 3464, pp 1-15 Steels, L.; Emergent Adaptive Lexicons; 1996; In: Maes, P. and Mataric, M.J. and Meyer, J.-A. and Pollack, J. and Wilson, S.W. (eds) From Animals To Animats 4: Proceedings of the Forth International Conference on Simulation of Adaptive Behavior, SAB'96, Complex Adaptive Systems, pp. 562-567, Cambridge, MA: The MIT Press Zipf, G. K.; The Meaning-Frequency Relationship of Words; Journal of General Psychology 33, 251–256 (1945).