Download
1 / 13
130 likes | 217 Views
Sampath Jayaram. EARLY LEXICAL DEVELOPMENT IN A SELF-ORGANIZING NEURAL NETWORK. Introduction. Connectionist modeling of language acquisition has made significant progress, but 3 drawbacks exist that need to be addressed when considering future work in this area.
E N D
Sampath Jayaram EARLY LEXICAL DEVELOPMENT IN A SELF-ORGANIZING NEURAL NETWORK
Introduction Connectionist modeling of language acquisition has made significant progress, but 3 drawbacks exist that need to be addressed when considering future work in this area. Some language acquisition models use artificially generated or limited sets of vocabulary. Most models used supervised learning through back propagation. No model exists that can account for the incremental nature of lexical growth. Catastrophic interference.
Introduction • To address these 3 problems, DevLex was developed. • DevLex is a self-organizing neural network model of the development of the lexicon. • Combines dynamic learning properties & scalability. • Earlier work has shown that self-organizing neural networks, especially SOMs are very suitable to model the human lexicon. • Based on the results of earlier work, DevLex accounts for the following phenomena : • emergence & categorization of linguistic categories • early lexical confusion during naming • age of acquisition (AoA) effects
Introduction • Some psycholinguistic phenomena that have motivated making DevLex. • Neuropsychological & neuroimaging studies have shown distinct areas of cortical activation to nouns, verbs, adjectives, etc. Nativists say that this process is hardwired but others believe it to be an emergent process. • Lexical confusion (e.g. put instead of give , or take instead of bring) • AoA – a model has to display both plasticity in learning as well as stability in representation.
The DevLex model • On training the network, the word form & word meaning are fed to the network. Through self-organization, they form activation patterns in the P-GMAP and S-GMAP respectively. • Weights around the winner are updated, and simultaneously associative weights are also updated by Hebbian learning. • The combination of these establishes relationships between word forms and their meanings. • These associative links are used to model production (semantics to word form) and comprehension (vice-versa). • GMAPs are used because of the growing nature of the lexical learning task : both the size of the lexicon & the input space.
Phonological & semantic representations • The PatPho system is used to provide input to the network for the phonological forms of the words. It forms phonological patterns for words (upto trisyllabic) and converts them to vector form to be used as input. • For semantic representations 2 forms were used. • WCD (word co-occurrence detector) • WordNet • A combination of the 2 was found to be more useful than either of them alone, and so that is what is used as input for the semantic representation.
Growing Map • GMAPs are a novel combination of SOMs and Adaptive Resonance Theory (ART). • DevLex functions in 2 modes • First as a SOM for map organization (node neighbourhood) • Then in ART mode for vocabulary growth (node recruitment). • This combination corresponds to maturational processes in the human brain. • Acceleration in vocabulary growth – increase in the number of synaptic connections within & across cortical regions.
Input Lexicon • For the input lexicon, the vocabulary from CDI (Communicative Development Inventories) was used. • 500 words were considered. • These were split into 10 stages of 50 each. • Each of these 50 were split into 5 stages of 10 each. • Consider the vocabulary composition as being split into 4 categories – nouns, verbs, adjectives and closed-class words. • There is a noun bias early on as compared to other classes.
Simulation Results - Category Emergence & Reorganization • On plotting graphs of the measure of compactness of different categories of words, all of the categories moved from a lesser compact to a more compact packing. Nouns were the only exception, deviating very slightly, probably due to the noun bias early on. • On plotting the map reorganization for all the categories, the magnitude of reorganization decreased as the stages of learning wore on, with high reorganization early on in the SOM mode, and more stability towards the end stages, analogous to what happens in the human brain.
Lexical confusion • 75% of the confused words were found to be semantically related in the beginning, and that number went up to 90%. • This is in accordance with what happens in the human brain. • Most of the words confused by children were found to be semantically related.
Age of Acquisition effects. • A word is said to be acquired when both the following conditions hold : • The word must have a resource allocated in both GMAPs • Unambiguous link between form & meaning. • The earlier a word is encountered, the sooner it will have a resource allocated to it. • The second condition takes longer due to the slow nature of associative weights adjustment due to Hebbian learning. • This is analogous to humans (for eg. A child calls both a dog and a cat as doggie).
Conclusion • DevLex is a cognitively plausible, linguistically scalable model that accounts for lexical development in children. • The model develops topographically organized representations for linguistic categories, displays lexical confusion, and shows AoA effects. • This helps in strengthening the notion that self-organizing neural networks are psychologically & biologically plausible models of language acquisition.
