1 / 42

Connectionist Models of Language

Connectionist Models of Language. Stephen E. Nadeau, M.D. Malcom Randall DVA Medical Center Department of Neurology, University of Florida College of Medicine, Gainesville, FL. Other References.

trish
Download Presentation

Connectionist Models of Language

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Connectionist Models of Language Stephen E. Nadeau, M.D. Malcom Randall DVA Medical Center Department of Neurology, University of Florida College of Medicine, Gainesville, FL

  2. Other References • Nadeau SE. Connectionist models and language. In Nadeau SE, Gonzalez Rothi LJ, Crosson B, eds. Aphasia and Language: Theory to Practice. New York, Guilford Publications, 2000:299-347. • Nadeau SE. Phonology: A review and proposals from a connectionist perspective. Brain Lang 2001;79:511-579. • Nadeau SE, Gonzalez Rothi LJ. Rehabilitation of language disorders. In Ponsford J (Ed). Cognitive and behavioral rehabilitation: From neurobiology to clinical practice. New York: Guilford, 2004:129-174. • H.L. Roth, S.E. Nadeau, and K.M. Heilman, Naming concepts: evidence of two routes,Neurocase 2006;12:61-70.

  3. Plaut DC. Relearning after damage in connectionist networks: toward a theory of rehabilitation. Brain & Lang 1996;52:25-82

  4. McClelland JL, Rumelhart DE. A distributed model of human learning and memory. In: McClelland JL, Rumelhart DE, PDP Research Group, eds. Parallel Distributed Processing. Cambridge: MIT Press, 1986:170-215. vol 2).

  5. Rumelhart DE, Smolensky P, McClelland JL, Hinton GE. Schemata and sequential thought processes in PDP models. In: McClelland JL, Rumelhart DE, the PDP Research Group, eds. Parallel distributed processing. Cambridge, Massachusetts: MIT Press, 1986:7-57. vol 2).

  6. Rumelhart DE, Smolensky P, McClelland JL, Hinton GE. Schemata and sequential thought processes in PDP models. In: McClelland JL, Rumelhart DE, the PDP Research Group, eds. Parallel distributed processing. Cambridge, Massachusetts: MIT Press, 1986:7-57. vol 2).

  7. Rumelhart DE, Smolensky P, McClelland JL, Hinton GE. Schemata and sequential thought processes in PDP models. In: McClelland JL, Rumelhart DE, the PDP Research Group, eds. Parallel distributed processing. Cambridge, Massachusetts: MIT Press, 1986:7-57. vol 2).

  8. Rumelhart DE, McClelland JL. On learning the past tenses of English verbs. In: McClelland JL, Rumelhart DE, the PDP Research Group, eds. Parallel distributed processing. Cambridge, Massachusetts: MIT Press, 1986:216-271. vol 2).

  9. Plaut DC, McClelland JL, Seidenberg MS, Patterson K. Understanding normal and impaired word reading: computational principles in quasi-regular domains. Psychol Rev 1996;103:56-115.

  10. Seidenberg MS, McClelland JL. A distributed, developmental model of word recognition and naming. Psych Rev 1989;96:523-568

  11. Nadeau SE. Phonology: A review and proposals from a connectionist perspective. Brain Lang 2001;79:511-579.

  12. I’ve been retired since 1972 with /cardimiapesun/ (cardiomyopathy)Ten per cent [of] the people[with] the /catraps/ (cataracts) has the [problem with the] retina.I was in the /bizzet/ (business) of records…/fotegraph/ (phonograph) records…for the/shusta (distribution?)…In other words, I was a /eksiev/ (executive).Look, I think it’s /porten/ (important).I can’t [say]/tivelsha/, /diveltsher/ (television), uh TV.

  13. Nadeau SE. Phonology: A review and proposals from a connectionist perspective. Brain Lang 2001;79:511-579.

  14. Properties of PDP Networks • The knowledge is in the connections • A network supports processing, working and long-term memory • Hidden units + nonlinearity support associations between orthogonal representations • Distributed representations • Content-addressable memory • Graceful degradation • Inference, generalization, confabulation • Bottom-up and top-down processing • Learning of implicit rules, structure of knowledge domain through experience

  15. Roth HL, Nadeau SE, Hollingsworth AL, Cimino-Knight AM, Heilman KM. Naming Concepts: Evidence of Two Routes. Neurocase 2006;12:61-70.

  16. Grammar • Syntax (knowledge of word order and sentence structure) • Sentence organization (verb argument structure, embedded clauses, passive voice constructions) • Phrase structure rules • Grammatic morphology (grammatic modifications of words; words of primarily grammatic function)

  17. Genesis of Grammar • Chomsky: a grammar generator • A connectionist conceptualization: • Sentence organization reflects the ways in which the brain manipulates distributed concept representations • Phrase structure rules & grammatic morphology are emergent products of pattern associator networks responsible for articulatory and inscriptional output that incorporate sequence knowledge • Several aspects of grammar are dependent on working memory systems (sustained selective engagement)

  18. Neural Network Representation of Word Classes • Concrete nouns • Distributed concept representations • Adjectives, intransitive verbs, auxiliar verbs • Modification of distributed concept representations • Main transitive verbs, locative prepositions • Reciprocal modification and linkage of two or more distributed concept representations • Abstract words • Juxtaposition of multiple distributed concept representations; limbic distributed representations

  19. Concrete Nouns

  20. Adjectives, Intransitive Verbs:Modification of Distributed Concept Representations • The obese, pockmarked, oily-haired, slovenly, unctuous, check-shirted, plaid-jacketed man with the striped pants, food-stained paisley tie, goatee, wire-rimmed glasses, bundle of pens in his pocket, and clip board in his hand

  21. Main Transitive Verbs, Locative PrepositionsReciprocal modification and linkage of distributed concept representations • Verb argument structure • The old manAgent shot the burglarGoal. • The manAgent gave flowersTheme to MaryGoal. • HeAgent knew (Mary would arrive soon)Theme. • Locative prepositions • The book is on the table.

  22. Abstract Words • Juxtaposition of distributed concept representations; limbic distributed representations • “Intellectual”

  23. Broca’s Aphasia Cinderella uh … scrubbing and uh … hard worker and wants to go a ball. Step fa … mother uh I want to go. Can’t do it. Not … no well why not? I don’t know. Because uh I uh uh … scrubbing uh uh watchacallit uh uh working … object and so clean out bad. Cinderella uh seems like animals love her. Because uh dress … Horse help her. And stepmother uh uh … ruin dress. I don’t know why. Probably because cute. Mad because uh uh … stepmother really ugly. Dress broken and now can’t do it because what dress? Mother Teresa … not exactly … uh uh magic godmother! That’s it. Godmother dress. Don’t worry. I can fix it. And … beautiful. Now carriage where? I don’t know how do you go here? Because castle big. Probably uh mountain castle. How do you get uh here? Oh don’t worry. I can uh … pumpkin and uh … servants and horse and beautiful carriage and so magic. But, better midnight be here because uh uh pumpkin carriage gone. Midnight be here …. Thompson (2001)

  24. Broca’s Aphasia

  25. Broca’s Aphasia • Agrammatism • Impaired sentence organization • Simplification of syntax • Greater difficulty accessing nouns than verbs • Greater impairment in use of verbs with complex argument structure • Limited lexical priming • Resistance to concatenating adjectives • Deficits in selective engagement of distributed concepts representations and manipulation of those representations

  26. Phrase Structure Rules • Word sequence knowledge — an extrapolation of phonemic sequence knowledge

  27. Grammatic Morphology • Free grammatical morphemes: articles, auxiliary verbs, conjunctions, some prepositions • Bound grammatic morphemes: e.g., affixes specifying case, number or tense

  28. Grammatic Morphology • Articles • Sequence knowledge underlying phrase structure rules • Lexical semantic knowledge: definite vs indefinite (but also case, number, gender) • Working memory of current linguistic context • Article production in Broca’s aphasias

  29. Conduction Aphasia I’ve been retired since 1972 with /cardimiapesun/(cardiomyopathy). Ten percent [of] the people [with] the /catraps/(cataracts) has the [problem with the] retina. I was in the /bizzet/(business) of records…/fotegraphy/(phonograph) records…for the/shusta/(distribution?)…In other words, I was a /eksiev/(executive). Look, I think it’s /porten/(important). I can’t [say]/tivelsha/, /diveltsher/(television), uh TV.

  30. Conduction Aphasia

  31. Wernicke’s Aphasia Tell me what happened that brought you here. I been going all right caut so far. They got hold of my lackid last night. Yard in the back. I don’t know what all kapt it. Old himbone and boy I hadn’t seen crossways…on time and best size. You asked when they gave me tham garden sizm made me so sore…just bugarden you know. I didn’t go a slooten until way in the night-time afterwards. Course there was a long lot of, somebody’s bunyangin or something.

  32. Wernicke’s Aphasia

  33. Transcortical Sensory Aphasia

  34. Other Aphasias • Anomic • Transcortical motor • Mixed transcortical

  35. Summary: Domains of Neural Network Knowledge Underlying Language Function • Lexical semantic function • Semantic • Sequence • Pattern associators linking • Semantic & sequence knowledge (the basis for the phonological lexicons) • Grammar • Elaboration of sequence knowledge (affixes, phrase structure rules) • Lexical syntactic • Selection of an articulatory sequence by configurations of reciprocally related concept representations (transitive verbs, locative prepositions) • Selection of an articulatory sequence by configurations of yet to be spoken concept representations and working memory of already spoken representations (articles, pronouns, many auxiliary verbs)

  36. Summary of Aphasia Types • Brocas • Grammar, articulation • Conduction • Phonological sequence • Wernicke’s • Lexical semantic, phonological sequence • TCSA • Semantic, lexical semantic • Anomic • Semantic, lexical semantic in output • TCMA

More Related