PDP Models of Morphology

1. PDP Models of Morphology Psych 419/719 April 3, 2001

2. What is Morphology? • Derives from MORPH, meaning “to change” • Defines the rules of a language governing how words can be changed into new words.

3. Different Kinds of Morphology • Inflectional • Plural (DOG->DOGS) • Past tense (BUG->BUGGED) • 3rd person singular (LIKE->LIKES) • Derivational: • GOVERN->GOVERNMENT • POMPOUS->POMPOSITY • Compound Words: • Houseboat, boathouse • Infixing

4. Meows, purrs plural + + s Cat The General Character... Meaning Sound

5. Exceptions are Legion.. • Plural: MICE, MEN, GEESE, KNIVES • Past tense: RAN, LIT • Derivational: there’s GOVERN in GOVERNMENT, but no DEPART in DEPARTMENT • Compounds: A bookcase is a container for books, a suitcase is a container for suits, but a staircase isn’t really a container for stairs, a pocketbook isn’t a book, a bookworm isn’t a worm, etc.

6. Despite This, We Can Generalize • WUGS are either more than one WUG, or the act of the verb WUG (e.g., “Bob wugs his car”) • GLORPED is doing GLORP in the past • ESTRANGEMENTALITY is the processes of ESTRANGEMENT

7. The Standard Account • Part of your knowledge of your language is knowing the rules that govern such transformations • Rules operate over standard linguistic units • Stems, affixes, suffixes, etc. • Knowledge of rules is independent of knowledge of mapping from sound to meaning • The exceptions must be memorized by a separate system

8. What PDP Networks Are Like • They can memorize exceptions, but attend to statistical regularities as well. • They’re good at such tasks, but not as good at partitioning training into qualitatively different categories, like rules and exceptions

9. The PDP Story • You learn to associate word forms to their meanings • Regularities are easy to learn • So you pick up on them, • And languages evolve to use them. • “Morphology” is simply attending to these regularities

10. Non-Morphological Regularities • Onomatopoeia - where the sound of a word has something to do with its meaning • whistle, whir, whip, whiz • boom, bang, clang • Words with similar sounds mean similar things • glitter glisten, gleam • sparkle, sputter, sprite • Common Latin root • include, exclude, preclude

11. .. Leads to Different Interpretations • On the standard account, these kinds of regularities are not “morphological” and as such are not handled by the morphological system • On the PDP account, they’re just another regularity for the sound to meaning system to learn. Less predictable, but still there.

12. Other Effects in Morphology • Claim: for some items, you regularize the inflection if the meaning is far from the original meaning • The batter flied out to left field • Maple Leafs vs. Timber Wolves • Sometimes the reverse: • The suspect has been held up in the apartment for 6 hours (c.f. HOLED/HOLD)

13. U-Shaped Learning • Initially, children seem to learn word forms whether they are regular or exception • At some point, performance on exceptions drops.. Children regularize them (saying “eated” for “ate”) • Ultimately, performance recovers

14. The Standard Account of U-Shaped learning • Initially children are memorizing word forms. • Then, they infer the rule for morphology • This results in interference between memorized forms and rule-generated forms • Competition between two systems • Eventually, rule is learned, exceptions are put in “exception box”

15. The PDP Account of U-Shaped Learning • A single homogenous system is learning the task • Interference between regular and irregular forms is not competition between two separate, atomic systems • … but rather, results from the normal dynamics of learning in a single system

16. The Facts of U-Shaped Learning • When you look closely at children’s performance, there isn’t a global “switch” from good to poor performance on all exceptions • As might be predicted by standard account • Rather, there are micro-U trends by item over development • And by the way, it doesn’t happen that often...

17. Micro-U Shaped Learning • This falls out naturally from the PDP account. • Not quite as straightforward for the standard account.

18. Review: Rumelhart & McClelland’s Past Tense Model • Created a two layer network to map uninflected forms to inflections. • Initially, introduced high frequency (mostly exception) words in training. • Then, switched to whole training set • One system learned exceptions and regulars • Demonstrated U-Shaped learning at a global scale

19. Criticisms of R&M Past Tense Model • The frequency shift was bogus • The representation was poor • The errors it made were implausible • Didn’t account for semantic effects • The U-Shaped learning it modeled isn’t what actually happens • Wrong theory: people aren’t just inflecting base forms when making the past tense in normal language

20. Another Try:Plunkett & Marchman ‘91 • Used more reasonable phonological representation • Did not introduce explicit frequency shift • Found that parameters for type/token frequency to induce best match to people was that of actual English • … But, had very few items (500), and did not master vocabulary early in training

21. One More Time:Plunkett and Marchman ‘93 • Increased training set size gradually, one item at a time. • Two conditions: • Add new item when existing ones mastered. Result: Got Stuck • Add new item when certain amount of time passed. Result: Much better

22. Stem To Inflected Word:Summary • Models were able to reproduce U-Shaped learning with a high degree of fidelity to what children do • Provided an account of effect of vocabulary size on interference and generalization that is absent in standard account

23. A New Attack:Marslen-Wilson and Tyler • Looked at priming effects in lexical decision • You get a prime such as BAKE, then have to make lexical decision on BAKER • Crossed semantic overlap with phonological overlap

24. Marslen-Wilson’s Manipulation • Semantic, Phonological, Morphological related • bake / baker • Phonological but not semantic • corn / corner • Semantic but not phonological • cook / baker • Result: only morphological condition primed reliably • Conclusion: Morphology is special

25. The Counter to Marslen-Wilson:Gonnerman ‘98 • Marslen-Wilson observed weak (ns) priming in other conditions • Maybe just phonological or semantic isn’t enough; need both • M-W’s semantic primes weren’t very closely related • Should get graded priming if items sufficiently related, and enough subjects

26. Gonnerman’s Results • If semantic relatedness is high, get priming even with no morphological relationship • Jubilee - jubilant, fork - spoon • Priming effects are in fact graded: semantic overlap gives some priming, semantic and phonological gives more • No effect of morphology independent of semantic and phonological overlap!

27. The Gonnerman & Devlin Model • Maps forms onto meaning • Manipulated phonological and semantic overlap • Results broadly replicated that of empirical study Meaning Word Form

28. Brain Damage Can YieldDouble Dissociations • Some patients exhibit an impairment in generating irregular past tenses • Say runned instead of ran • While others have an impairment in generating novel past tenses • Can’t say glorped for the past tense of glorp • Taken as evidence (Ullman and colleagues) for two systems in morphology

29. The Patients • Those impaired on rules: • Patients with Parkinson’s disease, or left inferior cortex (including Broca’s area) • Those impaired on exceptions: • Patients with damage to Wernike’s area, either from Alzheimer’s disease or lesion

30. The “rules” are in the left inferior frontal area; Broca’s or the basal ganglia The “exceptions” are part of declarative memory, near Wernike’s area Phonological knowledge is needed more for rule-like items Semantic knowledge is needed more for exception processing The Two Accounts Standard Account PDP Account

31. Modeled rule impairment with phonological damage And exception impairment with semantic damage The Joanisse & Seidenberg Model • Four tasks: • Speaking • Hearing • Repeating • Past tense formation Speech Out Semantics Speech in

32. Results of Simulation • Phonological damage impaired rule performance more than exceptions • Semantic damage impaired exceptions more than rule performance

33. Another Angle: Why Do WeGeneralize At All? • In English, the regulars are by far more common than exceptions. Easy to decide it’s the default. • But: in German, there are several forms of the past tense. The “default” is low in frequency • But: Other forms are phonologically conditioned!

34. The Case of the Rats Eaters • People can form compound nouns, by glueing two words together (rat eater, pig farmer, etc). • It has been observed that people don’t like to use plurals as the head of a compound • rat eater (ok) • mice eater (ok) • rats eater (bad)

35. The Standard Account: The Level-Ordering Hypothesis • Rules are applied at different levels of representation • Each level doesn’t have access to input to previous levels • Irregular inflections stored in lexicon • Compounding applies to items in lexicon • Regular inflection happens after compounding

36. Problems With This Account • People use regular compounds in plurals all the time • parks department, weapons inspector, pilots union, communications industry, compounds research • If compounds are generated by rule, acceptability shouldn’t be frequency sensitive • But it is

37. The PDP-Inspired Alternative • There are cues to what is acceptable • Account hinges on what makes a good modifier • Children learn that modifiers generally aren’t semantically or phonologically plural • Consider adjectives: red balloons, not reds balloons • Irregulars are semantically but not phonologically plural, so not great, but better than pure plurals

38. Next Time: Reading and Dyslexia • Optional reading on the class web page • No class April 10