200 likes | 209 Views
Biological and cognitive plausibility in connectionist networks for language mode l ling. Maja Anđel Department for German Studies University of Zagreb. Connectionist networks. AI networks for simulating cognitive processes – language implementing the “computer metaphor”
E N D
Biological and cognitive plausibility in connectionist networks for language modelling Maja Anđel Department for German Studies University of Zagreb
Connectionist networks • AI networks for simulating cognitive processes – language • implementing the “computer metaphor” • architectural inspiration: human brain (network of artificial neurons)
Connectionist networks • subsymbolic, not symbolic processing = no division in hardware/software – software “built in” to the hardware structure
Models in history (’80) • Rumelhart & McClelland (1986): English past tense – modeling the U-shaped learning curve
Models in history (’80) • McClelland & Kawamoto (1986): Thematic roles in sentences • “How to represent sequences?” w i c k e l f e a t u r e s w i c i c k c k e k e l e l f l f e ...
outputs hidden units inputs Models in history (’80) • Elman (1990): Words of different length • “How to represent sequences?” context units
Today – biological plausibility • O’Reilly (1998): Six principles of biological plausibility: • biological realism • distributed representations • inhibitory competition • bidirectional propagation of activation (top-down and bottom-up) • error-driven task learning • Hebbian learning New algorithms that satisfy all the constraints!
Today – biological plausibility • O’Reilly (1998): Six principles of biological plausibility: • biological realism
Today – biological plausibility • O’Reilly (1998): Six principles of biological plausibility: • distributed representations
Today – biological plausibility • O’Reilly (1998): Six principles of biological plausibility: • distributed representations
Today – biological plausibility • O’Reilly (1998): Six principles of biological plausibility: • inhibitory competition
Today – biological plausibility • O’Reilly (1998): Six principles of biological plausibility: • bidirectional propagation of activation (top-down and bottom-up) conceptual perceptual d o g
Today – biological plausibility • O’Reilly (1998): Six principles of biological plausibility: • error-driven task learning desired outcome error computation transfer function error backpropagation input signal
Today – biological plausibility • O’Reilly (1998): Six principles of biological plausibility: • error-driven task learning network state transfer function input signal desired outcome error computation
Today – biological plausibility • O’Reilly (1998): Six principles of biological plausibility: • Hebbian learning "The general idea is an old one, that any two cells or systems of cells that are repeatedly active at the same time will tend to become 'associated', so that activity in one facilitates activity in the other." (Hebb 1949, p. 70) "When one cell repeatedly assists in firing another, the axon of the first cell develops synaptic knobs (or enlarges them if they already exist) in contact with the soma of the second cell." (Hebb 1949, p. 63)
Today – biological plausibility • New algorithms – combination of new principles • Complexity brings better results • Old models sucessfully transposed • morphology acquisition • syntax processing • semantic categorization • Computational neuroscience