James L. McClelland

1. Development, Disintegration, and Neural Basis of Semantic Cognition:A Parallel-Distributed Processing Approach James L. McClelland Department of Psychology andCenter for Mind, Brain, and ComputationStanford University In collaboration with: Tim Rogers, Karalyn Patterson, Matt Lambon Ralph, and Katia Dilkina

2. language Parallel Distributed Processing Approach to Semantic Cognition • Representation is a pattern of activation distributed over neurons within and across brain areas. • Bidirectional propagation of activation underlies the ability to bring these representations to mind from given inputs. • The knowledge underlying propagation of activation is in the connections. • Experience affects our knowledge representations through a gradual connection adjustment process

3. A Principle of Learning and Representation in PDP Networks • Learning and representation are sensitive to coherent covariation of properties across experiences.

4. What is Coherent Covariation? • The tendency of properties of objects to co-occur in clusters. • e.g. • Has wings • Can fly • Is light • Or • Has roots • Has rigid cell walls • Can grow tall

5. Development and Degeneration • Sensitivity to coherent covariation in an appropriately structured Parallel Distributed Processing system underlies the development of conceptual knowledge. • Gradual degradation of the representations constructed through this developmental process underlies the pattern of semantic disintegration seen in semantic dementia.

6. Some Phenomena in Development • Progressive differentiation of concepts • Overgeneralization • Illusory correlations

8. The Rumelhart Model

9. The Training Data: All propositions true of items at the bottom levelof the tree, e.g.: Robin can {grow, move, fly}

10. Target output for ‘robin can’ input

11. aj wij ai neti=Sajwij wki Forward Propagation of Activation

12. Back Propagation of Error (d) aj wij ai di ~ Sdkwki wki dk ~ (tk-ak) Error-correcting learning: At the output layer: Dwki = edkai At the prior layer: Dwij = edjaj …

15. Early Later LaterStill Experie nce

17. What Drives Progressive Differentiation? • Waves of differentiation reflect coherent covariation of properties across items. • Patterns of coherent covariation are reflected in the principal components of the property covariance matrix. • Figure shows attribute loadings on the first three principal components: • 1. Plants vs. animals • 2. Birds vs. fish • 3. Trees vs. flowers • Same color = features covary in component • Diff color = anti-covarying features

18. Overgeneralization of Frequent Names to Similar Objects “goat” “tree” “dog”

20. Illusory Correlations • Rochel Gelman found that children think that all animals have feet. • Even animals that look like small furry balls and don’t seem to have any feet at all. • A tendency to over-generalize properties typical of a superordinate category at an intermediate point in development is characteristic of the PDP network.

21. A typical property thata particular object lacks e.g., pine has leaves An infrequent, atypical property

22. A Sensitivity to Coherence Requires Convergence A A

23. Development and Degeneration • Sensitivity to coherent covariation in an appropriately structured Parallel Distributed Processing system underlies the development of conceptual knowledge. • Gradual degradation of the representations constructed through this developmental process underlies the pattern of semantic disintegration seen in semantic dementia.

24. Disintegration of Conceptual Knowledge in Semantic Dementia • Progressive loss of specific knowledge of concepts, including their names, with preservation of general information • Overgeneralization of frequent names • Illusory correlations

25. Picture namingand drawing in Sem. Demantia

27. Grounding the Model in What we Know About The Organization of Semantic Knowledge in The Brain • Evidence from Martin and others indicates that specialized brain areas subserve many different kinds of semantic information. • Semantic dementia results from progressive bilateral disintegration of the anterior temporal cortex. • Rapid acquisition of new knowledge depends on medial temporal lobes, leaving long-term semantic knowledge intact. language

28. Medial Temporal Lobe Proposed Architecture for the Organization of Semantic Memory name action motion Temporal pole color valance form

29. temporal pole name function assoc vision Rogers et al (2005) model of semantic dementia • Gradually learns through exposure to input patterns derived from norming studies. • Representations in the temporal pole are acquired through the course of learning. • After learning, the network can activate each other type of information from name or visual input. • Representations undergo progressive differentiation as learning progresses. • Damage to units within the temporal pole leads to the pattern of deficits seen in semantic dementia.

30. omissions within categ. superord. Errors in Naming for As a Function of Severity Simulation Results Patient Data Severity of Dementia Fraction of Neurons Destroyed

31. temporal pole name function assoc vision Simulation of Delayed Copying • Visual input is presented, then removed. • After several time steps, pattern is compared to the pattern that was presented initially. • Omissions and intrusions are scored for typicality

32. Omissions by feature type Intrusions by feature type IF’s ‘camel’ DC’s ‘swan’ Simulation results

34. Development and Degeneration • Sensitivity to coherent covariation in an appropriately structured Parallel Distributed Processing system underlies the development of conceptual knowledge. • Gradual degradation of the representations constructed through this developmental process underlies the pattern of semantic disintegration seen in semantic dementia.

35. Further Investigations • Lexical and Semantic Deficits in SD • Individual Differences • Laterality Effects • Unilateral Resections and Recovery • Effects of TMS on Semantic Task Performance • Category specific deficits: • Why they may occur in some types of patients but not others.

36. Further Investigations • Lexical and Semantic Deficits in SD • Individual Differences • Laterality Effects • Unilateral Resections and Recovery • Effects of TMS on Semantic Task Performance • Category specific deficits: • Why they may occur in some types of patients but not others.

37. Relation between lexical and semantic deficits (Patterson et al, 2006) • Tested 14 SD patients, each assigned a ‘Semantic Score’ based on 3 standard tests. • Then tested each patient on: • Reading HF&LF Reg. and Exc. Words • Spelling HF&LF Reg. and Exc. Words • Past Tense Inflection, HF&LF R&E Words • Lexical Decision: fruit/frute, flute/fluit • Object Decision (at right) • Delayed Copying Test

38. Reg. Exc. Reg. Exc.

39. The Integrated Connectionist Model

40. Category Specific Deficits: What is the cause? • SD patients typically do not show a differential deficit for living things; however, Herpes Encephalitis patients often do. • Alternative explanations? • Differential lesion location • Differential nature of the lesion itself • In the Rogers Et Al model, removing links produces the SD pattern, but adding noise to connection weights produces the Herpes pattern (Lambon-Ralph, Lowe, and Rogers, 2007).

41. Naming of Animals and Artifacts at the Basic Level: Patients and Simulation Results From Lambon-Ralph, Lowe, & Rogers 2007

42. Predictions from Model • Herpes patients should make relatively more semantic errors, SD patients relatively more omissions. • When tested at the subordinate level of naming, differences between Herpes and SD patients should go away.

44. Conclusions • A single set of principles provide an integrated framework for developing an integrated understanding of both the development and the neural basis of semantic cognitive abilities. • For neuropsychology, the models capture the graceful degredation of semantic knowledge with brain damage, as well as frequency, typicality, and even some aspects of apparent domain-specificity effects. • There are several promising avenues for further extension of the model.

45. The Integrated Connectionist Model

46. A Sensitivity to Coherence Requires Convergence A A

47. 12345678910111213141516 Properties Coherent Incoherent CoherenceTraining Patterns is can has is can has … Items No labels are provided Each item and each property occurs with equal frequency

48. Effect of Coherence on Representation

49. Another key property of the model • Sensitivity to coherent covariation can be domain- and property-type specific, and such sensitivity is acquired as differentiation occurs. • Obviates the need for initial domain-specific biases to account for domain-specific patterns of generalization and inference.

50. Differential Importance (Marcario, 1991) • 3-4 yr old children see a puppet and are told he likes to eat, or play with, a certain object (e.g., top object at right) • Children then must choose another one that will “be the same kind of thing to eat” or that will be “the same kind of thing to play with”. • In the first case they tend to choose the object with the same color. • In the second case they will tend to choose the object with the same shape.