Introduction

1. Interactions Between Hemispheres When Disambiguating Ambiguous Homograph Words During Silent Reading

2. Introduction • How people understand the meaning of written words? • Based on previous psycholinguistic research on the human understanding of the meaning of written words, we created a computational simulated neuronal network model.

3. Introduction • Human understanding of written words required different source of information from long-term memory. • To understand the meaning of the letters“wind” or “ ס פ ר ” we use: • Lexical - prior degree of frequency or familiarity with the homograph • Contextual – prior contextual information • Phonological –Do we use it?

4. Examples of Homographs • The businessman went to the bank. • The fisherman fished on the bank.

5. Dominant on her finger. • Homophonic Homograph She has a unique ring Subordinate on her phone. • Heterophonic Homograph (rare in English) Dominant the silver. Polish people are nice. Subordinate

6. Hebrew • Things get even more complicated in Hebrew. • Most of the words written in Hebrew are omit vowels, nevertheless we understand what we read and we disambiguate words on a daily bases. • Hebrew has Homophonic and Hetrophonic Homographs in abundance.

7. Standard Model in Silent Reading • Although we have two hemispheres in our brain, both specializing in different tasks, most of the connectionist models that explain how people read do not take into account their properties. The Diagram is taken from Seidenberg, M.S., & McClelland J. L.

8. Split Brain Movie (Michael Gazzaniga) Split Brain Scientific American Frontiers http://pbs-saf.virage.com/cgi-bin/visearch?user=pbs-saf&template=template.html&squery=Pieces%2Bof%2BMind

9. Split Brain in Human • Why do we need two hemispheres? • What do we benefit from using both of them for the same task?

10. Input Input Input Semantic Meaning Phonology Orthography Output Output Output Connectionist Network

11. The Split Reading Simulation Model

12. Additional results-Humans vs. Simulation

13. Why Do We Need Two Hemispheres? • Example for using both hemisphere • You have a very nice ring On your IPhone. For the bullfight For your engagement.

14. Results on the Model (counter clues) “LH only” - LH receives counter clues without intervention from RH. “RH only” - RH receives counter clues without intervention from LH. “LH+RH+Low Noise“ - LH receives counter clues and RH phonologic and semantic information containing additional small random values.

15. “LH only”, “RH only”

16. LH failed recovery from dominant meaning

17. “LH + RH + Low noise”

18. The Connected Model • How we can connect the models of the two hemispheres in a more natural way that will allows for recovering from change of meaning (as in humans) ? • yet still maintains the previous results (that successfully model the human differential response) ?

19. Corpus Callosum Model Left Hemisphere Network: Right Hemisphere Network: Semantic Meaning Semantic Meaning Phonology Phonology Corpus Callosum Orthography Orthography

20. Tests: • First, learning stage was done while the LH and RH are disconnected. We connected them only while testing the model • Second, learning stage was done when the LH and RH are connected via the CC. This was performed in two manners: • Free learning - no restriction on the CC weights • Restricted learning - the weights on the CC did change but were limited to 0.1 - 0.3

21. Results - Homophones

22. Results - Homophones • Connected learning yield the following results: • Free learning of CC weights caused the LH and RH to lose their special properties . • Restricted learning cause the LH and RH to not lose their special properties. With CC(weights restriction to 0.25).RH & LH can perform the "Change of heart". Note LH recovery is partial Free learning RH & LH cannot perform the "Change of heart" for homophones

23. Results - Hetrophones

24. Results - Hetrophones With CC(weights restriction to 0.25)Both LH and RH can perform the "change of heart" for Heterophones The same diagram as the previous figure but presented here with standard deviation

25. Connected learning vs. Separate learning • Connected learning has better performance in convergence time then with separate learning. • Free learning of the CC weights causes the network to lose the "weakly coupled" proportions and therefore the LH and RH lose their special properties. • Learning with bounded weights on the CC produces the desired properties provided that the CC bounded weights are less in proportion to the interior hemispheric weights.

26. Summary • Presented Models of both the RH and LH, with architectural differences between the hemispheres • The hemispheres are linked together in a natural fashion, both during during learning and functioning • Connections between the hemispheres allow additional functionality for the LH as observed in humans ("change of heart"); and the hemispheres also perform at comparative speeds that also qualitatively match human DVF experiments • Our work predicts certain relative connectivity strengths between the two hemispheres in architectural regions; and thus suggests new human experiments.

27. Thanks to HIACS Research Center

28. Thanks for listening

29. Additional Results • The subordinate results are contestant with what in known from human experiments at 1000SOA. • The dominant results are contestants with what in known from human experiments although this has not been tested at 1000SOA

30. Additional results-Humans vs. Simulation אוניה פלפל שמחה שנים *

31. Homophone vs. Hetrophone Words • 1000ms after the context was handed out, the ambiguous word was presented. • The result revealed that Heterophones and Homophones were disambiguated differently in the two cerebral hemispheres in a reverse pattern • The results of the computational model on subordinate meaning parallel the ones that are related to our research involving humans • The results of the computational model on dominant meaning parallel the ones that are related to what we know about humans although this has not been tested at 1000SOA

32. Additional results-Humans vs. Simulation

33. Differences in weight between units I and J Learning Function: Changes in weights Learning constant Output of Target unit I Output of Target unit J Sum across inputs and weights to unit I from feedback from unit J i j

34. Activation Function Sum of products of unit j by all its input weights value of unit I in the next iteration: t+1 decay constant Value of unit i from previous iteration Input to unit I i

35. Activation Function value of unit I in the next iteration: t+1 Sum of products of unit j by all its input weights decay constant Value of unit i from previous iteration Input to unit I i

36. Learning Function: Changes in weights Target activation of unit I Learning constant Differences in weight between units I and J Target activation of unit J Sum across inputs to unit I from feedback from unit J

37. Limit function Activation Function decay constant Sum of products of unit j by all its input weights value of unit I in the next interation: t+1 Value of unit i from previous iteration Input to unit I

38. Change from Dominant to Subordinate Meaning • You have a very nice ring on your phone. • חולצהמטיילת במדבר יהודה, במצב קל, ואושפזה בבית החולים סורוקה

39. Assessment 1: Access to Meaning • We measured the network’s number of iterations for all the units until they became saturated • Units in the fields of pronunciation, part of speech, and meaning • We compared the pattern of activity of the RH and LH networks’ response corresponding to the dominant and subordinate meaning of a given homograph across the iterations.

40. Previous finding

41. Conclusions • Left Hemisphere: • All sources of information (e.g., phonology and semantics) areavailableimmediately. • As a result, selection processes are faster and more sensitive to phonological information. • Right Hemisphere: • Not all sources of information are available immediately. • As a result, selection processes are slower and less sensitive to phonological information.

42. Representation The 288 features are grouped into sets of 16: • 3 character x 16 bit → 48 bit Spelling • 5 character x 16 bit → 80 bit Pronunciation • 2 character x 16 bit → 32 bit Part of Speech • 8 character x 16 bit → 128 bit Meaning For example: • ספרsefernoדפיקריאה • ספרsaparnoגוזרשיער

43. Benefits of the Model • Psychological: • Expands the traditional model to include hemispheric differences in understanding words during the reading process • Furthers the understanding of Dyslectic deficiencies and enhances the ensuing methods of treatment • Validates existing and future behavioral findings • Computational: • Validates assumptions regarding the organization of information in the brain

44. Computational Results

45. Left Hemisphere Network: Right Hemisphere Network: Phonology Phonology Semantic Meaning Semantic Meaning Orthography Orthography The Split Reading Model

46. Orthography Phonology Part of Speech Meaning Detailed description of the representation of the dominant sense of “ "ספר

47. Learning Function: Changes in weights i j

48. Activation Function i

49. Activation Function i

50. 48 Meanings Representation • 24 polarized 3-letter noun homographic pairs: • 12 Homophonic • 12 Heterophonic • Words are represented as distributed patterns of activity over a set of simple processing units.