Language

1. Language Overview: I. What is Linguistics? II. Physiological Foundations III. Aspects of Language

2. I. What is Linguistics? • - Linguistics as the scientific study of language attempts to answers the question – what is language? and how is it represented in the mind? • - Language is a system that uses physical symbols (sound, mark on paper) to express meaning: • How is it that a particular sequence of sounds/written symbols gives rise to a meaningful interpretation in the mind?

3. Language is: • 1. Communicative: Permits communication between individuals • 2. Arbitrary: The relationship between language symbols and their meaning is arbitrary • 3. Structured: The pattern of language symbols is not arbitrary: • “the boy ran from the dog” vs “the dog ran from the dog” • 4. Generative: Basic units of language (words) can be used to build limitless number of meanings. A sentence that has never seen before can be understood with no problem.

4. 5. Complex: There are an infinitely large set of possible expressions, only a small portion of which are meaningful. • “The dog irritated Mary” (yes) • “Mary irritated the dog” (yes) • “Dog the Mary irritated” (no) • “Irritated the dog Mary” (no) • ….. • For a given list of 10 words, the number of possible sentences is 10! (= 3,628,800!!)

5. 6. Innate: • The number of all possible 10-word sentences is (105)(105) …(105) = 1050!!! • Yet the language is learned effortlessly by all normal humans before they reach the age of five, with virtually no direct instruction and with little or no systematic presentation of relevant data. • Any child has the capacity to acquire any natural language regardless of genetic heritage. • Notion of linguistic universals: Genetically determined, hard-wired predisposition of linguistic features common to all languages (e.g., visual & auditory perception)

6. Computability of Language • Is language ( production & comprehension) a NP hard problem? - Re Searl’s Chinese Room Argument, it may not actually possible to build the look-up table because doing so would be NP-hard.

7. II. Physiological Foundations 1. Language Areas2. Varieties of Aphasia

8. 1. Language Areas

9. Lateralization of language function Left brain: 98% of right-handed people have almost all of the language functions represented in the left-hemisphere of the neocortex Right-hemisphere processes language functions regarding metaphorical or artistic meaning of language (e.g., humor) In left-handed people, language function is represented predominantly in the right-hemisphere of the neocortex

10. Broca’s Area • Production of spoken language (Motor programs for controlling speech sounds) Wernicke’s Area • Comprehension of language (Interpretation of spoken and written words) Arcuate fasciculus • Connection between Broca’s and Wernicke’s areas Visual Cortex • Processing of written language Angula Gyrus • Connection among Broca’s, Wernicke’s and visual cortex Mortor Cortex Making of speech sounds (i.e. controlling of vocal muscles) ***Web site***

11. 2. Varieties of Aphasia • Broca's aphasia: • - Damage to Broca’s area • - Prevent a person from producing speech. • - Nonfluent, telegraphic speech • - Words are not properly formed (more like a telegram) • (e.g.) • “I’m a sig … no… man … uh, well, … again.” • “Well..mess..uh..sgga..diz..es..” • - Person can understand language

12. Wernicke's aphasia: • - Damage to Wernicke’s area • - Loss of the ability to understand language • - Fluent but unintelligent speech • - Can form words properly but the words that are put together make no sense • (e.g.) • “I go to a dog of cookies in TV” • “I cooked a radio for my mother on the door” • Conduction aphasia: • - Damage to Arcuate fasciculus • - Fluent speech/good comprehension, but unable to repeat what is head or read

13. Acquired alexia: - Damaged connection between visual cortex and Wernicke’s area - Inability to read, but can see words Agraphia: - Inability to write words - damages to where?? Dyslexia: - loss & deficits of reading skills, spelling and recognizing word sounds

14. III. Aspects of Language 1. Phonetics, Syntax & Semantics 2. Speech Perception3. Language Comprehension - Lexical Processing - Sentence Processing4. Language Acquisition

15. 1. Phonetics, Syntax & Semantics • Phonetics: Sound structure of language • Syntax: Grammatical structure of language • Semantics: Meaning structure of language

16. PHONETICS When language is spoken, words are represented as sequences of discrete sounds or utterances -- an acoustic signal produced in the upper respiratory and vocal tracts. • One-to-one relationship between an abstract symbol and the corresponding phoneme. • Bet <--> /bet/

18. SYNTAX From a statistical viewpoint, language consists of a set of arbitrary but distinct symbols that have high order statistical dependence (I.e., hierarchical constraints; no dependence, no language). • ‘t’ ‘n’ (high freq), ‘x’ ‘z’ (low freq) • ‘think’ ‘thank’ vs ‘thunk’ ‘thnki’ • ‘%^#*’ ‘&#@!!’ vs ‘%#*^’ ‘!#$@^&’ • “I think of you” vs “I of you think”

19. Syntax is the study of sentence structure. It attempts to describe what is grammatical in a particular language in terms of rules, that detail an underlying structure and a transformational process. (e.g.) - Subject-verb-object sentence order: “John hit the ball” - Transformational process: “The ball was hit by the john” - Ungrammatical: “The ball hit by the john was” Question: What makes a sentence grammatical, and what makes it ungrammatical?

20. Linear-chain Hypothesis A linear-chain grammar proposes that grammatical sentences are constructed word by word, by selecting next word in a sentence based on the associations of the rest of the words in the sentence, for example, “The boy took his baseball bat and the hit the _____” Grammatical yet unlikely formed by associations: “Colorless green ideas sleep furiously” (Chomsky, 1957)

21. Phase-structure Grammar Grammatical sentences are intrinsically nonlinear, and therefore must be represented as hierarchies.

22. Phase markers: Sentence = noun phrase (NP) + verb phrase (VP) Noun phrase (NP) = determiner (Det) + noun (N) Verb phrase (VP) = verb (V) + noun phrase (NP) - Recursive (infinite embedding) - Ambiguity resolution (e.g.) “Tonight’s TV program discusses stress, exercise and sex with Dick Cavett.”

23. SEMANTICS Semantics is the study of meaning. It is concerned to describing how we represent the meaning of a word in the mind and how we use this representation in constructing sentences. A sentence is an expression of world knowledge and/or personal belief that is true or can be true under certain circumstances. Sentences are related as synonymous or contradictory (I.e., similarity structure). • “John believes the earth is flat” • “John believes the earth is round” • “John think the earth is plane” • “John think the earth is a plane”

24. Mental Lexicon Hypothesis It proposes that words are recognized through a matching process in which an input word is compared with a mental dictionary called a lexicon. • Dictionary of sound/spelling-meaning pairs for all the words we know: (e.g.) Word “Black”: -Spelling: Black - Pronunciation: /blaek/ - Part of speech: adjective - Meaning pointer: ->

25. Mental Lexicon

26. 2. Speech Perception • The first step in comprehending spoken language is to identify the words being spoken, performed in multiple stages: • 1. Phonemes are detected (/b/, /e/, /t/, /e/, /r/, ) • 2. Phonemes are combined into syllables (/be/ /ter/) • 3. Syllables are combined into words (“better”) • 4. Word meaning retrieved from memory

28. Two Characteristics of Speech Perception • 1. Segmentation: • The listener must unpack a continuous stream of acoustic signal into ordered segments of phonemes (consonants & vowels). • Further complicated by the fact that information about different phonetic segments is transmitted in parallel by the same acoustic segment. • --- An daunting task

30. 2. Context effects: • (a) correct mispronunciation • (spoken) ‘We had a lot of compsiny over the weekend’ • (heard) ‘We had a lot of company over the weekend’ • (b) phoneme filling-in • /?ash/ vs /?ask/ • (spoken) /?ash/ --> (heard) /dash/ • (spoken) /?ask/ --> (heard) /task/ • /tash/, /dask/ - not found in English

31. Such context effects suggest a top-down, knowledge-driven, & memory-based process of speech perception. • In sum, Segmentation & Context are two major bottle necks that we must overcome in the development of computer speech recognition.

32. 3. Language Comprehension • In normal conversation we usually understand the meanings of the utterances without conscious effort as soon as we hear. • The fact that language comprehension proceeds with such little difficulty is a tribute to the sophistication and efficiency of the language processing system--I.e., pre-hard wired.

34. Auditory & visual analysis: • Syntactic Translation: • Translate identified words into a syntactic representation, that specifies a temporal sequence of semantic categories and relations (e.g., V, S, N, Det, NP, etc.) • Mental Lexicon in Memory: • Hypothesized storage place in memory for all information pertaining to words such as pronunciation, meaning, spelling, grammatical class, etc.

35. Spreading Activation in Mental Lexicon Network

36. Lexical Access Mechanism: • Both meanings of an ambiguous word are initially retrieved and then the meaningfully interpretable, appropriate one is selected. • A: “I need a new sink in the kitchen” • B: “I was sinking deep in the swimming pool”

37. How did we learn of the mechanism ? • -- Through semantic priming experiments • Priming Target Lexical decision time • Sink Swim 600 ms • Car Swim 750 ms • ?%#$ Swim 800 ms • Sink Kitchen 605 ms • Car Kitchen 760 ms • ?%#$ Kitchen 800 ms

38. GTL (Grand Theory of Language)

39. 4. Language Acquisition 1. Language is universal across all human societies. That is, all societies use language in similar ways. 2. Despite the apparent diversity of human languages, any language can be learned by anybody. 3. Accordingly, languages must have some common underlying structures. Universal Grammar (UG; Chomsky, 1965): A set of abstract innate (i.e., hard-wired in the brain) principles that are universal to all natural languages. Each language is nothing but a specific implementation of these principles (e.g., temperature by F, C or K)

40. Discussion of the Paper “Actions from thoughts” by M A Nicolelis • HBMIs: • Type I (cochlear implant) vs Type 2 (brain-controlled robot arm) • Challenges of building HBMIs: • Recording brain signals (EEG; multichannel single-cell recording) • Processing the signals & generating control outputs (NNs; learning rules) • Example HBMIs: • Epilepsy control • Prosthetic robot arm • How about possible dangers and pitfalls? • Neuronal ‘vote’ • ‘Brain pacemaker’ • ‘Phantom limb’ • ‘Virtual arm’