Speech perception

1. Speech perception Introduction What is auditory agnosia? Types of auditory agnosia Cognitive models Perceptual or semantic deficit? Brain imaging Summary

2. Basic speech Perception • Physical signal varies in amplitude, frequency and time. • From these parameters speech perception occurs - phonemes (buh, tuh) are extracted. • Perception of phonemes in normal speech is categorical i.e. if physical characteristics of the signal are changed slowly then there is a sudden change in the phoneme perceived.

3. Sound waves

4. Variability in the speech stream • How do we recognise the same sounds spoken in different ways e.g. by two different people? • Speech perception is categorical. • The speech recognition system can modify fuzzy input to give the listener the correct sound.

5. Speech perception • When one listens to a spoken word it is necessary to perform at least three operations: • 1) process a series of sound waves; • 2) make fine distinctions between similar patterns of sounds; • 3) extract meaning from the utterance.

6. Seamlessness of speech • There are no natural breaks in speech. • We “hallucinate” word boundaries • Oronyms • The stuffy nose can lead to problems • Speech perception errors • Eugene O'neil won a Pullet Surprise

7. What is auditory agnosia? • Auditory agnosia refers to the defective recognition of auditory stimuli in the context of preserved hearing - as tested with audiometry. • Primary signs of the disorder include difficulty in understanding the meaning of spoken words. • The term `auditory agnosia' can refer to a generalized disorder affecting perception of all types of auditory stimuli including non-verbal sounds, speech and music (e.g., Miceli, 1982).

8. Neuropathology • Auditory agnosias are associated with bilateral, or unilateral lesions of the leftsuperior temporal cortex (Auerbach et al., 1982; Varney and Damasio, 1986) although some cases have been described following unilateral right temporal lobe damage (Roberts et al., 1987). • By far the most common cause is cerebro-vascular accident but some cases have been reported following encephalitis (Arias et al., 1995), head injury (Franklin, 1989) and slowly progressive cortical atrophy (Otsuki et al 1998).

9. Types of auditory agnosia • Disorders of sound recognition can be divided into `apperceptive' and`associative' subtypes after Lissauer's (1890) visual agnosia distinction: • Apperceptive auditory agnosia refers to impaired acoutsical analysis of the perceptualstructure of an auditory stimulus (frequency, pitch, timbre). • Associative auditory agnosia refers to an inability to associate a successfully perceived auditory stimulus with a conceptual (semantic) meaning.

10. Spoken word recognition • Morton (1970) proposed a stage model of auditory word recognition distinguishing between 3 phases: • Auditory analysis system • identifies phonemes in the speech wave. • Auditory input lexicon • identifies the phonological properties of known words. • Semantic system • identifies the meanings of known words.

12. Apperceptive auditory agnosias • Apperceptive auditory agnosias can be broken down into at least three discrete types: • 1) Pure auditory agnosia refers to impairment of non-verbal, environmental sounds after damage to the right temporal lobe (Albert et al., 1972). • 2) Pure word deafness refers to the selective impairment of word perception (Albert & Bear, 1974; Saffran et al., 1976; Coslett et al., 1984). • 3) Amusia to describe deficits in the processing of musical melodies (e.g., Peretz et al., 1994).

13. Pure word deafness • Patient cannot understand spoken words but can discriminate verbal and nonverbal sounds and can speak without impairment (no anomia). • They are unable to repeat back words they cannot understand and can discriminate between vowels but not syllables suggesting a specific perceptual deficit (Saffran et al. 1976). • There seems to be a specific impairment to the perception of speech like sounds e.g. certain speech sounds (phonemes) are more affected more than other types of speech sounds.

14. Pure word deafness • Klein & Harper (1956). • RC. • Stroke patient who could hear everything.... • even a leaf falling, but it sounds far away. You think you can catch it and it fades away... jumbled together like foreign folk speaking in the distance. You feel it should be louder but when everyone shouts its still more confusing.

15. Phonemic boundaries • Blumstein et al (1977) asked PWD patients (all with lesions to left superior temporal gyrus) to discriminate between two similar phonemes e.g., "da" and "ta” that were distorted so the acoustic properties fell on a continuum. • Patients had "fuzzier" boundaries compared with controls who showed a clear point when discriminations of phonemes changed. • Pure word deafness is a disorder at the stage where different acoustic characteristics are classified as instances of a single phoneme.

16. Word meaning deafness • Word meaning deafness is an impairment to the comprehension of words in the absence of an impairment to the auditory analysis of words. • Word meaning deafness patients at test show: • preserved repetition, phoneme discrimination and auditory lexical decision (unlike the PWD patients) showing the auditory input lexicon is dissociable. • impaired comprehension from spoken input only. • whereas comprehension is perfectly normal when patient is given written words to match with pictures (Franklin et al., 1996; Kohn and Friedman, 1986).

17. Bramwell (1897) • 26 year old woman who had a stroke. • Difficulty understanding speech but not deaf: • “Is it not a strange thing that I can hear the clock ticking and cannot hear you speak? Now let me think what that means” • When asked questions verbally: • e.g., "Do you like to come to Edinburgh?" • Could not understand but repeated words and wrote them down and could then answer the question!

18. Bypass route • One way to repeat a word would be to activate its entry in the auditory input lexicon, access semantics and then release its activation in the speech output lexicon. • BUT people can repeat aloud nonwords e.g fep. • Therefore, cognitive models of spoken word processing must propose a bypass route from auditory analysis system to the phoneme level that would enable the processing of nonwords.

20. McCarthy and Warrington (1984). • Patient ORF. • Could repeat words better than nonwords of the same length. • Words: 85% • Nonwords: 39% • Nonwords can only be repeated via the bypass route, whereas words can be repeated by either route. • Therefore, suggests impairment to bypass route.

21. Evidence for a bypass route • Beauvois et al (1980) reported a patient JL who could repeat words but not nonwords aloud. • Martin and Saffran (1992) reported patients who also could not repeat nonwords and also made semantic errors when repeating words. • Repetition was more difficult for abstract words than concrete words - called deep dysphasia. • Supports the existence of a nonlexical bypass route for spoken word repetition as nonwords cannot be repeated via the lexical pathway.

22. Brain imaging studies of speech perception • Is there evidence for separate lexical and non-lexical processing centres in the human brain? • Mummery et al (1999) compared speech perception to silent rest using PET and found that speech perception activated the dorso-lateral temporal (auditory) cortex - bilaterally. • Suggests sound based representations stored in superior temporal lobe structures on both sides of the brain - however - recent MEG evidence shows the left hemisphere is specialised for processing rapid temporal events e.g. speech.

23. Summary • Studies of patients with auditory agnosia have contributed to the development of theories of normal spoken word recognition and to the identification of necessary cognitive processes. • Studies of auditory agnosia have shown that knowledge about spoken words and their meaning can be dissociated into modules. • Studies of auditory agnosia have lead to the identification of speech and non-speech areas in the brain using brain imaging methods.

24. Reading • Parkin, A. Chapter 7. • Ellis, A.W. & Young, A.W. Ch 6. • McCarthy, R & Warrington, E. (1990) Cognitive Neuropsychology: A clinical introduction - 6. • Ellis, A. W. (1984) Introduction to Bramwell's case of word meaning deafness. Cognitive Neuropsychology, 1, 245-258. • Hickok, G & Poeppel, D (2000). Towards a functional neuroanatomy of speech perception. Trends in Cognitive Sciences, 4(4), 131-138.

25. Non speech stimuli • The non-speech stimuli was signal correlated noise (SCN) which forms a stimulus with the same instantaneous amplitude as the speech signal but contains none of the spectral information such as transients and formants that lead to intelligible speech. • SCN preserves segmental information about manner of articulation, rhythm and syllabicity of the speech without comprehension.

26. Results • Brain regions that showed an increase in activation with increasing rates of presentation of both speech and SCN were bilateral primary auditory cortex. • Brain regions that increased in rate with speech only were bilateral superior temporal lobe regions anterior to the primary auditory cortex. • There was also a region of left/right asymmetry i.e. a greater level of activation for speech in the left hemisphere in the core of Wernicke’s area.

27. Auditory analysis deficits • Pure word deafness. • Impaired speech perception in the context of good speech production. • Hempel & Stengel (1940). • 34 year old. • Fell from a bus. • First thought to be deaf. • BUT audiometric testing normal.

28. Pure word deafness • Intact perception of nonverbal environmental sounds. • Read well and with understanding • Could write, therefore, semantics intact. • Couldn't repeat - early processing problem • Claimed that much of what he heard conveyed no meaning to him: • I can hear you dead plain but I can't understand what you say. The noises are not quite natural. I can hear but not understand.

29. Word meaning deafness • Spoken word comprehension is graded by imageability/concreteness—words referring to concrete exemplars are better understood than abstract concepts—and errors are semantically related (Franklin et al., 1994, 1996). • Word meaning deafness is a language-specific deficit of the highest order (highly modular)? • Tyler and Moss (1997) suggest the behavioural pattern may be the by-product of a generalized (earlier) auditory processing impairment.

30. Issues in auditory agnosia • Are there separate modules for speech and non-speech sounds in the human brain? • Is verbal memory category specific? • What is the nature of the deficit in pure word deafness? • Pre-Phonemic • Phonemic (analyzing a string of speech sounds into constituent phonemes). • Left hemisphere specialized for making fine temporal discriminations between rapidly presented auditory stimuli.

31. Prephonemic level • Aurebach et al, (1982) • Patients with bilateral temporal lobe lesions • Impaired perception of spoken words • Found to be impaired at: • detecting gaps between stimuli • determining when two stimuli were simultaneous. • Argued for a basic (prephonemic) disorder of acoustic perception. • See also Tyler and Moss (1997).

32. Phonemic level • Blumstein et al (1977) • Patients with left hemisphere temporal lesions. • Asked to discriminate phonemes e.g., "da”/"ta”. • If systematically distort phonemes so sounds fall on a continuum normal Ss showed a clear point when their discriminations change. • Pure word deaf patients "fuzzier" boundaries. • Disorder at the stage where sounds of different acoustic characteristics are classified as being instances of a single phoneme.

33. Muddying the picture • Pure word deafness. • Some patients have difficulty with non speech sounds (e.g., environmental stimuli). • Performance is generally improved by slowing down (so not all-or -none).

34. During scanning subjects heard either spoken words or SCN sounds, presented binaurally, at rates of 1, 5, 15, 30, 50 and 75 per min. • The presented words were bisyllabic nouns, matched for frequency concreteness and imageability. • 6 subjects, all right-handed males.

35. Auerbach et al (1982) suggested that word deafness was due to the loss of pre-phonological auditory processing • Associated with bilateral temporal lobe lesions, but that a unilateral left temporal lesion causes word deafness due to a deficit in phoneme discrimination and identification. • A more recent clinical and psychophysical study of a patient developed word deafness after a right temporal infarct (Praamstra et al, 1991).

36. Heilman et al (1976) • Left temporal lobe damage • Unable to understand spoken words (speech) • Hearing was normal • Could repeat - so not an auditory analysis problem • Could perform picture-word matching tasks in written form, so not a semantic impairment • Could also produce spoken names to pictures • Argues for an impairment at the level of the auditory input lexicon (or its connection to semantics - how might we test for this?).

37. Partial Comprehension – Loss is rarely complete; patient often retains superordinate (category) information but loses detailed subordinate information (Warrington, 1975). • Amount lost varies from category to category.

38. Basic Model • Three main components to comprehension and production of words: • 1. Analysis and representation of the sounds of incoming words • 2. Analysis and representation of meanings of words • 3. Synthesis and production of word sounds

39. 1. Problems identifying familiar word sounds – no hearing impairment, speech okay but can’t repeat a heard word. • The problem lies in the sensory and perceptual processes, the end result of which is normally the auditory percept (compare with the visual percept in agnosia). • Some authors distinguish between pre-phonemic (sensory) and phonemic (perceptual) disorders (Auerbach et al., 1982). Others don’t (McCarthy & Warrington, 1991).

40. The "Two types" school say • There are failures in sensory processing - "word sound deafness" i.e. failures in the processing of frequency, amplitude, duration or temporal discrimination of the incoming speech sounds. • Albert & Bear (1974) reported that impaired temporal acuity lead to words sounding like a foreign language; patient couldn’t tell whether one or two clicks unless they were very well separated; difficulties in determining auditory sequences; improved if words presented slowly.

41. There are failures in phonemic processing - "word form deafness" i.e. inability to discriminate phonemes reliably (Blumstein, Baker & Goodglass, 1977). • Can’t identify the word being spoken. (see ohps)

42. Word meaning deafness • Problems identifying familiar word meanings • Interested in those cases where word sound perception is okay but meaning is still not understood (Gianotti, Caltagirone & Ibba, 1975). • The disorder applies to some, not all, words. • These patients can repeat words back so perception must be okay (Geschwind, Quadfasel & Segarra, 1976). • So, meaning problem must be in the semantic system or in access to it.

43. 2. Tested using speech-picture matching (can’t match picture of horse with word horse), speech-definition matching (can’t match definition of horse with word horse).

44. Characteristics of Word Meaning Deafness • 1. Word frequency - massive effect on comprehension (Schuell, Jenkins & Landis (1961). • 2. Conceptual areas selectively impaired (i.e. category specificity)

45. 1. Colour names, body parts, action names (Goodglass, Klein, Carey & Jones, 1966) (see W&M, p 130) • 2. Concrete/abstract. Loss of abstract words is common (Goldstein, 1948) but this could be due to difficulty. • Warrington(1975) and Warrington & Shallice (1984) report two patients (AB and SBY respectively) who showed greater loss for concrete words. • So, double dissociation between concrete and abstract words.

46. 3. Within concrete - Many selective losses here Most common is that food and living things impaired much more than objects (Warrington & Shallice, 1984 cases SBY, JBR; also patients KB and ING). • The Double Dissociation comes from patients VER & YOT, who were better on living things (Warrington & McCarthy, 1983, 1987). See W&M, page 132.

47. Causes of word meaning deafness • Two possible general causes (McCarthy & Warrington) • 1. Disconnection - Geschwind, Quadfasel & Segarra (1968) - repetition ok but both comprehension and spontaneous speech impaired - implies disconnection of perceptual system from meaning of words (affecting comprehension) and meaning of words from production of words (affecting spontaneous speech).

48. Heilman, Tucker & Valenstein (1976) found a patient whose repetition was ok, could name from pictures (output ok) but no comprehension - disconnection of perceptual system from meaning of words. i.e. disconnection of perceptual system from meaning established.

49. 2. Damage to semantic system implied by category effects (since some categories ok general access to semantic system must be ok) and partial knowledge (couldn't happen without access being ok).

50. Implications of category effects for organisation of semantic knowledge. • The category effects inform us about organisation of semantic knowledge. • Early explanations were in terms of animate/inanimate distinction. • Replaced by sensory (e.g. food and living things) vs functional (objects) distinction. • Because the representations of these two classes formed from different origins the double dissociation is plausible.