Considerations for the Development and Fitting of Hearing-Aids for Auditory-Visual Communication

1. Considerations for the Development and Fitting of Hearing-Aids for Auditory-Visual Communication Ken W. Grant and Brian E. Walden Walter Reed Army Medical Center Army Audiology and Speech Center Washington, DC 20307-5001

2. Typical Listening Environments for Multi-Memory Hearing Aids • Quiet • Background Noise • Low-frequency • High-frequency • Multiple Talkers • Reverberation • Music and Environmental Sounds

3. Auditory-Visual Listening Environments • Face-to-face communication is the most common of all listening environments. • Should hearing aids be programmed differently when visual speech cues are available?

4. If we improve auditory-only speech recognition, do we necessarily improve auditory-visual speech recognition? • What speech information is provided by speechreading? • What speech information is provided by hearing aids? • To what extent is the information provided by speechreading and by hearing aids redundant? • What frequency regions best convey information that is complementary to speechreading?

5. If we improve auditory-only speech recognition, do we necessarily improve auditory-visual speech recognition? • Recognition of medial consonants (/ACA/) spoken by a female talker and recorded on optical disk. • Manipulated the auditory intelligibility by band-pass filtering. • Compared A and AV speech recognition scores for normal-hearing subjects.

6. 100 90 Auditory-Visual Consonant Recognition (%) 80 r = 0.38 70 30 40 50 60 70 80 90 100 Auditory Consonant Recognition (%)

7. What speech information is provided by speechreading? • Recognition of medial consonants (/ACA/) spoken by female talker and recorded on optical disk. • Speechreading only. • Measured percent information transmission of voicing, manner, and place cues.

8. Visual Feature Distribution %Information Transmitted re: Total Information Received 0% 3% 4% Voicing Manner Place Other 93%

9. HYPOTHESIS: The amount of benefit obtained from the combination of visual and auditory speech cues depends of the degree of redundancy between the two modalities. • Speechreading provides information primarily about place-of-articulation. • Hearing aids that provide primarily (redundant) place information will result in small AV benefit. • Hearing aids that provide (complementary) voicing and manner information will result in large AV benefit.

10. 50 40 30 AV Benefit (AV-A) 20 10 r = -0.87 0 0 5 10 15 20 25 30 Auditory Place Information (re: Information Received) 60 50 40 AV Benefit (AV-A) 30 20 10 r = 0.88 0 60 70 80 90 100 Auditory Voicing+Manner Information (re: Information Received)

11. What speech information is provided by hearing aids? • Twenty-five patients fit with the ReSound BT2 multi-band wide dynamic range compression hearing system. • Recognition of medial consonants (/ACA/) under four receiving conditions (speech level at 50 dB SPL): • Unaided Listening (without hearing aid or visual cues) • Aided Listening (with hearing aid, no visual cues) • Unaided Speechreading (without hearing aid) • Aided Speechreading (with hearing aid)

12. 0 E Right (N=25) 10 I Left (N=25) 20 I E 30 E I I E 40 E I Hearing Threshold (dB HL) E 50 I I E 60 I E E I I E I E 70 NU-6 80 Right: M=86.1% (sd: 6.7%) Left: M=85.1% (sd: 6.8%) 90 100 100 1000 10000 Frequency (Hz)

13. 100 90 Listening With Hearing Aid 80 70 60 50 40 Voicing 30 20 Manner 10 Place 0 100 90 80 70 60 50 40 Listening With Speechreading 30 20 10 0 100 90 80 70 60 50 40 Hearing Aid Plus Speechreading 30 20 10 0 0 10 20 30 40 50 60 70 80 90 100 Consonant Recognition Without Hearing Aid (%)

14. To what extent is the information provided by speechreading and by hearing aids redundant? • Amplification and speechreading provide somewhat redundant information. • Hearing aid provided information primarily about place-of-articulation. Smaller gains over unaided hearing were achieved for voicing and manner cues. • Speechreading provided substantial improvement over unaided hearing for place and some improvement for manner. No benefit for voicing.

15. What frequency regions best convey information that is complementary to speechreading? • Auditory recognition of medial consonants (/ACA/) by normal hearing subjects. • Band-pass filtered speech conditions with equal Articulation Index. • Analyzed confusions for information transmission of voicing, manner, and place features.

16. 100 AI=0.1 Voicing 90 Manner Place 80 V 70 60 Percent Information Transmitted M 50 40 30 20 P 10 0 200 1000 6000 Center Frequency (Hz)

17. 100 AI=0.2 V 90 M 80 70 60 Percent Information Transmitted 50 40 P 30 20 Voicing Manner 10 Place 0 200 1000 6000 Center Frequency (Hz)

18. 100 AI=0.3 V 90 M 80 70 60 Percent Information Transmitted 50 P 40 30 20 Voicing Manner 10 Place 0 200 1000 6000 Center Frequency (Hz)

19. SUMMARY • Improving auditory speech recognition does not necessarily improve AV speech recognition. • To improve AV speech recognition A and V cues should be maximally complementary. • Speechreading provides information about place-of-articulation. • Hearing aids tend to provide mostly place information, making them somewhat redundant with speechreading. • Complementary cues to speechreading (voicing and manner) are best conveyed by low-frequencies.

20. Recommendations for designing hearing aids for Auditory-Visual speech communication • Programming should focus on improving the recognition of voicing information, and to a lessor extent, manner-of-articulation information. • Since voicing and manner information are primarily low frequency information, extend frequency response to include this region. • May need to consider effects of compression on low-frequency amplitude envelope. • Traditional concerns about upward spread of masking may not be warranted under auditory-visual conditions.

21. Acknowledgement • NIH Grant DC00792 • ReSound Corporation, Redwood City, CA • Department of Clinical Investigation, Walter Reed Army Medical Center, Washington, DC