Speech Based Optimization of Hearing Devices

1. Speech Based Optimization of Hearing Devices Alice E. Holmes, PhD College of Public Health & Health Professions

2. Research Team • Alice E. Holmes • Audiology • UF Communicative Disorders • Rahul Shrivastav • Speech Science • UF, Communication Sciences and Disorders • Lee Krause • Engineer • Audigence • Purvis Bedenbaugh • Neuroscience • East Carolina University

3. The Problem • Programming is based on electrically measured dynamic ranges of pulsed stimuli (non-speech) • Current programming methods have numerous options

4. Purpose • The goal is to understand speech, the tuning of the device should be based on speech and not tones. • Development of a standard metric to understand the strengths and weaknesses of the individual CI user. • An automated process needs to exist to optimize the CI mapping strategy based on input from the speech feature error matrix.

5. Vision born in 2002 after Lee Krause received cochlear implant Speech Science Technology Strong grounding in clinical audiology, speech intelligibility through 5-year Univ. of Florida & Audigence interaction Strong technical team assembled to solve complex problems using optimization theory Innovative Patent Protected Solution Positioned for Success • 7,206,416 - Speech based optimization of Digital Hearing devices (US, Australia, EPO) • CIP Telephony domain (filed published) • Patient classification (filed Sept 2008 -US) • Reduction in test time (filed Sept 2008 -US) • Optimization Algorithms I -- Optimization Algorithms II (filed Aug 2008 -US) 5

6. Cochlear Implants • Comprehensive approach for the Implant domain • Standard/automated fitting approach • Improved device performance • Integrated rehabilitation • Patient population results used to drive future research • Supports telemedicine • Supports multiple languages • Original approach was designed as a two stage process • First level of optimization focused on the signal (rate, loudness growth, FAT) • Second level of optimization was the fine tuning (individual sensor gain, frequency range) • Current work focused on individual parameters • Future work focused on getting to optimization in less time • Business rules • Data mining • Full automated (speech recognition)

8. Intermediate Signal, Sint Brain, B Cochlear Implant Device, D Output Signal, Sout Input Signal, Sinp Almost nothing is known about the function B Overview

9. Stimuli (Sinp) • Chosen from a set of phonemes • Each phoneme is characterized by a set of 9 auditory distinctive features for English [Jakobson, Fant & Halle, 1963] • Each feature is weighted by importance in the language

10. Device Parameters (D) • Any device parameter -- E.g. Signal processing strategies, stimulation rate, pulse width, threshold (T), comfort values (C), number of channels or maxima, deactivation of selected channels, frequency allocation, gain, global T and C modifiers, Q-value, base level, jitter, channel ordering • All of these parameters together characterize the function D • Adjusting certain parameters to decrease errors in one feature might lead to an increase in error in another feature -- How to adjust the parameters such that the overall performance is enhanced? -- An optimization problem

11. Artificial Intelligence • What to optimize? -- Minimize error function • From patient experiments, we can get data for different values of the parameters and the corresponding errors -- The dimension of this data is equal to the number of independent parameters -- Many parameters, hence very high dimension leading to the “curse of dimensionality”

12. Artificial Intelligence • To reduce the complexity of the problem -- Patient-independent knowledge should be available (e.g. as rules) -- Patient-specific knowledge should be statistically extracted from the performance of each patient -- Model field theory approach

13. Initial Clinical Trial • 20 adults with • N24 or New Freedom implants • Freedom Processors • Adjusted the following parameters • Rate • Loudness growth • Frequency allocation tables • Outcome measures • CNC lists in quiet • BKB-SIN • Subjective questionnaire

15. Initial Clinical Trial • The Optimization program was designed to interface with a customized version of Cochlear Corp. Custom Sound so that programming changes recommended by the algorithm could be tested seamlessly. • All stimuli were presented through a direct connection to the speech processor and at a constant level across all test sessions (approximately 60 dBA). • 3 Sessions – two weeks apart

16. Session 1 • Baseline performance obtained using subject’s current map on outcome measures • The T- and C- values were obtained at multiple pulse rates • The optimization routine completed • Map with lowest NWE was selected and programmed in to processer for use until next session

17. Optimization (Clarujust™ ) • A series of VCV syllables were presented & verbal responses were recorded by the researcher. • NWE for the processor setting was calculated • The next combination of FAT, PR & LG was automatically recommended & tested. • Procedure was repeated for 30 minutes

18. Session 2 • Outcome performance using Opt 1 was evaluated using CNC lists in quiet and BKB-SIN measurements as reported in Session 1. • The Optimization procedure described above was then repeated to obtain Optimization 2 (Opt 2) and programmed into their speech processor. • Subjects were asked to use the optimized map until Session 2.

19. Session 3 • Outcome performance using Opt 2 was evaluated using CNC lists in quiet and BKB-SIN measurements as reported above. • Subjects then chose the maps that they wanted saved in their speech processors for regular/everyday use.

24. CNC Word ANOVA-R • Significant difference among the three conditions using Greenhouse-Geisser analysis (p < 0.004). • Further trend analyses indicated a significant ascending omnibus trend from baseline (p < 0.004) • Pairwise comparisons significant differences between baseline and Opt 1 (p < 0.025) and between Baseline and Opt 2 (p < 0.015).

25. CNC Phoneme ANOVA-R • Significant difference among the three conditions using Greenhouse-Geisser analysis (p < 0.008). • Further trend analyses indicated a significant ascending trend from baseline (p < 0.015) • Pairwise comparisons showed significant differences between base line and Opt1 (p < 0.003) and between Baseline and Opt 2 (p < 0.04).

26. BKB-SIN ANOVA-R • Significant difference among the three conditions using Greenhouse-Geisser analysis (p < 0.03). • Further trend analyses indicated a significant ascending quadratic trend from baseline (p < 0.009). • Pairwise comparisons showed significant differences between baseline and Opt 1 (p < 0.03)

27. Subjective Results • At the end of this clinical trial, 17 out of 20 patients preferred to continue using one of their optimized maps. • Subjective ratings in various situations were also obtained from each subject (Holden, et al, J Am Acad Audiol 18:777–793, 2007)

29. Conclusions • The optimization method used in this study resulted in improved subject performance in all outcome measures. • Speech perception was significantly better in word and phoneme identification with optimized maps. • In addition, subjectsperformed better in noise using the optimizedmaps. • Subjective tests suggest that patients preferred the optimized maps in their daily lives.

30. What is Next? • One patient has been successfully mapped from his initial hook-up • Continue to refine process with CI technology • Currently doing a clinical trial with hearing aid programming • Future applications • Hybrids • Audiologic rehabilitation • Cell phones • ????

31. Thank you to the students involved • Hannah Siburt • Kevin Still • Elyse Swartz • Bekah Gathercole

32. Acknowledgments • This project is funded by Audigence, Inc. and theFlorida High Tech Corridor Council. • We wish to thank Cochlear Corporation for supplying the fitting software platform and for their extensive and timely technical support. • We also want to thank our subjects for their willingness to participate in the experiment.