Adviser: Dr. Yeou -Jiunn Chen Presenter: Ming –Da Lee

1. An Objective Identification of Spectral Distinctiveness on Acoustic cue to Subjects with Hearing Loss Adviser: Dr. Yeou -Jiunn Chen Presenter: Ming –Da Lee

2. Outline 1. Introduction 1.1 Motivation 1.2 Types of articulation errors 1.3 Paper review 1.4 Purpose 2. Methods and results 2.1 Linguistic Analysis 2.2 Speech Cue Identification • DTW • Identification for Acoustic cue 3.Experimental Results and Discussions 3.1 Materials 3.2 Manipulation of Acoustic Cue 3.3 Subjective Measurements 3.4 Results of Subjective Measurements 4.Future work 5. References

3. Motivation • Hearing loss • conductive hearing loss • sensory-neural hearing loss • mixed hearing loss • Articulation Disorder • Hard to talk with each other • Hard to learn • Speech-language pathologists usually supervise an auditory training

4. Introduction • Types of Articulation errors • 替代音 (Substitution) 如：『公公』 說成 『咚咚』 • 省略音 (Omission) 如：鞋 說成 ㄧㄝˊ • 添加音 (Addition) 如：『吃飯』 說成 『ㄔㄨ飯』 • 聲調錯誤 (Tone Errors) • 歪曲音 (Distortion) • 整體性的語音不清(parsing ambiguities)

5. Types of articulation errors • 前置化（或稱舌尖音）：發音位置在前面的發音方法。 • 後置化（或稱舌根化）：發音位置在舌根後的發音方法。 • 不送氣化：以不送氣音取代送氣音。 • 塞音化：以 ㄅ、ㄆ、ㄉ、ㄊ、ㄍ、ㄎ 替代其他音。 • 塞擦音化：以 ㄐ、ㄓ、ㄗ 替代 ㄒ、ㄕ、ㄙ。 • 省略聲母：ㄧ ㄨㄚ 代替 ㄒㄧ ㄍㄨㄚ 。 • 省略界音：ㄚ 代替 ㄨㄚ。 • 韻母簡化： • 去聲隨鼻韻（鼻音缺乏）。 • 複韻母簡化：以單韻母替代複韻母。 替代音 省略音

6. Paper review • F. Li and J.B. Allen, (2011), “Manipulation of consonants in natural speech,” IEEE Transactions on Audio, Speech, and Language Processing 19, 496–504. • F. Li, A. Menon, and J.B. Allen, (2010), “A psychoacoustic method to find the perceptual cues of stop consonants in natural speech,” J. Acoust. Soc. Amer., vol. 127, no. 4, pp. 2599–2610.

7. Paper review

9. Automatic speech recognition (ASR) • MFCC • PLP • RASTA • DTW

10. Purpose • Speech cue identification • Objective • Distinctiveness on Acoustic cue

11. Methods and results • 2.1 Linguistic Analysis

12. Consonant Affected By Vowel

16. DTW • Butterworth • Order-6 • Band 500Hz • Overlap 250hz • FFT • Frame 400 • Overlap 200

17. DTW • Euclidean Distance • dist(x, y) = |x - y| = [ (x1-y1)2 + (x2-y2)2 + ... + (xn-yn)2 ]1/2 • Boundary conditions • (p1, q1) = (1, 1), (pk, qk) = (m, n) • D(X, Y) = ∣ i(X) - j(Y)∣ + min{D(X -1, Y ), D(X -1, Y-1), D(X, Y -1)}

20. DTW Path

22. Distance Table

24. Identification for Acoustic cue

25. Materials • TIMIT • 16K Hz • 16 bit

26. Manipulation ofSpeech Cue

27. MFCC

28. PLP

29. MFCC

30. PLP

31. Results of Subjective Measurements(MFCC)

32. Results of Subjective Measurements(PLP)

33. Future work • Speech Cue Enhancement • Speech Cue Verification

34. References • 李育穎 (1999) 語障者語音溝通輔助器的研究及製作，碩士論文，國立成功大學電機工程學系。 • 葉向林 (2003) 聽障者之語音增強與轉換，碩士論文，國立清華大學電機工程學系。 • 陳明聰 (2000)，身心障礙者中文替代鍵盤與輸入法輔助學習系統之設計及應用成效研究，碩士論文，國立臺灣師範大學特殊教育研究所。 • 陳雅菁 (1995) 類神經網路在兒童構音異常診斷上之應用，碩士論文，國立成功大學工程科學研究所。 • M.S. Glassman, and M.B. Starkey (1988), "Speech therapy using computer based minimal consonant pair discrimination," Proceedings of the Annual International Conference of the IEEE in Engineering in Medicine and Biology Society, vol. 4-7, pp1421-1422. • 陳信全 (2003) 可攜式語障者發音訓練器之研製，碩士論文，南台科技大學電機工程系。 • F. Li and J.B. Allen, (2011), “Manipulation of consonants in natural speech,” IEEE Transactions on Audio, Speech, and Language Processing 19, 496–504. • F. Li, A. Menon, and J.B. Allen, (2010), “A psychoacoustic method to find the perceptual cues of stop consonants in natural speech,” J. Acoust. Soc. Amer., vol. 127, no. 4, pp. 2599–2610. • Bocklet, T.; Maier, A.; Eysholdt, U.; Noth, E.,"Improvement of a speech recognizer for standardized medical assessment of children's speech by integration of prior knowledge",Spoken Language Technology Workshop (SLT), 2010 IEEE, Page(s): 259 – 264. • Park, S.H.Dept. of Electr. Eng., Yonsei Univ., Seoul Kim,D.J.Lee, J.H.;Yoon, T.S. “Integrated speech training system for hearing impaired”, IEEE,Transactions on Rehabilitation Engineering, 1994

35. 林寶貴，語音障礙與矯治，五南圖書出版社，1994。林寶貴，語音障礙與矯治，五南圖書出版社，1994。 • 賴湘君，構音異常的診斷及矯治，語言治療教育專題研討專輯，台北市政府教育局，pp.123~133，1990。 • 湯士民，應用錯誤型態分析於英語發音輔助學習，碩士論文，國立成功大學學資訊工程學系，2004。 • 王小川，語音訊號處理，全華出版社，2004 • K. Resmi, S. Kumar, H.K. Sardana, and R. Chhabra, “Graphical Speech Training system for hearing impaired,” in Proc. of International Conference of Image Information Processing, pp. 1-6, 2011. • E. Eriksson, O. Balter, O. Engwall, A.M. Öster, and H. Kjellström, “Design Recommendations for a Computer-Based Speech Training System Based on End-User Interviews,” in Proc. Tenth International Conference on Speech and Computers, Patras, Greece, pp. 483-486, 2005. • R. Sirichokswad, P. Chanyagorn, W. Charoensuk, K. Lertsukprasert, K. Wanichthanarak, P. Boonpramook, “Development of Auditory Training Program for Hearing Impaired Children,” in Proc. of The 3rd International Symposium on Biomedical Engineering, Bangkok, Thailand, 2008. • V.S. Selvam, V. Thulasibai, R. Rohini, “Speech Training System Based on Resonant Frequencies of Vocal Tract,” in Proc. of International Conference on Advanced Communication Technology,pp. 674-679, 2011. • American Speech-Language-Hearing Association(2005),”Roles and responsibilities of speech-language pathologists with respect to alternative communication: Position statement Retrieved”, May 31, 2009 • Beukelman, D., & Mirenda, P. ,Augmentative and alternative communication: Supporting children & adults with complex communication needs (3rd ed.),”Baltimore, Maryland: Paul H. Brookes,2005