1 / 13

Outline

Outline. Introduction Methods Problems to be solved Demos. Speech Assessment. Speech assessment: How to assess an utterance for the purpose of learning a spoken language? Assessment levels: syllables, words, sentences, paragraphs

deannawells
Download Presentation

Outline

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Outline • Introduction • Methods • Problems to be solved • Demos

  2. Speech Assessment • Speech assessment: How to assess an utterance for the purpose of learning a spoken language? • Assessment levels: syllables, words, sentences, paragraphs • Assessment criteria: timbre, tone, energy, rhythm, co-articulation, … • Feedbacks: High-level correction and suggestions

  3. Related Disciplines • Related disciplines for speech assessment: • Language learning: • CALL: Computer Assisted Language Learning • CAPT: Computer Assisted Pronunciation Training • Speech technology: • UV: Utterance Verification

  4. Our Approach • Basic approach to timbre assessment • Lexicon net construction (Usually a sausage net) • Forced alignment to identify phone boundaries • Phone scoring based on several criteria, such as ranking, histograms, posterior prob., etc. • Weighted average to get syllable score • Weighted average to get sentence score

  5. Basic Assessment Criteria • Timber • Based on acoustic models • Tone • Based on tone recognition (for tonal language) • Based on pitch similarity with the target utterance • Energy • Based on energy comparison with the target utterance • Rhythm • Based on duration comparison with the target utterance • Fluency

  6. Additional Assessment Criteria • English • Stress • Levels (word or sentence) • Meanings • Intonation • Declarative sentence • Interrogative sentence • Co-articulation • A red apple. • Did you call me? • Hit and run • Mandarin • Tone • Retroflex or not • Co-articulation • 兒化音

  7. Problems to be Solved • Score related • Optimization • Consistency • Interpretability • Confusing phone id. (日本人的發音) • Slightly adaptation • Paragraph-level assessment • Contents construction

  8. Demo: Practice of Mandarin Idioms of Length 4 (一語中的) • Level (difficulty) of an idiom is based on it’s freq. via Google search: • 孤掌難鳴 ===> 260,000 • 鶼鰈情深 ===> 43,300 • 亡鈇意鄰 ===> 22,700 • 舉案齊眉 ===> 235,000 • Can be adapted for English learning • Next step: multi-threading, fast decoding via FSM

  9. Support Mandarin & English Support user-defined recitation script Next step: multithreading for recording & recognition Demo: Recitation Machine（唸唸不忘）

  10. Dialog-based practice and evaluation Demo: Dialog Practice via Videos

  11. Demos on PC and PMP • PC軟體 • Lucy’s Café: Speech and Score • PMP • 華語練習機

  12. Demo: Embedded Systems • Chicken run (落跑雞) • Penguin for Tang Poetry (唐詩企鵝) • Robot Fighter (蘿蔔戰士) • Singing Bass & Dog (大嘴鱸魚和唱歌狗)

  13. On-going Work • On-going work: • Tone recognition and assessment • Retroflex & nonretroflex recognition • Detection of “兒化音” • Demo page: • http://mirlab.org/mir_main/demo.htm

More Related