A Multimedia English Learning System Using HMMs to Improve Phonemic Awareness for English Learning

1. A Multimedia English Learning System Using HMMs to Improve PhonemicAwareness for English Learning Yen-ShouLai, Hung-Hsu Tsai and Pao-Ta Yu Chun-Yu Chen

2. Outline Introduction Background The MEL system Speech recognition algorithm of the MEL system The models of the MEL system Experiment and results Discussion and conclusion

3. Introduction Phonemic awareness can let students more effectively acquire reading and spelling abilities Teachers in Taiwan often concentrate on speech skills and neglect recognition of phonemic voice System already exists can not offer learners with learning feedback and high quality of voice To overcome these problems , this system include : Dialogue speech tool Phonemic clustering Mastery learning theory

4. Background The pronunciation difference of EFL learners Lack: Sounds of some English words do not exist Substitution: Learners substituted English pronunciation by similar native language Simplification or complexity: Learners often add or omit one consonant due to side effect of speaking mother tongue Epenthesis: CVC becomes CVCV

5. Background Mastery learning well organized teaching materials and effectively managing student’s learning process are two effective instruction factors can be accomplished by following procedures: divide the concepts and materials into relatively small and sequential learning units get the results where the learners have reached the learning level or not , and also to reflect feedback on their learning Learners who have not mastered a unit, should enter the process of remedial activities or corrections for fully mastering the unit

6. Background

7. Background A review of an ASR system based on HMMs Frame blocking Feature extraction HMM

8. The MEL system MEL system which consists of three modules Speech Analysis Module Mastery Learning Module Management Module for teachers

9. Speech recognition algorithm of the MEL system Adaptive phoneme clustering (APC) algorithm is used in the design of Speech Analysis Module APC algorithm includes two steps Specifies which cluster the input phoneme belongs to The cluster is decided by K-means algorithm The input phoneme is recognized by involving all HMMs in the specified cluster Phonemes classified into clusters to form a hierarchical recognition model with two levels

10. Speech recognition algorithm of the MEL system Training HMM model with APC algorithm Frame has a set of feature parameters = (, ,……. ) the training-pattern set can be expressed as a form = (| j=1,2….,….n ) after feeding the K-means algorithm with the set, a new training-pattern setcan be obtained and represented by = (| i=1,2,….j=1,2….,….n )

11. Speech recognition algorithm of the MEL system is employed to construct a classification model which is realized by HMMs

12. Speech recognition algorithm of the MEL system Confusing phoneme A set of confusing phonemes is specified because some phonemes apparently exist in more than one cluster Reconstructed the 2-level HMMs model with (+1) clusters for the confusing phonemes is extended toby replacing with ( if is a feature of a confusing phoneme A phoneme is considered as a confusing phoneme if its error rate / exceeds a threshold

13. The models of the MEL system Speech analysis module Speech Analysis Module is used to analyze the speech signal and to detect the correction of student’s pronunciation the input speech signal is evaluated in terms of the four factors: Pronunciation Intonation Tempo Volume For the pronunciation , the interval of the voice signal to analyze the pronunciation error if the pronounce score is less than 60

14. The models of the MEL system For the intonation, the MEL system provides the intonation curves of the teacher with students For the analysis of tempo, the system can compare the voice speed every student reads with teachers

15. The models of the MEL system For the analysis of volume, the MEL system compares the volume of students with teacher’s voice for their volume stress

16. The models of the MEL system Mastery learning module for students The students’ learning module includes Learning Material Self-test Error Analysis Video Camera Record Learning History Query for learning history Management module for teachers This module can be used to manage instructional materials, manage students’ profiles, assess students’ tests, and query students’ learning histories

17. Experiment and results Participants 120 third-grade students from an elementary school, as the experimental group and the control group Use the phonemic awareness scores to classify students into three categories Assessment materials Phonemic awareness test Learning achievement test

18. Discussion and conclusion The APC method is employed in the design of the MEL system for reducing the computation time of the hierarchical HMMs The MEL system can promote the phonemic ability of the students with the middle and the low phonemic ability The MEL system can interactively provide concrete feedbacks. It is helpful for self-regulated learning