On the use of Intonation in ASR: preliminary results

1. On the use of Intonation in ASR: preliminary results March, 13th 14th 2003 Meeting COST 275 Halmstad

2. OUR GROUP • UPV EI Bilbao • Inma Hernáez (leader), Eva Navas, Jon Sanchez • UVA ETSII Valladolid • Valentín Cardeñoso, Isaac Moro, Carlos Vivaracho, David Escudero. • Involved in a CICYT Project of Biometrics with Javier Ortega (Madrid) and Marcos Faundez (Barcelona). • Experience in ASR and in Modelling Intonation

3. OUR AIM • To work in the CICYT Project with the rest of the groups. • To apply our knowledge in Intonation to the field of ASR. • Here we present our preliminary results.

4. INTRODUCTION • Why to make use of intonation in ASR? • It is a feature that characterize to the speaker: • Speakers of the same group have a similar prosody. • Each speaker can have its own prosody. • It is a very robust feature • Different sessions • Different microphones • Other experiences in applying intonation to ASR • SUPER SID: very simple model of intonation.

5. INTRODUCTION • Aim of this preliminary work • To show the potential capabilities of intonation facing different sessions and microphones • To show that it can be important to make use of “sophisticated” models for getting benefits in ASR. • Overview • Presentation of the model of intonation. • The corpus. • The experiment of speaker verification. • Considerations about the robustness of the results. • Consideration about the use model of intonation. • Conclusions and future work.

6. Modelling Intonation

7. Modelling Intonation

8. Modelling Intonation BASIC IDEA FOR ITS APPLICATION TO ASR: TO COMPARE THE MODELS OF DIFFERENT SPEAKERS

9. Modelling Intonation

10. The Corpus • Recorded at EUPMT by Marcos Faundez • One paragraph read by 16 speakers in 2 sessions with 3 microphones. • Each paragraph = 11 sentences, 106 stress groups. 3816 intonation units. • Speakers are male and in the same social group. • The pitch was obtained automatically and segmented into intonation units by hand. • Intonation was parameterised according to the intonation model.

11. The Experiment • Speaker Verification. • We have 6 recordings for each of the Speakers: 5 for modelling and 1 for testing. • Each Speaker will have each Impostor. The impostor is modelled with the samples of the rest of speakers. • We will repeat the experiment of verification six times (one for each of the possible set of tests) for each of the speakers. • The classifier is based on Decision Trees C.45. Freeware WEKA.

12. Results Low rates, except for some of the speakers

13. Results: robustness No significant changes when different test input

14. Results: relevance of prosodic knowledge. Some parts of the utterance are more relevant depending of the speaker

15. Conclusions and future work • Promising results: some speakers are recognised with high rates. Results are robust to changes in the session and in the microphones. • Future work: To test the benefits of including this results in a ASR system. • To explore the use of our methodology for modelling intonation in a more general way. • Making use of more classes of intonation. • Getting knowledge of which of the classes of intonation are more relevant for characterizing to the speaker. • New corpura are welcome.

16. Stop the war • Thank you