Speech based Drug Information System for Aged and Visually Impaired Persons

1. Speech based Drug Information System for Aged and Visually Impaired Persons Géza Németh, Gábor Olaszy, Mátyás Bartalis, Géza Kiss, Csaba Zainkó, and Péter Mihajlik Department of Telecommunications and Media Informatics, BME, Budapest, Hungary { nemeth, olaszy, bartalis, kgeza, zainko, mihajlik }@tmit.bme.hu To try dial:+36-1-886-9490 (Hungarian only) 4. ASR for drug names 7. Evaluation 8. Sample dialogue (translated) 1. System summary • Medicine Line (MLN) is an automatic Hungarian telephone information system operating in Hungary since December 2006. • It is primarily intended for visually handicapped persons and elderly people with normal speech functions.[1] • The caller tells the drug name, the chapter title etc., which is processed by a specialized ASR module. • Medicine Line reads the Patient Information Leaflets (PIL) chapter by chapter. In Hungary there are about 5000 different medicine types approved by the National Institute of Pharmacy. • The output is given by a TTS synthesizer specialized to read drug names, Latin words and pharmaceutical texts correctly. • The MLN system ensures 24 hour access. Userdials +36-1-886-9490 Machine: Welcome to the Drug Information System. Please tell the drug name after the beep! U: Algopyrin M: Algopyrin, If this is not the desired drug please press ‘#’. M: Three versions of this drug are available in our database: Algopyrin 1g/2ml injection (first); Algopyrin 500 mg pill (second); Algopyrin Complex pill (third). Please tell the desired number. U: Second M: Algopyrin 500 mg pill, Please pronounce one of the following chapter titles: What is the drug used for? Before use. How to use? Side effects. How to store? U: Side effects. Machine begins to read the chapter Side effects. General purpose recognizer [2] adapted to drug names with the following features: • MFCC+BEQ acoustic features • GMM-HMM acoustic models are trained on 20 hours of SpeechDat-like data (Hungarian only) • ML estimation of GMM parameters up to 10 mixtures • Speaker independent decision-tree state clustered cross-word Hungarian triphone models • Hungarian grapheme to phoneme rules tailored to drug names +manual control on the output transcription • 5Kword vocabulary • 0-gram language model • Off-line recognition network optimization using WFST algorithms • Real-time one-pass decoding of up to 32 channels ASR test by phone for 1321 drug names • 3 male and 3 female speakers. • Aged between 30-65. • 96.97 % is the recognition rate after the first utterance. 2. System components 5. TTS for pharmaceutical texts 9. Future plans Reading medical terminology by TTS [3] required the following adaptations: • Pronunciation sub-module for non-Hungarian words, abbreviations, chemical expressions. Original examples:N-hepa; hidroxi-propil-metil-cellulóz; 40 µg PGE; 800 mOsm/1; kallikrein inactivator unit; HMG-CoA; non-Hodgkin lymphoma • Special prosody module for pharmaceutical texts with 34 new rules for pause and 12 new rules for prosody. This module handles long and complicated sentences in PILs, text parts between brackets, long enumerations etc. Translated example: If you feel side effects, as for example squeamishness of stomach, sweating, shaking, weakness, giddiness, dryness in the mouth, sleepiness, sleeplessness, costiveness, diarrhea, less appetite, nervousness, excitement, headache, sexual troubles, please ask your doctor to modify the dosage. • Extend Medicine Line to other languages • looking for partners (e.g. FP7 call) • User behaviour analysis • Extension with other functions (e.g. coupling to home care systems) Acknowledgements Complex systemevaluationin 3 user goups • under 25 years (15 persons) • 25-60 years (33 persons, including 7 visually impaired) • over 60 years (12 persons) The project was realized together with the National Institute of Pharmacy, Budapest, Hungary. This work was supported by the Hungarian National Office for Research and Technology (GVOP project no. 3.1.1-2004-05-0426). System evaluation results clearly show the difference between the young and the elderly generation: • Group A found the voice of the TTS less intelligible than Group C. • Elderly people (C) found the speed of synthetic speech mostly very good, but young people (A) regarded it too slow. • People of Group C found user friendliness acceptable, and persons of group A had an even better opinion. • Only question 4 was evaluated by all groups similarly. 3. User controls Key references • Speed: normal/faster (selection at the beginning of the dialogue) • Read previous sentence • Read next sentence • Repeat the actual sentence • Jump to the beginning of the chapter • Jump to the end of the chapter • Stop/continue [1] Henton, C. "Bitter Pills to Swallow. ASR and TTS have Drug Problems" Int. Journal of Speech Technology. 8,2005., pp. 247-257 [2] Fegyó, T., Mihajlik, P., Szarvas, M., Tatai, P., Tatai, G., "VOXenter - Intelligent voice enabled call center for Hungarian", In Proc. Eurospeech ‘03, pp. 1905-1908. [3] Olaszy, G., Németh, G., Olaszi, P., Kiss, G., Zainkó, Cs., Gordos, G. "Profivox – a Hungarian TTS System for Telecommunications Applications", Int. Journal of Speech Technology, Vol 3-4. Kluwer Academic Publishers, 2000., pp. 201-215. 6. Automatic updater maintanence • An automatic updater helps the operator to maintain the database to load new drugs or leaflets or delete old ones. • A new entry can be automatically checked for proper recognition and pronunciation performance.