Automatic Transcription Reconstruction System (ATRS)

1. Automatic Transcription Reconstruction System (ATRS) "I can't believe it’s not literal!" Serguei Pakhomov Michael Schonwetter Joan Bachenko Lernout & Hauspie Healthcare Systems Group

2. Outline of Talk • Start Demo Processing • Define Problem • Describe ATRS Components • Display ATRS Demo Results

3. Start Demo Processing • Start Demo Processing

4. Medical Transcription Operation • Partial Transcriptions are the commercial product of the operation • Partial Transcripts are plentiful • Can be paired with speech files Human Transcriptionist Tel. Speech Partial Transcription

5. Sample Partial Transcription

6. Literal Transcription Generation

7. Summary • Problems Addressed • Partial Transcriptions are Available but Inadequate • Literal Transcriptions are Essential • Human Generated Literal Transcriptions are: • Expensive & Error Prone • Suggested Solution • Recycle Partial Transcriptions with ASR to Generate Semi-Literal Transcriptions

8. ATRS I/O • Inputs: • Partial transcript (RTF) • Digitized telephony speech (8KHz Mulaw) • Outputs • semi-literal transcript in its several variants • speech-text alignment for assisting in generating literal truth Semi-Literal for AM Partial Transcription ATRS Semi-Literal for LM Speech Aligned Semi-Literal with Digitized Speech

9. Description of DPTRS Dictionary Rec. Output Recognizer Integrator Speech Semi- Literal Transcript APFSM Partial Transcript

10. Supporting Models • Probabilistic Finite State Model (PFSM) • Filled Pause Model • Background Model • Augmented Probabilistic Finite State Model (APFSM)

11. Training Corpus with natural Filled Pauses (FP) Partial transcription corpus with no FP’s FP distribution extractor FP distributor Partial transcription corpus with artificial FP’s FP distribution model Language modeling software Filled Pause Model Filled Pause Model

12. Background Model Literal Transcriptions Corpus Partial Transcriptions Corpus Difference Extractor Corpus of phrases spoken but not transcribed (Out Of Transcription(OOT) corpus) Language modeling software Background Model

13. Generate Dictionary • Reduce phonetic confusability • limit entries to those items in the transcription (and supporting models). • Dynamically generate pronunciations • for items in the partial transcript which are out of vocabulary.

14. Recognition Pass • Dictation processed by recognition engine using: • APFSM • Custom Dictionary • SI Acoustic Model

15. Integration • Recognizer output (HYP) is compared to Partial transcript (REF). • For Acoustic Modeling: • Matches • Substitutions: Use REF portion • Insertions: Filled-Pauses, Punctuation • For Language Modeling: • Matches • Substitutions: Use REF portion • Insertions: Use ALL Insertions

16. Integrator 1 2 3 4 5 REF HYP LABEL AM LM semi-litsemi-lit that that MATCH that that she she MATCH she she be me SUBSTITUTION bebe treated treated MATCH treated treated for -- DELETION --for twelve twelve MATCH twelve twelve weeks weeks MATCH weeks weeks -- ah INSERTION ahah -- period INSERTION periodperiod -- on INSERTION -- on three three MATCH three three -- excuse INSERTION -- excuse -- me INSERTION -- me plantar plantar MATCH plantar plantar warts warts MATCH warts warts

17. Semi-Literal Transcript

18. Results: • Compare to Literal Transcripts (n=774) • Alignment of Partial vs. Literal • Alignment of Semi-Lit vs. Literal • yields 4.4% (absolute) better alignment

19. View Demo Results • View Demo Results

20. Contact Information • Contact Info • Serguei Pakhomov • Spakhomov@LHSL.com • Michael Schonwetter • Mschonwetter@LHSL.com • Joan Bachenko • Joan-B@LHSL.com