1 / 23

Seminar Speech Recognition Projects E.M. Bakker LIACS Media Lab Leiden University

Seminar Speech Recognition Projects E.M. Bakker LIACS Media Lab Leiden University. Project Outline. Implementation Project Modules Speech Database Speech Signal Analysis Hidden Markov Models + Training Language Models + Training Recognition Algorithms Evaluation. Implementation.

woods
Download Presentation

Seminar Speech Recognition Projects E.M. Bakker LIACS Media Lab Leiden University

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Seminar • Speech Recognition Projects • E.M. Bakker • LIACS Media Lab • Leiden University

  2. Project Outline • Implementation • Project Modules • Speech Database • Speech Signal Analysis • Hidden Markov Models + Training • Language Models + Training • Recognition Algorithms • Evaluation

  3. Implementation • A Safe C++ Programming Style • Not to be used in C++ • Syntax and Programming Style • Conventions • Basic Design Rules • Program Services • Memory Services • Diagnostics • Important Topics • Portability • Testing • Reliability

  4. Implementation: A Safe C++ Programming Style • Features to be avoided, or not to be used in C++ • C inherited features • if(c=0), ?:, , ,goto, break, continue, union, struct, bit-wise, (&& || !), int, short, double, unsigned, ++, --, explicit constant numbers, cast, variable argument lists • Preprocessor features • macros for constants, macros for functions, #pragma, compiler/platform specific directives • Object Oriented • global data, global non-member functions, public data, friend, overloading operators@, ++,... • Memory and pointer-related • pointers, new, delete, malloc, free(), pointers to functions, ->, ->* .*, const char*, NULL, type &ref - t, type count[], type *count, type *count[], type (*count)[], type (&count) • printf, scanf, assembly language, object passed by val and temporary objects

  5. Implementation: Syntax and Programming Style • Programs in plain English • Meaningful names • One statement per line • const: for data and methods whenever possible • variables: local whenever possible • private/protected data members whenever possible • do not use confusing syntax like • if (a) • for (I=0;I++<4;) • always use default in switch-statement • use assert in all the critical points

  6. Implementation: Conventions • Functions and methods: My_Example_Function() • Variables: my_example_var • Classes: MyExampleClass • Constants: MY_EXAMPLE_CONSTANT • In general: meaningful names, except for indices • Comment: • file-description • version history (bugs new functionality) • user information (user guide) • implementation information (reference guide) • code comment

  7. Implementation: Basic Design Rules • Project modularity achieved through classes. • Structure the program by Classes only (only methods are allowed, no separate functions) • Project is decomposed into modules with as little cross-dependence as possible • One module per class • Classes should have minimal interfaces • Modules should have minimal dependencies • Implementation issues hidden from clients (information hiding) • Inheritance should be extensively used • Advantages: • Improved readability • Reduced maintenance work • Improved robustness

  8. Implementation: Program Services • Safe memory management • memory service • dynamic memory management: C++ without pointers • Diagnostics • decide which data must be checked when, and define the actions • File management, user interfaces • User program configuration management • Text data management • Mathematical data management

  9. Implementation: Memory Services & Diagnostics • Memory Services • Diagnostics

  10. Some Important Topics • Portability • portability and defined options in files: compatib.h, defopt.h, Boolean.h • Testing • test routines and version history • Reliability • readability • maintainability

  11. RES General Specification • RES (Recognition Experimental System) is an HMM based experimental tool for continuous multispeaker speech recognition. The system works on recorded speech files and it basically includes: • the batch modules for acoustic model initialization and training • grammar models training • phoneme/word recognition • performance evaluation. • RES is state of art in speaker independent phonetic recognition: • with 69.2% of percent correct using all TIMIT test data using context independent phonetic models. • It yields 87.83% of percent correct in speaker independent word recognition on ATIS using context independent phonetic models not optimally tuned on this database.

  12. RES General Specification • How to build an ASR system for a different language? • we need many segmented speech recordings to feed the training programs and get good HMM models of our voices. • use a freeware program like Snack 1.4 (search on the Internet) to prepare the data. • search a Dutch multispeaker phonetic database. • Design and feed the right language-model. • Speech samples to train and test the RES system? • You can download speech samples from Linguistic Data Consortium (LDC) after you have obtained a user account.

  13. General Specification

  14. General Specification • Required C++ custom libraries: • none • Portability: • Linux • Windows 3.x, Windows 95, NT • DOS with DjGpp • Compilers: • Ms Visual C++ >4.0 • DjGpp version 2.8.1 or • GNU Linux Gpp version 2.8.1 or newer

  15. Speech Database • Speech data retrieval • Speech files: • NIST1A (ATIS x, TIMIT), • MS WAV • custom, adding software drivers • Label File: • ATIS • IMIT • various subsets, custom labels alphabets included in a file, custom label handling supplying a driver. • Other options: • overlap • window length • file buffering

  16. Speech Signal Analysis • Feature Extraction • Signal processing: • Any concatenation of processing blocks is allowed. Each block performs a class of processing and the actual processing is specified by the options. • Available processing blocks: • Preemphasis_and_Hamming • Mean_Subtraction • FFT • MFCC with Log/non Log Energy • any order differences • Other Blocks can be added supplying proper drivers.

  17. Hidden Markov Models • HMM model Initialization • HMM topology: • 4 predefined types with configurable number of states. • Acoustic Units: • as allowed by the available database • emission densities: • Untied Gaussian mixtures • full or diagonal covariance matrix • number of mixtures configurable for each acoustic unit • Initialization method: • maximum distortion splitting on segmented database

  18. Hidden Markov Models Training • Training algorithm: • Single and Simultaneous Model Re-estimation Baum-Welch. • parameter re-estimation: selective by configuration.

  19. Language Models • Language Model: • unigram and bigram on words and phonemes • Smoothing techniques • Good-Turing, non-linear and linear interpolation model • Word Clustering: minimum mean square error on transition probability • Perplexity • word and phoneme based computation

  20. Recognition • Recognition • Recognition Unit: • acoustic units, words • Algorithm Type • Viterbi with Beam search and Window search pruning strategies

  21. Evaluation • Evaluation: Wagner-Fisher algorithm

  22. Projects 1. Dutch Speech Corpus + Database Interface (2 groups) • in an early phase some example classes should be available, like counting, etc. • maybe use tools like ‘praat’ (for wav labeling with phonetics), etc. 2. Signal Analysis and Feature Extraction (2 groups) 3. HMM Initialization + HMM Dutch Phonetic Training (2 groups) 4. Dutch Language Model + Word Class Training (2 groups) • in an early phase some examples should be available 5. Recognition (2 groups) Evaluation (All)

  23. Project Designs The design of the project should contain the following: • The implementation goals • The underlying technique and theory • A functional description of the starting-code and tools • The design of new code and functionality • Implementation goals and a time-scheme • NB if it is considered difficult to obtain all the goals within the current time-frame, team up with the other team • Interfacing • Define the module-interfaces • Define the time-path for the essential module-inputs • Define a realistic time-path for the (partial-)outputs of the module.

More Related