Isolated-Word Speech Recognition Using Hidden Markov Models

1. Isolated-Word Speech Recognition Using Hidden Markov Models 6.962 Week 10 Presentation Irina Medvedev Massachusetts Institute of Technology April 19, 2001

2. Outline • Markov Processes, Chains, Models • Isolated-Word Speech Recognition ¤ Feature Analysis ¤ Unit Matching ¤ Training ¤ Recognition • Conclusions

3. Markov Process • The process, x(t), is first-order Markov if, for any set of ordered times, , • The current value of a Markov process depends all of the memory necessary to predict the future • The past does not add any additional information about the future

4. Markov Process • The transition probability density provides an important statistical description of a Markov process and is defined as • A complete specification of a Markov process consists of • first-order density : • transition density :

5. Markov Chains • A Markov chain can be used to describe a system which, at any time, belongs to one of N distinct states, • At regularly spaced times, the system may stay in the same state or transition to a different state • State at time t is denoted by qt Fully Connected Markov Model

6. Markov Chains • The state transition probabilities are made according to a set of probabilities associated with each state • These probabilities are stored in the state transition matrix • where N is the number of states in the Markov chain. • The state transition probabilities are • and have the properties and

7. Hidden Markov Models • Hidden Markov Models (HMMs) are used when the states are not observable events. • Instead, the observation is a probabilistic function of the state rather than the state itself • The states are described by a probability model • The HMM is a doubly embedded stochastic process

8. HMM Example: Coin Toss • How do we build an HMM to explain the observed sequence of head and tails? • Choose a 2-state model • Several possibilities exist 1-coin Model: Observable 2-coin Model: States are Hidden

9. Hidden Markov Models • Hidden Markov Models are characterized by • N, the number of states in the model • A, the state transition matrix • Observation probability distribution state • Initial state distribution, • Model Parameter Set:

10. Left-Right HMM • Can only transition to a higher state or stay in the same state • No-skip constraint allows states to transition only to the next state or remain the same state • Zeros in the state transition matrix represent illegal state transitions 4-state left-right HMM with no skip transitions

11. Isolated-Word Speech Recognition • Recognize one word at a time • Assume incoming signal is of the form: • silence – speech – silence • Feature Analysis • Training • Unit Matching • Recognition

12. Feature Analysis • We perform feature analysis to extract observation vectors upon which all processing will be performed • The discrete-time speech signal is with discrete Fourier transform • To reduce the dimensionality of the V-dim speech vector, we use cepstral coefficients, which serve as the feature observation vector for all future processing

13. Cepstral Coefficients • Feature vectors are cepstral coefficients obtained from the sampled speech vector • where is the periodogram estimate of the power spectral density of the speech • We eliminate the zeroth component and keep cepstral coefficients 1 through L-1 • Dimensionality reduction =

14. Properties of Cepstral Coefficients • Serve to undo the convolution between the pitch and the vocal tract • High-order cepstral components carry speaker dependent pitch information, which is not relevant for speech recognition • Cepstral coefficients are well approximated by a Gaussian probability density function (pdf) • Correlation values of cepstral coefficients are very low

15. Modeling of Cepstral Coefficients • HMM assumes that the Markovian states generate the cepstral vectors • Each state represents a Gaussian source with mean vector and covariance matrix • Each feature vector of cepstral coefficients can be modeled as a sample vector of an L-dim Gaussian random vector with mean vector and diagonal covariance matrix

16. Formulation of the Feature Vectors

17. Unit Matching • Initial Goal: obtain an HMM for each speech recognition unit • Large vocabulary (300 words): recognition units are phonemes • Small-vocabulary (10 words): recognition units are words • We will consider an isolated-word speech recognition system for a small vocabulary of M words

18. Notation • Observation vector is , where each is a cepstral feature vector and is the number of feature vectors in an observation • State Sequence is , where each • State index • Word index • Time index • The term model will be used for both the HMM and the parameter set describing the HMM,

19. Training • We need to obtain an HMM for each of the M words • The process of building the HMMs is called training • Each HMM is characterized by the number of states, N, and the model parameter set, • Each cepstral feature vector, , in state, , can be modeled by an L-dim Gaussian pdf • where is the mean vector and is the covariance matrix in state

20. Training • A Gaussian pdf is completely characterized by the mean vector and covariance matrix • The model parameter set can be modified to • The training procedure is the same for each word. For convenience, we will drop the subscript from

21. Building the HMM • To build the HMM, we need to determine the parameter set that maximizes the likelihood of the observation for that word. • Objective: • The double maximization can be performed by optimizing over the state sequence and the model individually

22. Uniform Segmentation Determining the initial state sequence 50 segments  8 states

23. Maximization over the Model • Given the initial state sequence, we maximize over the model • The maximization entails estimating the model parameters from the observation given the state sequence • Estimation is performed using the Baum-Welch re-estimation formulas

24. State transition matrix Mean vector per state Covariance matrix per state Initial state distribution Re-estimation Formulas is the number of feature vectors in state

25. Model Estimation

26. Maximization over the state sequence • Given the model, we maximize over the state sequence • The probability expression can be rewritten as

27. Maximization over the state sequence • Applying the logarithm transforms the maximization of a product into a maximization of a sum • We are still looking for the state sequence that maximizes the expression • The optimal state sequence can be determined using the Viterbi algorithm

28. Trellis Structure of HMMs • Redrawing the HMM as a trellis makes it easy to see the state sequence as a path through the trellis • The optimal state sequence is determined by the Viterbi algorithm as the single best path that maximizes

29. Training Procedure Uniform Segmentation Cepstral Calculation Estimation of (Baum-Welch) State Sequence Segmentation (Viterbi) No Converged? Yes

30. Recognition • We have a set of HMMs, one for each word • Objective: Choose the word model that maximizes the probability of the observation given the model (Maximum Likelihood detection rule) • Classifier for observation is • The likelihood can be written as a summation over all state sequences

31. Recognition • Replace the full likelihood by an approximation that takes into account only the most probable state sequence capable of producing the observation • Treating the most probable state sequence as the best path in the HMM trellis allows us to use the Viterbi algorithm to maximize the above probability • The best-path classifier for observation is

32. Recognition Index of recognized word Cepstral Calculation Select Maximum

33. Conclusion • Introduced hidden Markov models • Described process of isolated-word speech recognition ¤ Feature vectors ¤Unit matching ¤ Unit matching Training ¤ Recognition • Other considerations • ¤ Artificial Neural Networks (ANNs) for speech recognition • ¤ Hybrid HMM/ANN models • ¤ Minimum classification error HMM design