1 / 38

Bayesian Adaptive Learning for Hidden Markov Model Speech Recognition

This research explores Bayesian adaptive learning for HMM speech recognition, including MAP estimates for discrete and semi-continuous HMMs.

ebritton
Download Presentation

Bayesian Adaptive Learning for Hidden Markov Model Speech Recognition

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Bayesian Adaptive Learning of the Parameters of Hidden Markov Model for Speech Recognition (Maximum a Posterior, MAP) Qiang Huo(*) and Chorkin Chan(**) (*)Department of Computer Science The University of Hong Kong, Hong Kong(**) Department of Radio and ElectronicsUniversity of Science and Technology of China,P.R.C. Presenter：Hsu Ting-Wei 2006/02/16

  2. Outline • Introduction • Maximum a Posterior (MAP) Estimate for Discrete HMM • Maximum a Posterior (MAP) Estimate for Semi-continuous HMM • Conclusion NTNU Speech Lab

  3. Introduction • The widespread popularity of the HMM framework can mainly be attributed to the existence of the efficient training procedures for HMM. • HMM parameter estimators have been derived purely from the training observation sequences without any prior information included. • Baum Welch and segmental k-means are two most commonly used procedures for the estimation of HMM parameters. • Bayesian inference approach provides a convenient method for combining sample and prior information. NTNU Speech Lab

  4. Introduction (cont.) ex: Prior ML ML NTNU Speech Lab

  5. Introduction (cont.) 當 f 函式的model的參數pi,a,b假設為獨立時,1.在DHMM中 搭配的prior function叫Dirichelet分布2.在SCHMM中 搭配的prior fuction為Dirichelet + Normal-Wishart分布 HMM中每個model中的states的參數的機率組合成一分布型態,如:常態分布,高斯分布 Prior ML f 函式: 給定lambda下X所成分布 Prior: 收集許多lambda所求得之分佈，再取log所的分布,其中prior的參數叫 hyperparameter + Q 輔助函式 此即ML的概念,利用EM去對Q函式估測但估測出來的機率不可靠 = R 輔助函式 此即MAP的概念,再利用EM去對R函式估測估測出來的機率較可靠,因為Knowledge更多 NTNU Speech Lab

  6. MAP Estimate for Discrete HMM Inference : NTNU Speech Lab

  7. MAP Estimate for Discrete HMM (cont.) Definition : NTNU Speech Lab

  8. MAP Estimate for Discrete HMM (cont.) Prior : hyperparameter NTNU Speech Lab

  9. MAP Estimate for Discrete HMM (cont.) Q-function : E Step NTNU Speech Lab

  10. MAP Estimate for Discrete HMM (cont.) Q-function : NTNU Speech Lab

  11. MAP Estimate for Discrete HMM (cont.) R-function : NTNU Speech Lab

  12. MAP Estimate for Discrete HMM (cont.) Lagrange Multiplier M Step Initial probability sum=1 NTNU Speech Lab

  13. MAP Estimate for Discrete HMM (cont.) Transition probability NTNU Speech Lab

  14. MAP Estimate for Discrete HMM (cont.) Observation probability NTNU Speech Lab

  15. MAP Estimate for Discrete HMM (cont.) • How to choose the initial estimate for ? • One reasonable choice of the initial estimate is the mode of the prior density. NTNU Speech Lab

  16. MAP Estimate for Discrete HMM (cont.) • What’s the mode ? • So applying Lagrange Multiplier we can easily derive above modes. • Example : NTNU Speech Lab

  17. MAP Estimate for Discrete HMM (cont.) • Another reasonable choice of the initial estimate is the mean of the prior density. NTNU Speech Lab

  18. Model 1 Model 2 Model M MAP Estimate for Semi-continuous HMM NTNU Speech Lab

  19. MAP Estimate for Semi-continuous HMM (cont.) Definition : precision : covarience的倒數 mean NTNU Speech Lab

  20. MAP Estimate for Semi-continuous HMM (cont.) Prior : independent NTNU Speech Lab

  21. MAP Estimate for Semi-continuous HMM (cont.) Q-function : E Step NTNU Speech Lab

  22. MAP Estimate for Semi-continuous HMM (cont.) Q-function : NTNU Speech Lab

  23. MAP Estimate for Semi-continuous HMM (cont.) R-function : NTNU Speech Lab

  24. MAP Estimate for Semi-continuous HMM (cont.) Initial probability M Step NTNU Speech Lab

  25. MAP Estimate for Semi-continuous HMM (cont.) Transition probability NTNU Speech Lab

  26. MAP Estimate for Semi-continuous HMM (cont.) Mixture weight NTNU Speech Lab

  27. MAP Estimate for Semi-continuous HMM (cont.) Differentiating w.r.t and equate it to zero. Differentiating w.r.t and equate it to zero. NTNU Speech Lab

  28. MAP Estimate for Semi-continuous HMM (cont.) • Case1: Full Covariance matrix case NTNU Speech Lab

  29. MAP Estimate for Semi-continuous HMM (cont.) NTNU Speech Lab

  30. MAP Estimate for Semi-continuous HMM (cont.) NTNU Speech Lab

  31. MAP Estimate for Semi-continuous HMM (cont.) NTNU Speech Lab

  32. MAP Estimate for Semi-continuous HMM (cont.) Case1: Full Covariance matrix case The initial estimate can be chosen as the mode of the prior PDF And also can be chosen as the mean of the prior PDF NTNU Speech Lab

  33. MAP Estimate for Semi-continuous HMM (cont.) • Case2: Diagonal Covariance matrix case NTNU Speech Lab

  34. MAP Estimate for Semi-continuous HMM (cont.) NTNU Speech Lab

  35. MAP Estimate for Semi-continuous HMM (cont.) NTNU Speech Lab

  36. MAP Estimate for Semi-continuous HMM (cont.) NTNU Speech Lab

  37. MAP Estimate for Semi-continuous HMM (cont.) Case2: Diagonal Covariance matrix case The initial estimate can be chosen as the mode of the prior PDF And also can be chosen as the mean of the prior PDF NTNU Speech Lab

  38. Conclusion • The important issue of prior density is discussed. • Some application : • Model adaptation, HMM training….. NTNU Speech Lab

More Related