Structural Maximum A Posteriori Linear Regression for Fast HMM Adaptation

1. Structural Maximum A Posteriori Linear Regression for Fast HMM Adaptation Author: O Siohan, TA Myrvoll, CH Lee Presenter: Davidson Date: 20071212 ISCA 2000 / ASR 2000

2. Outlines • Introduction • Informal description of structural MAPLR • Structural MAPLR algorithm • Experiments and results • Conclusions

3. Introduction • Indirect adaptation • Transformation-based techniques (e.g. MLLR) • Model parameters are transformed by a shared function • Also called global adaptation • Good for insufficient amount of adaptation data • Performance saturates quickly for larger amount of adaptation data

4. Introduction (cont.) • Direct adaptation • Attempts to directly re-estimate the model parameters • Only re-estimates acoustic units for which adaptation data is available • Local adaptation • Bayesian learning, often implemented via MAP estimation • Asymptotically converges to MLE for large amount of adaptation data

5. Introduction (other approaches) • MLLR  MAP • MLLR  MAP  jointly re-estimating the model and transformation parameters using a common MAP estimation criterion • Structural MAP (SMAP) • Use prior distribution of the transformation parameters to constrain the estimation via the use of MAP criterion.

6. Introduction (SMAPLR) • Structural Maximum A Posteriori Linear Regression • Prior densities are structured in a tree • Transformation matrices are derived using a MAP criterion instead of ML estimation in MLLR

7. Informal description of SMAPLR • Tree-based MLLR algorithm • Additional transformation priors • Adding structure to the transformation priors

8. MLLR algorithm • Similar classes of sounds (models) should undergo the same transformation • Clustering is defined statically, disregarding the amount of adaptation data • New technique: dynamically controlling the number of transformation clusters based on the available amount of adaptation data • Acoustic units are arranged in a tree structure

9. MLLR (cont.) • Only estimate the transformation matrices of the nodes that have sufficient amount of adaptation data

10. MLLR (cont.) • Each node has a transformation matrix • Transformation matrix can be derived using MLE as in MLLR • Bottom-up approach to determine the cut so that each transformation is the most specific one • Complexity of the transformation can be controlled dynamically based on the size of the adaptation data and the data threshold • Sensitive to small changes in the location of the cut

11. Adding transformation priors • Constrain transformation by using a MAP estimation criterion rather than MLE in MLLR