1 / 11

BOOSTED MMI FOR MODEL AND FEATURE-SPACE DISCRIMINATIVE TRAINING

BOOSTED MMI FOR MODEL AND FEATURE-SPACE DISCRIMINATIVE TRAINING. Daniel Povey, Dimitri Kanevsky, Brian Kingsbury, Bhuvana Ramabhadran, George Saon and Karthik Visweswariah IBM T.J. Watson Research Center Yorktown Heights, NY, USA Presented by Yueng-Tien ,Lo. Outline. MMI review BMMI

euridice
Download Presentation

BOOSTED MMI FOR MODEL AND FEATURE-SPACE DISCRIMINATIVE TRAINING

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. BOOSTED MMI FOR MODEL AND FEATURE-SPACE DISCRIMINATIVE TRAINING Daniel Povey, Dimitri Kanevsky, Brian Kingsbury, Bhuvana Ramabhadran, George Saon and Karthik Visweswariah IBM T.J. Watson Research Center Yorktown Heights, NY, USA Presented by Yueng-Tien ,Lo

  2. Outline • MMI review • BMMI • improvements to Model-Space updates • Experiments Results • conclusion

  3. MMI review

  4. BOOSTEDMMI • Enforce a soft margin that is proportional to the number of errors in a hypothesised sentence • Boost the likelihood of the sentences that have more errors thus generating more confusable data • b : boosting factor : accuracy of a sentence s given the reference Sr

  5. IMPROVEMENTS TO MODEL-SPACE UPDATES(1/2) • Canceled statistics • cancel the statistics accumulated on each frame from the numerator and denominator • The only effect this canceling has is to change the Gaussian-specific learning-rate constant

  6. IMPROVEMENTS TO MODEL-SPACE UPDATES(2/2) • I-smoothing to previous iteration • back off to the MMI estimate (based on one iteration of EBW starting from the current iteration’s statistics). • Rules for obtaining to be the largest of: ( I ) the typical case: ( I I )double the smallest • Lead to being positive definite] • I-smoothing to the previous iteration can be better than I-smoothing to the ML estimate

  7. Experimental conditions

  8. EBN50: MPE vs (B)MMI, 4th iter, =0.053 • 0.6% from canceling statistics • 0.9% from boosting • The baseline is MPE I-smoothed to MMI (MPE -> MMI).

  9. EBN700 setup: MPE vs (B)MMI • BMMI-c->ML is better than MPE->MMI • 0.3% from boosting ,0.7% from canceling of statistics

  10. EBN50: I-smoothing -> ML vs -> previous iteration(=0.053) • I-smoothing to the previous iteration rather than the ML estimate • Smoothing to the previous iteration is better, by 0.2% for MPE and 0.4% for BMMI-c

  11. conclusion • We have Introduced… A modification to the MMI objective function - Error-boosting A modification to the MMI training procedure – the canceling of statistics • We have experimented with a totally MMI-based model-space discriminative training procedure and found that it sometimes but not always leads to substantial improvements over our previous MPE-based procedure

More Related