1 / 32

Discriminative Classifiers vs Hidden Markov Models

This chapter discusses the differences between generative and discriminative classifiers, specifically focusing on the Perceptron model and Hidden Markov Models (HMMs). It explores the use of HMMs in reasoning over time and introduces concepts such as conditional independence, stationary distributions, and the Forward and Viterbi algorithms. Additionally, it discusses the application of HMMs in speech recognition, robot localization, and web link analysis.

twest
Download Presentation

Discriminative Classifiers vs Hidden Markov Models

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CHAPTER 8 DISCRIMINATIVE CLASSIFIERSHIDDEN MARKOV MODELS

  2. Generative vs. Discriminative

  3. The Perceptron Model

  4. Example: Spam

  5. Binary Decision Rule

  6. Online Perceptron Training

  7. Perceptron Training Illustration

  8. Properties of Perceptrons

  9. Issues with Perceptrons

  10. Reasoning over Time • Often, we want to reason about a sequence of observations  Speech recognition  Robot localization  User attention • Need to introduce time into our models • Basic approach: hidden Markov models (HMMs) • More general: dynamic Bayes’ nets

  11. Markov Models

  12. Conditional Independence

  13. Weather Example

  14. Mini-Forward Algorithm

  15. Example

  16. Stationary Distributions • If we simulate the chain long enough:  What happens?  Uncertainty accumulates  Eventually, we have no idea what the state is! • Stationary distributions:  For most chains, the distribution we end up in is independent of the initial distribution  Called the stationary distribution of the chain  Usually, can only predict a short time out

  17. Example: Web Link Analysis

  18. Mini-Viterbi Algorithm

  19. Hidden Markov Models

  20. Example

  21. Conditional Independence

  22. HMM Applications

  23. Forward Algorithm

  24. Viterbi Algorithm

  25. Viterbi Example

  26. Viterbi Properties • Designed for computing the most likely state hidden sequence given a sequence of observations in Hidden Markov Models • Two passes, forward to compute the forward probabilities, and then backward to reconstruct the maximum sequence • What’s the time complexity? • O(d2n) - Why is this exciting? • There are many extensions to the basic Viterbi algorithm which have been developed for other models which have similar local structure: syntactic parsing, for instance.

  27. Speech in an Hour

  28. HMMs for Speech

  29. HMMs for Continuous Obs.? • Before: discrete, finite set of observations • Now: spectral feature vectors are real-valued! • Solution 1: discretization • Solution 2: continuous emissions models  Gaussians  Multivariate Gaussians  Mixtures of Multivariate Gaussians • A state is progressively:  Context independent subphone (~3 per phone)  Context dependent phone (=triphones)  State-tying of CD phone

  30. ASR Lexicon: Markov Models

  31. Viterbi with 2 Words + Unif. LM

  32. Conclusion • Perceptron  A discriminative model, an alternative to generative models like Naïve Bayes  Simple classification rule, based on a weight vector  Simple online learning algorithm, guaranteed to converge if training set is separable • Hidden Markov Models  A special kind of Bayesian Network designed for reasoning about sequences of hidden states  Polynomial time inference for most likely state sequence (Viterbi) and marginalization (Forward- Backward)  Many applications

More Related