Hidden Markov Models

1. Hidden Markov Models IP notice: slides from Dan Jurafsky

2. Markov Chains Hidden Markov Models Three Algorithms for HMMs The Forward Algorithm The Viterbi Algorithm The Baum-Welch (EM Algorithm) Applications: The Ice Cream Task Part of Speech Tagging Outline

3. Chomsky Grammars • Distinguish grammatical English from ungrammatical English: • John thinks Sara hit the boy • The hit thinks Sara John boy • John thinks the boy was hit by Sara • Who does John think Sara hit? • John thinks Sara hit the boy and the girl • Who does John think Sara hit the boy and? • John thinks Sara hit the boy with the bat • What does John think Sara hit the boy with? • Colorless green ideas sleep furiously. • Green sleep furiously ideas colorless.

4. Miniature English grammar and lexicon S → NP VP S → Aux NP VP S → VP NP → Pronoun NP → Proper-Noun NP → Det Nominal Nominal → Noun Nominal → Nominal Noun Nominal → Nominal PP VP → Verb VP → Verb NP VP → Verb NP PP VP → Verb PP VP → VP PP PP → Preposition NP Det → that | this | a Noun → book | flight | meal | money Verb → book | include | prefer Pronoun → I | she | me Proper-Noun → Houston | TWA Aux → does Preposition → from | to | on | near | through

5. Sentence Generation S → VP VP → Verb NP Verb → book NP → Det Nominal Det → that Nominal → Noun Noun → flight Book that flight

6. Acceptors and Transformers • Chomsky’s grammars are about which utterances are acceptable • Other research programs are aimed at transforming utterances • Translate a English sentence into Japanese… • Transform a speech waveform into transcribed words… • Compress a sentence, summarize a text… • Transform a syntactic analysis into a semantic analysis… • Generate a text from a semantic representation…

7. Strings and Trees • Trees are a useful representation for grammatical structure • A sentence is a noun phrase (NP) followed by a verb phrase (VP) • A noun phrase is a determiner (DT) followed by a noun (NN) • A noun phrase is a noun phrase (NP) followed by a prepositional phrase (PP) • A PP is a preposition (IN) followed by an NP • A string is acceptable if it has an acceptable tree • Transformations may take place at the tree level S NP VP NP PP IN NP

8. Natural Language Processing • 1980s: Many tree-based grammatical formalisms • 1990s: Regression to sequence-based formalisms • Hidden Markov Models (HMM), Finite-State Acceptors (FSAs) and Transducers (FSTs) • N-gram models for accepting sentences [e.g., Jelinek 90] • Taggers and other statistical transformations [e.g., Church 88] • Machine translation [e.g., Brown et al. 93] • Software toolkits implementing generic weighted FST operations [e.g., Mohri, Pereira, Riley 00]

9. Natural Language Processing • 2000s: Emerging interest in tree-based probabilistic models • Machine translation • [Wu 97, Yamada & Knight 02, Melamed 03, Chiang 05, …] • Summarization • [Knight & Marcu 00, …] • Paraphrasing • [Pang et al 03, …] • Question answering • [Echihabi & Marcu 03, …] • Natural language generation • [Bangalore & Rambow 00, …]

10. FSA and FST Acceptors Transducers c c:z c:z/.7 c/.7 a:x/.5 a/.5 a:x a Unweighted e:y e e:y/.5 e/.5 Weighted

11. Finite-State Transducer (FST) Original input: Transformation: q k : ε k q2 i : AY q n : N qfinal g : ε n q3 q4 t : T h : ε i g h t

12. Finite-State (String) Transducer Original input: Transformation: k q2 n i g k : ε q2 i : AY q h n : N qfinal g : ε q3 q4 t : T h : ε t

13. Finite-State (String) Transducer Original input: Transformation: k N n i g k : ε q2 i : AY q h n : N qfinal g : ε q3 q4 t : T h : ε t

14. Finite-State (String) Transducer Original input: Transformation: k N n AY i q g k :ε q2 i : AY q h n : N qfinal g : ε q3 q4 t : T h : ε t t

15. Finite-State (String) Transducer Original input: Transformation: k N n AY i g k : ε q3 q2 i : AY q h n : N qfinal g : ε q3 q4 t : T h : ε t

16. Finite-State (String) Transducer Original input: Transformation: k N n AY i g k : ε q2 i : AY q h n : N qfinal g : ε q3 q4 t : T q4 h : ε t

17. Finite-State (String) Transducer Original input: Transformation: k N n AY i g k : ε q2 i : AY q h n : N qfinal g : ε T q3 q4 t : T h : ε qfinal t

18. A Weighted Finite-State Automaton (WFSA) An FSA with probabilities on the arcs The sum of the probabilities leaving any arc must sum to one A Markov Chain (or observable Markov Model) a special case of a WFST in which the input sequence uniquely determines which states the automaton will go through Markov Chains can’t represent inherently ambiguous problems Useful for assigning probabilities to unambiguous sequences Definitions

19. Weighted Finite State Transducer • FST: FSA whose state transitions are labeled with both input and output symbols. • A weighted transducer puts weights on transitions in addition to the input and output symbols • Weights may encode probabilities, durations, penalties, ... • Used in speech recognition • Tutorial at http://www.cs.nyu.edu/~mohri/pub/hbka.pdf

20. Markov Chain for weather

21. Markov Chain for words

22. Markov Chain • First-order observable Markov Model • A set of states Q • q1, q2…qN sequence of states: state at time t is qt • Transition probabilities: • a set of probabilities A = a01a02…an1…ann. • Each aijrepresents the probability of transitioning from state i to state j • Distinguished start and end states

23. Markov Chain Markov Assumption: • Current state only depends on previous state P(qi | q1 … qi-1) = P(qi | qi-1)

24. Another representation for start state • Instead of start state • Special initial probability vector p • An initial distribution over probability of start states • Constraints:

25. The weather model using p

26. The weather model: specific example

27. Markov chain for weather • What is the probability of 4 consecutive warm days? • Sequence is warm-warm-warm-warm • i.e., state sequence is 3-3-3-3 P(3, 3, 3, 3) = 3a33a33a33a33 = 0.2 • (0.6)3 = 0.0432

28. How about? • Hot hot hot hot • Cold hot cold hot • What does the difference in these probabilities tell you about the real world weather info encoded in the figure?

29. HMM for Ice Cream • You are a climatologist in the year 2799 • Studying global warming • You can’t find any records of the weather in Baltimore, MD for summer of 2008 • But you find Jason Eisner’s diary • Which lists how many ice-creams Jason ate every date that summer • Our job: figure out how hot it was

30. Hidden Markov Model • For Markov chains, the output symbols are the same as the states. • See hot weather: we are in state hot • But in named-entity or part-of-speech tagging (and speech recognition and other things) • The output symbols are words • But the hidden states are something else • Part-of-speech tags • Named entity tags • So we need an extension! • A Hidden Markov Model is an extension of a Markov chain in which the input symbols are not the same as the states. • This means we don’t know which state we are in.

31. Hidden Markov Models

32. Assumptions • Markov assumption: P(qi | q1 … qi-1) = P(qi | qi-1) • Output-independence assumption

33. Eisner task • Given • Ice Cream Observation Sequence: 1,2,3,2,2,2,3… • Produce: • Weather Sequence: H,C,H,H,H,C…

34. HMM for ice cream

35. Different types of HMM structure Ergodic = fully-connected Bakis = left-to-right

36. The Three Basic Problems for HMMs • Problem 1 (Evaluation): Given the observation sequence O=(o1o2…oT), and an HMM model  = (A,B), how do we efficiently compute P(O| ), the probability of the observation sequence, given the model • Problem 2 (Decoding): Given the observation sequence O=(o1o2…oT), and an HMM model  = (A,B), how do we choose a corresponding state sequence Q=(q1q2…qT) that is optimal in some sense (i.e., best explains the observations) • Problem 3 (Learning): How do we adjust the model parameters  = (A,B)to maximize P(O|  )? Jack Ferguson at IDA in the 1960s

37. Problem 1: computing the observation likelihood • Given the following HMM: • How likely is the sequence 3 1 3?

38. How to compute likelihood • For a Markov chain, we just follow the states 3 1 3 and multiply the probabilities • But for an HMM, we don’t know what the states are! • So let’s start with a simpler situation. • Computing the observation likelihood for a given hidden state sequence • Suppose we knew the weather and wanted to predict how much ice cream Jason would eat. • i.e. P( 3 1 3 | H H C)

39. Computing likelihood of 3 1 3 given hidden state sequence

40. Computing joint probability of observation and state sequence

41. Computing total likelihood of 3 1 3 • We would need to sum over • Hot hot cold • Hot hot hot • Hot cold hot • …. • How many possible hidden state sequences are there for this sequence? • How about in general for an HMM with N hidden states and a sequence of T observations? • NT • So we can’t just do separate computation for each hidden state sequence.

42. Instead: the Forward algorithm • A kind of dynamic programming algorithm • Just like Minimum Edit Distance • Uses a table to store intermediate values • Idea: • Compute the likelihood of the observation sequence • By summing over all possible hidden state sequences • But doing this efficiently • By folding all the sequences into a single trellis

43. The forward algorithm • The goal of the forward algorithm is to compute P(o1, o2 … oT, qT = qF | l) • We’ll do this by recursion

44. The forward algorithm • Each cell of the forward algorithm trellis at(j) • Represents the probability of being in state j • After seeing the first t observations • Given the automaton • Each cell thus expresses the following probability at(j) = P(o1, o2 … ot, qt = j | l)

45. The Forward Recursion

46. The Forward Trellis

47. We update each cell

48. The Forward Algorithm

49. Decoding • Given an observation sequence • 3 1 3 • And an HMM • The task of the decoder • To find the best hidden state sequence • Given the observation sequence O=(o1o2…oT), and an HMM model  = (A,B), how do we choose a corresponding state sequence Q=(q1q2…qT) that is optimal in some sense (i.e., best explains the observations)

50. Decoding • One possibility: • For each hidden state sequence Q • HHH, HHC, HCH, • Compute P(O|Q) • Pick the highest one • Why not? • NT • Instead: • The Viterbi algorithm • Is again a dynamic programming algorithm • Uses a similar trellis to the Forward algorithm