1 / 27

PROBABILISTIC CFGs & PROBABILISTIC PARSING

PROBABILISTIC CFGs & PROBABILISTIC PARSING. Universita’ di Venezia 3 Ottobre 2003. Probabilistic CFGs. Context-Free Grammar Rules are of the form: S  NP VP In a Probabilistic CFG, we assign a probability to these rules: S  NP VP, P(SNP,VP|S). Why PCFGs?.

bishop
Download Presentation

PROBABILISTIC CFGs & PROBABILISTIC PARSING

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. PROBABILISTIC CFGs &PROBABILISTIC PARSING Universita’ di Venezia 3 Ottobre 2003

  2. Probabilistic CFGs • Context-Free Grammar Rules are of the form: • S  NP VP • In a Probabilistic CFG, we assign a probability to these rules: • S  NP VP, P(SNP,VP|S)

  3. Why PCFGs? DISAMBIGUATION: with a PCFG, probabilities can be used to choose the most likely parse ROBUSTNESS: rather than excluding things, a PCFG may assign them a very low probability LEARNING: CFGs cannot be learned from positive data only

  4. An example of PCFG

  5. PCFGs in Prolog (courtesy Doug Arnold) s(P0, [s,NP,VP] ) --> np(P1,NP), vp(P2,VP), { P0 is 1.0*P1*P2 }. ….vp(P0, [vp,V,NP] ) --> v(P1,V), np(P2,NP ), { P0 is 0.7*P1*P2 }.

  6. Notation and assumptions

  7. Independence assumptions PCFGs specify a language model, just like n-grams We need however to make some independence assumptions yet again: the probability of a subtree is independent of:

  8. The language model defined by PCFGs

  9. Using PCFGs to disambiguate: “Astronomers saw stars with ears”

  10. A second parse

  11. Choosing among the parses, and the sentence’s probability

  12. Parsing with PCFGs:A comparison with HMMs An HMM defines a REGULAR GRAMMAR:

  13. Parsing with CFGs: A comparison with HMMs

  14. Inside and outside probabilities(cfr. forward and backward probabilities for HMMs)

  15. Parsing with probabilistic CFGs

  16. The algorithm

  17. Example

  18. Initialization

  19. Example

  20. Example

  21. Learning the probabilities: the Treebank

  22. Learning probabilities Reconstruct the rules used in the analysis of the Treebank Estimate probabilities by:P(AB) = C(AB) / C(A)

  23. Probabilistic lexicalised PCFGs(Collins, 1997; Charniak, 2000)

  24. Parsing evaluation

  25. Performance of current parsers

  26. Readings • Manning and Schütze, chapters 11 and 12

  27. Acknowledgments • Some slides and the Prolog code are borrowed from Doug Arnold • Thanks also to Chris Manning & Diego Molla

More Related