1 / 31

Prototype-Driven Grammar Induction

Prototype-Driven Grammar Induction. Aria Haghighi and Dan Klein Computer Science Division University of California Berkeley. Grammar Induction. DT NN VBD DT NN IN NN The screen was a sea of red. First Attempt…. DT NN VBD DT NN IN NN

millie
Download Presentation

Prototype-Driven Grammar Induction

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Prototype-Driven Grammar Induction Aria Haghighi and Dan Klein Computer Science Division University of California Berkeley

  2. Grammar Induction DTNNVBD DT NN IN NN The screen was a sea of red

  3. First Attempt….. DTNNVBD DT NN IN NN The screen was a sea of red

  4. Central Questions • How do we specify what we want to learn? • How do we fix observed errors? What’s an NP? That’s not quite it!

  5. Experimental Set-up • Binary Grammar { X1, X2, … Xn} plus POS tags • Data • WSJ-10 [7k sentences] • Evaluate on Labeled F1 • Grammar Upper Bound: 86.1 Xi Xj Xk

  6. Experiment Roadmap • Unconstrained Induction • Need bracket constraint • Gold Bracket Induction • Prototypes and Similarity • CCM Bracket Induction

  7. Unconstrained PCFG Induction (Outside) 0 i j n (Inside) • Learn PCFG with EM • Inside-Outside Algorithm • Lari & Young [90] • Results

  8. Constrained PCFG Induction • Gold Brackets • Pereira & Schabes [93] • Result

  9. Encoding Knowledge What’s an NP? Semi-Supervised Learning

  10. Encoding Knowledge What’s an NP? For instance, DT NN JJ NNS NNP NNP Prototype Learning

  11. Grammar Induction Experiments • Add Prototypes • Manually constructed

  12. How to use prototypes? S ? VP ? PP ? NP ? NP ? NN koala VBD sat IN in DT the NN tree DT The ¦ ¦

  13. How to use prototypes? S VP NP ? PP NP JJ hungry NN koala VBD sat IN in DT the NN tree DT The ¦ ¦

  14. Distributional Similarity proto=NP  (VBD DT NN)  (IN NN)  (DT NN)  (VBN IN NN)  (TO CD CD) VP PP  (NNP NNP)  (MD VB NN)  (IN PRP) NP  (JJ NNS) { ¦ __ VBD : 0.3, VBD __ ¦ : 0.2, IN __ VBD: 0.1, ….. }  (DT JJ NN)

  15. Distributional Similarity  (VBD DT NN)  (IN NN)  (DT NN)  (VBN IN NN)  (TO CD CD) VP PP  (NNP NNP)  (MD VB NN)  (IN PRP) NP proto=NONE  (JJ NNS)  (IN DT)

  16. Prototype CFG+ Model CFG Rule S P (DT NP | NP) P (proto=NP | NP) VP NP Proto Feature PP NP NP JJ hungry NN koala DT the NN tree DT The VBD sat IN in ¦ ¦

  17. Prototype CFG+ Induction • Experimental Set-Up • BLIPP corpus • Gold Brackets • Results

  18. Summary So Far • Bracket constraint and prototypes give good performance!

  19. Constituent-Context Model Yield + P(VBD IN DT NN | +) P(NN__ ¦ | +) Context JJ hungry NN koala DT the NN tree DT The VBD sat IN in ¦ ¦ P(NN VBD| -) P(JJ __IN| -) - Klein & Manning ‘02

  20. Product Model • Different Aspects of Syntax • Intersected EM [Klein 2005, Liang et. al. ‘06] • Encourages mass on trees compatible with CCM and CFG DT NN + yield ¦ _VBD + context NP ! NP PP CFG CCM

  21. Grammar Induction Experiments • Intersected CFG and CCM • No prototypes • Results

  22. Grammar Induction Experiments • Intersected CFG+ and CCM • Add Prototypes • Results

  23. Reacting to Errors Our Tree Correct Tree • Possessive NPs

  24. Reacting to Errors New Analysis • Add Prototype: NP-POS NN POS

  25. Error Analysis Our Tree Correct Tree • Modal VPs

  26. Reacting to Errors New Analysis • Add Prototype: VP-INF VB NN

  27. Fixing Errors • Supplement Prototypes • NP-POS and VP-INF • Results

  28. Results Summary

  29. Conclusion • Prototype-Driven Learning Flexible Weakly Supervised Framework • Merged distributional clustering techniques with structured models

  30. Thank You! http://www.cs.berkeley.edu/~aria42

  31. Unconstrained PCFG Induction (Outside) 0 i j n (Inside) Xi Xi Xi Xi • Binary Grammar { X1, X2, … Xn} • Learn PCFG with EM • Inside-Outside Algorithm • Lari & Young [93] Xj Xk N Xk Xj V N V

More Related