1 / 31

Prototype-Driven Grammar Induction

Prototype-Driven Grammar Induction. Aria Haghighi and Dan Klein Computer Science Division University of California Berkeley. Grammar Induction. DT NN VBD DT NN IN NN The screen was a sea of red. First Attempt. DT NN VBD DT NN IN NN

chico
Download Presentation

Prototype-Driven Grammar Induction

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Prototype-Driven Grammar Induction Aria Haghighi and Dan Klein Computer Science Division University of California Berkeley

  2. Grammar Induction DTNNVBD DT NN IN NN The screen was a sea of red

  3. First Attempt DTNNVBD DT NN IN NN The screen was a sea of red

  4. Central Questions • How do we specify what we want to learn? • How do we fix observed errors? What’s an NP? That’s not quite it!

  5. Experimental Set-up • Binary Grammar { X1, X2, … Xn} plus POS tags • Data • WSJ-10 [7k sentences] • Evaluate on Labeled F1 • Grammar Upper Bound: 86.1 Xi Xj Xk

  6. Experiment Roadmap • Unconstrained Induction • Need bracket constraint! • Gold Bracket Induction • Prototypes and Similarity • CCM Bracket Induction

  7. Unconstrained PCFG Induction (Outside) 0 i j n (Inside) • Learn PCFG with EM • Inside-Outside Algorithm • Lari & Young [93] • Results

  8. Constrained PCFG Induction • Gold Brackets • Periera & Schables [93] • Result

  9. Encoding Knowledge What’s an NP? Semi-Supervised Learning

  10. Encoding Knowledge What’s an NP? For instance, DT NN JJ NNS NNP NNP Prototype Learning

  11. Grammar Induction Experiments • Add Prototypes • Manually constructed

  12. How to use prototypes? S ? VP ? PP ? NP ? NP ? NN koala VBD sat IN in DT the NN tree DT The ¦ ¦

  13. How to use prototypes? S VP NP ? PP NP JJ hungry NN koala VBD sat IN in DT the NN tree DT The ¦ ¦

  14. Distributional Similarity • Context Distribution  (DT JJ NN) = { ¦ __ VBD : 0.3, VBD __ ¦ : 0.2, IN __ VBD: 0.1, ….. } • Similarity  (DT NN)  (NNP NNP)  (DT JJ NN)  (JJ NNS) NP

  15. Distributional Similarity • Prototype Approximation (NP) ¼ Uniform ( (DT NN), (JJ NNS), (NNP NNP) ) • Prototype Similarity Feature • span(DT JJ NN) emits proto=NP • span(MD NNS) emits proto=NONE

  16. Prototype CFG+ Model S P (DT NP | NP) P (proto=NP | NP) VP NP PP NP NP JJ hungry NN koala VBD sat IN in DT the NN tree DT The ¦ ¦

  17. Prototype CFG+ Induction • Experimental Set-Up • BLIPP corpus • Gold Brackets • Results

  18. Summary So Far • Bracket constraint and prototypes give good performance!

  19. Constituent-Context Model

  20. Product Model • Different Aspects of Syntax • CCM : Yield and Context properties • CFG: Hierarchical properties • Intersected EM [Klein 2005] • Encourages mass on trees compatible with CCM and CFG

  21. Grammar Induction Experiments • Intersected CFG and CCM • No prototypes • Results

  22. Grammar Induction Experiments • Intersected CFG+ and CCM • Add Prototypes • Results

  23. Reacting to Errors • Possessive NPs Our Tree Correct Tree

  24. Reacting to Errors • Add Prototype: NP-POS NN POS New Analysis

  25. Error Analysis • Modal VPs Our Tree Correct Tree

  26. Reacting to Errors • Add Prototype: VP-INF VB NN New Analysis

  27. Fixing Errors • Supplement Prototypes • NP-POS and VP-INF • Results

  28. Results Summary

  29. Conclusion • Prototype-Driven Learning Flexible Weakly Supervised Framework • Merged distributional clustering techniques with supervised structured models

  30. Thank You! http://www.cs.berkeley.edu/~aria42

  31. Unconstrained PCFG Induction (Outside) 0 i j n (Inside) Xi Xi Xi Xi • Binary Grammar { X1, X2, … Xn} • Learn PCFG with EM • Inside-Outside Algorithm • Lari & Young [93] Xj Xk N Xk Xj V N V

More Related