1 / 14

Tree-edit CRFs for RTE

Tree-edit CRFs for RTE. Mengqiu Wang and Chris Manning. Tree-edit CRFs for RTE. Extension to McCallum et al. UAI2005 work on CRFs for finite-state String Edit Distance Key attractions:

martena-boyer
Download Presentation

Tree-edit CRFs for RTE

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Tree-edit CRFs for RTE Mengqiu Wang and Chris Manning

  2. Tree-edit CRFs for RTE • Extension to McCallum et al. UAI2005 work on CRFs for finite-state String Edit Distance • Key attractions: • Models the transformation of dependency parse trees (thus directly models syntax), unlike McCallum et al. ’05, which only models word strings • Discriminatively trained

  3. TE-CRFs model in details • First of all, let’s look at the correspondence between alignment (with constraints) and edit operations

  4. $ root $ root Q: A: substitute root root met VBD is VB substitute subj obj subj with who WP qword leader NN Bush NNP person Jacques Chirac NNP person insert det of Fancy substitute nmod the DT France NNP location president NN substitute delete nmod French JJ location substitute

  5. S2 S2 S3 S2 S1 S3 S2 S3 S1 S2 S3 S1 S3 S1 S1 S1 S2 S3 S1 S2 S1 TE-CRFs model in details • Each valid tree edit operation sequence that transforms one tree into the other corresponds to an alignment. A tree edit operation sequence is models as a transition sequence among a set of states in a FSM D, S, I D, S, I D, S, I S1 S2 D, E, I D, S, I S3 D, S, I D, S, I substitute insert substitute delete substitute substitute … … … … … … …

  6. … … … … … … … … … … … … … … … … … … … … … S2 S3 S2 S2 S2 S2 S3 S3 S3 S1 S3 S2 S1 S1 S3 S2 S1 S1 S1 S1 S1 S2 S1 S3 S1 S3 S1 S1 S1 S1 S2 S1 S1 S3 S1 S1 S1 S1 S1 S1 S1 S2 FSM substitute insert substitute delete substitute substitute … … … … … … … This is for one edit operation sequence substitute insert delete substitute substitute substitute insert substitute substitute delete substitute substitute substitute insert substitute substitute delete substitute There are many other valid edit sequences

  7. D, S, I D, S, I D, S, I D, S, I D, S, I D, S, I S1 S1 S2 S2 D, S, I D, S, I D, S, I D, S, I S3 S3 D, S, I D, S, I D, S, I D, S, I FSM cont. ε ε Positive State Set Start Stop ε ε Negative State Set

  8. FSM transitions Positive State Set … S1 S1 S2 S3 S2 S3 … S3 S3 S2 S3 S1 S2 … … … … … … … S1 S1 S2 S2 S2 … S2 S2 S3 S3 S2 … S1 S3 Stop Start Negative State Set … S1 S1 S2 S3 S2 S3 … S3 S3 S2 S3 S1 S2 … … … … … … … S1 S1 S2 S2 S2 … S2 S2 S3 S3 S2 … S1 S3

  9. What is the semantic interpretation of the FSM states? • At this moment since all the states in the FSM are all fully-connected, it’s unclear what they mean. We fix the number of states to 3, and experiments shows that setting it to 1 or 6 hurts performance. • We are running new experiments with more meaningfully designed FSM topologies, e.g., each states deterministically corresponds to a particular edit operation.

  10. Parameterization substitute S2 S1 positive or negative positive and negative

  11. Training using EM Jensen’s Inequality E-step M-step Using L-BFGS

  12. Features for RTE • Substitution • Same --Word/WordWithNE/Lemma/NETag/Verb/Noun/Adj/Adv/Other • Sub/MisSub -- Punct/Stopword/ModalWord • Antonym/Hypernym/Synonym/Nombank/Country • Different – NE/Pos • Unrelated words • Delete • Stopword/Punct/NE/Other/Polarity/Quantifier/Likelihood/Conditional/If • Insert • Stopword/Punct/NE/Other/Polarity/Quantifier/Likelihood/Conditional/If • Tree • RootAligned/RootAlignedSameWord • Parent,Child,DepRel triple match/mismatch • Date/Time/Numerical • DateMismatch, hasNumDetMismatch, normalizedFormMismatch

  13. Tree-edit CRFs for Textual Entailment • Preliminary results • Trained on RTE2 dev, tested on RTE2 test. • model taken after 50 EM iterations • acc:0.6275, map:0.6407 • SUM, acc=0.675 • QA, acc=0.64 • IR, acc=0.615 • IE, acc=0.58

  14. Work in progress • Implementing a unordered tree-edit algorithm, which would allow swapping of sub-trees • Use Stanford Parser dependency structure. Need to getting rid of cycles in CollapsedDependencyGraph (almost there, only have a few self-loops now). • Experiment with deterministic topologies • More features!!  • Training a separate model for each sub-task (is task information given at test time?)

More Related