a new string-to- dependency mt algorithm with a target dependency language model

1. A NewString-to-Dependency MTAlgorithm with a Target Dependency Language Model Libin Shen Jinxi Xu Ralph Weischedel

3. Dependency Structure Dependency represents the syntactic relations of the words in a sentence John leaves for Boston

4. String to Dependency Translation Translating from source-string to target-dependency

5. Prior Work on Using Linguistic Info in SMT Negative Evidence Syntax-MT team at JHU Summer Workshop 2003 Implemented 450 linguistics-motivated feature functions Almost none of them provided improvement in MT reranking Positive Evidence ISI�s syntax based system Translating source string into target CFG tree structure State-of-the-art performance in recent NIST evaluations Lesson Learned To use syntactic information in reranking does not help Difficult to restore syntactic structure from messy translations To use syntactic structure in decoding works Employing features related to the meaning of the text better

6. Motivation: To explicitly use syntactic information to measure the quality of translation in decoding Our Contribution: To use dependency LM as a feature P(T) = Proot ( leave ) � Pleft ( will | leave_H ) � Pleft ( John | will, leave_H ) � Pright( for | leave_H ) � Pright( with | for, leave_H ) � Pright( tomorrow | with, for ) � Pright( Boston | for_H ) � Pright( Mike | with_H ) Dependency Language Model

7. Outline Introduction Dependency structure String-to-dependency translation Dependency language model Overview of BBN�s String-to-Dependency MT System Rule extraction algorithm Decoding algorithm Key Features of the Decoding Algorithm Use of well-formed dependency structures Use of dependency LM in decoding Experiments and Conclusion

8. The HierDec System

9. The HierDec System HierDec is BBN�s Hierarchical MT Decoder From source-string to target-dependency-structure Extended the string-to-string approach of Hiero (Chiang, 2005) Main Components Rule extractor Input: bi-lingual training data with GIZA alignment and target parse trees Output: string to dependency transfer rules, e.g. Decoder A chart parsing algorithm that produces a shared forest of target dependency trees

10. Decoding as Chart Parsing

11. Outline Introduction Dependency structure String-to-dependency translation Dependency language model Overview of BBN�s String-to-Dependency MT System Rule extraction algorithm Decoding algorithm Key Features of the Decoding Algorithm Use of well-formed dependency structures Use of dependency LM in decoding Experiments and Conclusion

12. Need for Non-constituent Rules Rules based on complete constituents cannot represent many phrasal translations, e.g. ?? ? trade and economic

13. Prior Work on Non-constituent Rules Crucial to improve the coverage of non-constituent rules in tree based MT (DeNeefe et al., 2007) Prior work to handle this problem via tree rewriting

14. Data Structure for Dependency Requirement: Trade off between flexibility and complexity To cover non-constituent transfer rules, as just explained To exclude multi-level irregular dependency structures Critical for efficient dynamic programming To allow dependency LM score calculation on partial structures in decoding Our solution: well-formed dependency structures Linguistically motivated, to allow structures to represent modifiers for the same head word Three types of well-formed structures as building blocks Compositional operations to build larger well-formed structures

15. Three types of well-formed dependency structures Well-formed Dependency Structures

16. Dependency LM in Decoding

17. Outline Introduction Dependency structure String-to-dependency translation Dependency language model Overview of BBN�s String-to-Dependency MT System Rule extraction algorithm Decoding algorithm Key Features of the Decoding Algorithm Use of well-formed dependency structures Use of dependency LM in decoding Experiments and Conclusion

18. Experiments on MT04 Data Test: MT04 Chinese-English, Tuning: MT05, Training: GALE data Baseline: Our replication of Hiero (Chiang, 2005) 1.48 BLEU and 2.53 TER improvement before reranking 1.21 BLEU and 1.19 TER improvement after reranking Rule reduction by 80% thanks to the well-formed constraint

19. Related Work Relation to the multi-head structures used in parsing (Eisner and Satta, 1999) and (McDonald et. al. 2005) In parsing, there is no artificial ambiguity of derivation Our model is designed to memorize fragments of translation in the training data Charniak et. al. (2003) described a syntax-based LM in string-to-CFG-tree translation A two-step model due to computational complexity Using P(F|E) to generate the target CFG forest Using structural P(E) x P(F|E) to pick the top translation Our dependency LM models lexical items directly Compact enough to be incorporated into decoding

20. Conclusions A new model of string-to-dependency MT Use of multi-head dependency structures in rules Efficient dynamic programming Incorporated a target dependency LM in decoding To exploit long distance relations An interesting step towards understanding based MT Significant improvement over a baseline hierarchical phrase-based MT system on BLEU and TER Rule reduction by 80% thanks to the constraint of well-formed dependency Easy to be adapted to X-English translations

21. Thank You !