1 / 128

Tree-based and Forest-based Translation

Tree-based and Forest-based Translation. Yang Liu. Liang Huang. Institute of Computing Technology Chinese Academy of Sciences. Information Sciences Institute University of Southern California. Outline. Part 1: Tree-based Translation Overview and Motivation

neith
Download Presentation

Tree-based and Forest-based Translation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Tree-based and Forest-based Translation Yang Liu Liang Huang Institute of Computing Technology Chinese Academy of Sciences Information Sciences Institute University of Southern California ACL 2010 Tutorial, Uppsala, Sweden

  2. Outline • Part 1: Tree-based Translation • Overview and Motivation • Tree-to-String Model and Decoding • Tree-to-String Rule Extraction • Language Model-Integrated Decoding: Cube Pruning • Part 2: Forest-based Translation • Packed Forest • Forest-based Decoding • Forest-based Rule Extraction • Part 3: Extensions • Tree-to-Tree Translation • Tree Sequence-based Translation • Joint Parsing and Translation • Part 4: Conclusion ACL 2010 Tutorial, Uppsala, Sweden

  3. Я люблю тебя 당신을 사랑합니다 אני אוהב אותך من شما را دوست دارم Tôi yêu bạn Miluji tě Ti amo ผมรักคุณ わたしは、あなたを愛しています Ik hou van je Jag älskar dig Natural Languages are Different I love you 我爱你 Eu te amo Je t'aime Ich liebe dich Te quiero By Google Translate ACL 2010 Tutorial, Uppsala, Sweden

  4. connocting poopie HELP ONESELF TERMINATING MACHINE Translation is Hard! ACL 2010 Tutorial, Uppsala, Sweden

  5. 布什 与 沙龙 举行 了 会谈 bushi yu shalong juxing le huitan Bush held a talk with Sharon Machine Translation ACL 2010 Tutorial, Uppsala, Sweden

  6. Word-based MT bushi yu shalong juxing le huitan Bush held a talk with Sharon (Brown et al., 1993) ACL 2010 Tutorial, Uppsala, Sweden

  7. held a talk with Sharon Phrase-based MT bushi yu shalong juxing le huitan Bush (Koehn et al., 2003; Och and Ney, 2004) ACL 2010 Tutorial, Uppsala, Sweden

  8. X3 X2 X1 held a talk with Sharon X1 X2 X3 Hierarchical Phrase-based MT bushi yu shalong juxing le huitan Bush X3->(X1yu shalong X2, X1 X2with Sharon) (Chiang, 2005; Chiang, 2007) ACL 2010 Tutorial, Uppsala, Sweden

  9. NNP VBD DT NN IN NNP NP NP NP PP VP S Syntax-based MT bushi yu shalong juxing le huitan Bush held a talk with Sharon (Yamda and Knight, 2001; Galley et al., 2006; Shen et al., 2008) ACL 2010 Tutorial, Uppsala, Sweden

  10. Motivation • Human Translation • Understand the source sentence • Generate the target sentence • Compiling • Parse input program into a syntax tree • Generate code in machine language ACL 2010 Tutorial, Uppsala, Sweden

  11. Syntax-Directed Translation for Compiling • Input: y:=3*x+z • Parsing: := id + (y) id * (z) const id (3) (x) (Irons, 1961; Lewis and Stearns, 1968; Aho and Ullman., 1972) ACL 2010 Tutorial, Uppsala, Sweden

  12. Motivation • Human Translation • Understand the source sentence • Generate the target sentence • Compiling • Parse input program into a syntax tree • Generate code in machine language • Machine Translation • Parse the source sentence into a tree • Recursively transfer the tree into the target language ACL 2010 Tutorial, Uppsala, Sweden

  13. Syntax-Directed Translation for MT • Input: bushi yu shalong juxing le huitan • Parsing: IP VP PP VPB NPB P NPB VS AS NPB bushi yu shalong juxing le huitan (Huang et al., 2006) ACL 2010 Tutorial, Uppsala, Sweden

  14. Outline • Part 1: Tree-based Translation • Overview and Motivation • Tree-to-String Model and Decoding • Tree-to-String Rule Extraction • Language Model-Integrated Decoding: Cube Pruning • Part 2: Forest-based Translation • Packed Forest • Forest-based Decoding • Forest-based Rule Extraction • Part 3: Extensions • Tree-to-Tree Translation • Tree Sequence-based Translation • Joint Parsing and Translation • Part 4: Conclusion ACL 2010 Tutorial, Uppsala, Sweden

  15. IP X1:NPB X2:VP X1 X2 Tree-to-String Translation • Recursive rewrite by pattern-matching IP VP PP VPB NPB P NPB VS AS NPB bushi yu shalong juxing le huitan (Liu et al., 2006; Huang et al., 2006) ACL 2010 Tutorial, Uppsala, Sweden

  16. NPB bushi Bush Tree-to-String Translation • Recursive rewrite by pattern-matching VP PP VPB NPB P NPB VS AS NPB bushi yu shalong juxing le huitan (Liu et al., 2006; Huang et al., 2006) ACL 2010 Tutorial, Uppsala, Sweden

  17. VP X1:PP X2:VPB X2 X1 Tree-to-String Translation • Recursive rewrite by pattern-matching VP PP VPB P NPB VS AS NPB Bush yu shalong juxing le huitan (Liu et al., 2006; Huang et al., 2006) ACL 2010 Tutorial, Uppsala, Sweden

  18. PP VPB VS AS X1:NPB P NPB juxing le held a X1 yu shalong Tree-to-String Translation • Recursive rewrite by pattern-matching VPB VS AS NPB Bush juxing le huitan (Liu et al., 2006; Huang et al., 2006) ACL 2010 Tutorial, Uppsala, Sweden

  19. PP NPB huitan P NPB talk yu shalong Tree-to-String Translation • Recursive rewrite by pattern-matching NPB Bush held a huitan (Liu et al., 2006; Huang et al., 2006) ACL 2010 Tutorial, Uppsala, Sweden

  20. PP PP P X1:NPB P NPB yu with X1 yu shalong Tree-to-String Translation • Recursive rewrite by pattern-matching Bush held a talk (Liu et al., 2006; Huang et al., 2006) ACL 2010 Tutorial, Uppsala, Sweden

  21. NPB shalong Sharon Tree-to-String Translation • Recursive rewrite by pattern-matching NPB Bush held a talk with shalong (Liu et al., 2006; Huang et al., 2006) ACL 2010 Tutorial, Uppsala, Sweden

  22. Syntax-directed translation (e.g., Irons, 1961) Tree transducer (e.g., Knight and Graehl, 2005) Synchronous grammar (e.g., Eisner, 2003) … Tree-to-string translation Tree-to-String Translation • Recursive rewrite by pattern-matching Bush held a talk with Sharon (Liu et al., 2006; Huang et al., 2006) ACL 2010 Tutorial, Uppsala, Sweden

  23. VPB PP PP VS AS X1:NPB LCP P NPB P X1:IP LC juxing le yu shalong dang hou held a X1 with Sharon when X1 NP IP QP DNP X2:NP X1:NP VP X1:CD CLP X1:NP DEG X2:IP X3:VPB ben de X1 X1 X3 X2 X2 of X1 Expressive Power phrasal translation non-constituent phrase non-contiguous phrase word omission multilevel re-ordering lexicalized re-ordering (Knight and Graehl, 2005) ACL 2010 Tutorial, Uppsala, Sweden

  24. Outline • Part 1: Tree-based Translation • Overview and Motivation • Tree-to-String Model and Decoding • Tree-to-String Rule Extraction • Language Model-Integrated Decoding: Cube Pruning • Part 2: Forest-based Translation • Packed Forest • Forest-based Decoding • Forest-based Rule Extraction • Part 3: Extensions • Tree-to-Tree Translation • Tree Sequence-based Translation • Joint Parsing and Translation • Part 4: Conclusion ACL 2010 Tutorial, Uppsala, Sweden

  25. Tree-to-String Rule Extraction • Compute target spans IP “Bush … Sharon” VP “held … Sharon” PP VPB “with Sharon” “held a talk” NPB P NPB VS AS NPB “Bush” “with” “Sharon” “held” “held” “talk” bushi yu shalong juxing le huitan Bush held a talk with Sharon (Galley et al., 2004) ACL 2010 Tutorial, Uppsala, Sweden

  26. Tree-to-String Rule Extraction • Find admissible nodes IP “Bush … Sharon” VP “held … Sharon” PP VPB “with Sharon” “held a talk” NPB P NPB VS AS NPB “Bush” “with” “Sharon” “held” “held” “talk” bushi yu shalong juxing le huitan Bush held a talk with Sharon (Galley et al., 2004) ACL 2010 Tutorial, Uppsala, Sweden

  27. NPB bushi Bush Tree-to-String Rule Extraction • Extract minimal rules IP “Bush … Sharon” VP “held … Sharon” PP VPB “with Sharon” “held a talk” NPB P NPB VS AS NPB “Bush” “with” “Sharon” “held” “held” “talk” bushi yu shalong juxing le huitan Bush held a talk with Sharon (Galley et al., 2004) ACL 2010 Tutorial, Uppsala, Sweden

  28. VP X1:PP X2:VPB X2 X1 Tree-to-String Rule Extraction • Extract minimal rules IP “Bush … Sharon” VP “held … Sharon” PP VPB “with Sharon” “held a talk” NPB P NPB VS AS NPB “Bush” “with” “Sharon” “held” “held” “talk” bushi yu shalong juxing le huitan Bush held a talk with Sharon (Galley et al., 2004) ACL 2010 Tutorial, Uppsala, Sweden

  29. VPB VS AS X1:NPB juxing le held a X1 Tree-to-String Rule Extraction • Extract minimal rules IP “Bush … Sharon” VP “held … Sharon” PP VPB “with Sharon” “held a talk” NPB P NPB VS AS NPB “Bush” “with” “Sharon” “held” “held” “talk” bushi yu shalong juxing le huitan Bush held a talk with Sharon (Galley et al., 2004) ACL 2010 Tutorial, Uppsala, Sweden

  30. VPB NPB VS AS X1:NPB huitan juxing le VPB talk held a X1 VS AS NPB juxing le huitan held a talk Tree-to-String Rule Extraction • Get composed rules + = tree substitution (Galley et al., 2006) ACL 2010 Tutorial, Uppsala, Sweden

  31. Outline • Part 1: Tree-based Translation • Overview and Motivation • Tree-to-String Model and Decoding • Tree-to-String Rule Extraction • Language Model-Integrated Decoding: Cube Pruning • Part 2: Forest-based Translation • Packed Forest • Forest-based Decoding • Forest-based Rule Extraction • Part 3: Extensions • Tree-to-Tree Translation • Tree Sequence-based Translation • Joint Parsing and Translation • Part 4: Conclusion ACL 2010 Tutorial, Uppsala, Sweden

  32. NPB bushi Bush Bottom-up Decoding IP VP PP VPB NPB P NPB VS AS NPB Bush bushi yu shalong juxing le huitan (Liu et al., 2006; Huang et al., 2006) ACL 2010 Tutorial, Uppsala, Sweden

  33. NPB shalong Sharon Bottom-up Decoding IP VP PP VPB Sharon NPB P NPB VS AS NPB Bush bushi yu shalong juxing le huitan (Liu et al., 2006; Huang et al., 2006) ACL 2010 Tutorial, Uppsala, Sweden

  34. NPB huitan talk Bottom-up Decoding IP VP PP VPB Sharon talk NPB P NPB VS AS NPB Bush bushi yu shalong juxing le huitan (Liu et al., 2006; Huang et al., 2006) ACL 2010 Tutorial, Uppsala, Sweden

  35. PP P X1:NPB yu with X1 Bottom-up Decoding IP VP with Sharon PP VPB Sharon talk NPB P NPB VS AS NPB Bush bushi yu shalong juxing le huitan (Liu et al., 2006; Huang et al., 2006) ACL 2010 Tutorial, Uppsala, Sweden

  36. VPB VS AS X1:NPB juxing le held a X1 Bottom-up Decoding IP VP held a talk with Sharon PP VPB Sharon talk NPB P NPB VS AS NPB Bush bushi yu shalong juxing le huitan (Liu et al., 2006; Huang et al., 2006) ACL 2010 Tutorial, Uppsala, Sweden

  37. VP X1:PP X2:VPB X2 X1 Bottom-up Decoding IP held a talk with Sharon VP held a talk with Sharon PP VPB Sharon talk NPB P NPB VS AS NPB Bush bushi yu shalong juxing le huitan (Liu et al., 2006; Huang et al., 2006) ACL 2010 Tutorial, Uppsala, Sweden

  38. IP X1:NPB X2:VP X1 X2 Bottom-up Decoding Bush held a talk with Sharon IP held a talk with Sharon VP held a talk with Sharon PP VPB Sharon talk NPB P NPB VS AS NPB Bush bushi yu shalong juxing le huitan (Liu et al., 2006; Huang et al., 2006) ACL 2010 Tutorial, Uppsala, Sweden

  39. Beam Search IP VP PP VPB NPB P NPB VS AS NPB bushi yu shalong juxing le huitan (Liu et al., 2006; Huang et al., 2006) ACL 2010 Tutorial, Uppsala, Sweden

  40. held a talk with Sharon held a talk and Sharon held talks with Sharon held talks and Sharon … with Sharon held a talk and Sharon held talks Sharon with hold a talk Sharon and hold talks Exhaustive Search VP X1:PP X2:VPB VP1,6 X2 X1 PP1,3 VPB3,6 ACL 2010 Tutorial, Uppsala, Sweden

  41. Update Bigram LM Probability p1=p(“with’’) * p(“Sharon”|”with”) p2=p(“held’’) * p(“a”|”held”)*p(“talk”|”a”) with Sharon held a talk Only boundary words are used to update LM probability! with Sharonheld a talk p1*p2*p(“held”|”Sharon”)/p(“held”) held a talkwith Sharon p1*p2*p(“with”|”talk”)/p(“with”) ACL 2010 Tutorial, Uppsala, Sweden

  42. Exhaustive Search with a Bigram Language Model held * Sharon VP hold * Sharon X1:PP X2:VPB VP1,6 held * with hold * with X2 X1 … PP1,3 VPB3,6 with * Sharon held * talk and * Sharon held * talks Sharon * with hold * talk Sharon * and hold * talks ACL 2010 Tutorial, Uppsala, Sweden

  43. VP1,6 PP1,3 VPB3,6 Monotonicity PP1,3 and * Sharon Sharon * with Sharon * and with * Sharon monotonic 1.0 3.0 4.0 6.5 1.0 2.0 4.0 5.0 7.5 held * talk 1.1 2.1 4.1 5.1 7.6 held * talks VPB3,6 2.0 3.0 5.0 6.0 8.5 hold * talk 3.5 4.5 6.5 7.5 10.0 hold * talks (Huang and Chiang, 2005, 2007; Chiang, 2007) ACL 2010 Tutorial, Uppsala, Sweden

  44. VP1,6 PP1,3 VPB3,6 Non-Monotonicity PP1,3 log(p(with|talk))-log(p(with)) and * Sharon Sharon * with Sharon * and with * Sharon LM introduces non-monotonicity 1.0 3.0 4.0 6.5 1.0 2.0 + 0.5 4.0 + 2.0 5.0 + 4.0 7.5 + 4.0 held * talk 1.1 2.1 + 0.3 4.1 + 1.5 5.1 + 3.5 7.6 + 3.0 held * talks VPB3,6 2.0 3.0 + 0.5 5.0 + 2.0 6.0 + 4.0 8.5 + 4.0 hold * talk 3.5 4.5 + 0.3 6.5 + 1.5 7.5 + 3.5 10 + 3.5 hold * talks (Huang and Chiang, 2005, 2007; Chiang, 2007) ACL 2010 Tutorial, Uppsala, Sweden

  45. Cube Pruning PP1,3 queue and * Sharon Sharon * with Sharon * and with * Sharon 4-best 1.0 3.0 4.0 6.5 1.0 2.5 6.0 9.0 11.5 held * talk 1.1 2.4 5.6 8.6 10.6 held * talks VPB3,6 2.0 3.5 7.0 10.0 12.5 hold * talk 3.5 4.8 8.0 11.0 13.5 hold * talks (Huang and Chiang, 2005, 2007; Chiang, 2007) ACL 2010 Tutorial, Uppsala, Sweden

  46. Cube Pruning PP1,3 queue 2.5 and * Sharon Sharon * with Sharon * and with * Sharon 4-best 1.0 3.0 4.0 6.5 1.0 2.5 6.0 9.0 11.5 held * talk 1.1 2.4 5.6 8.6 10.6 held * talks VPB3,6 2.0 3.5 7.0 10.0 12.5 hold * talk 3.5 4.8 8.0 11.0 13.5 hold * talks (Huang and Chiang, 2005, 2007; Chiang, 2007) ACL 2010 Tutorial, Uppsala, Sweden

  47. Cube Pruning PP1,3 queue 2.4 6.0 and * Sharon Sharon * with Sharon * and with * Sharon 4-best 2.5 1.0 3.0 4.0 6.5 1.0 2.5 6.0 9.0 11.5 held * talk 1.1 2.4 5.6 8.6 10.6 held * talks VPB3,6 2.0 3.5 7.0 10.0 12.5 hold * talk 3.5 4.8 8.0 11.0 13.5 hold * talks (Huang and Chiang, 2005, 2007; Chiang, 2007) ACL 2010 Tutorial, Uppsala, Sweden

  48. Cube Pruning PP1,3 queue 3.5 5.6 6.0 and * Sharon Sharon * with Sharon * and with * Sharon 4-best 2.4 2.5 1.0 3.0 4.0 6.5 1.0 2.5 6.0 9.0 11.5 held * talk 1.1 2.4 5.6 8.6 10.6 held * talks VPB3,6 2.0 3.5 7.0 10.0 12.5 hold * talk 3.5 4.8 8.0 11.0 13.5 hold * talks (Huang and Chiang, 2005, 2007; Chiang, 2007) ACL 2010 Tutorial, Uppsala, Sweden

  49. Cube Pruning PP1,3 queue 4.8 5.6 6.0 7.0 and * Sharon Sharon * with Sharon * and with * Sharon 4-best 2.4 2.5 3.5 1.0 3.0 4.0 6.5 1.0 2.5 6.0 9.0 11.5 held * talk 1.1 2.4 5.6 8.6 10.6 held * talks VPB3,6 2.0 3.5 7.0 10.0 12.5 hold * talk 3.5 4.8 8.0 11.0 13.5 hold * talks (Huang and Chiang, 2005, 2007; Chiang, 2007) ACL 2010 Tutorial, Uppsala, Sweden

  50. Cube Pruning PP1,3 queue 5.6 6.0 7.0 and * Sharon Sharon * with Sharon * and with * Sharon 4-best 2.4 2.5 3.5 4.8 1.0 3.0 4.0 6.5 1.0 2.5 6.0 9.0 11.5 held * talk 1.1 2.4 5.6 8.6 10.6 held * talks VPB3,6 2.0 3.5 7.0 10.0 12.5 hold * talk 3.5 4.8 8.0 11.0 13.5 hold * talks (Huang and Chiang, 2005, 2007; Chiang, 2007) ACL 2010 Tutorial, Uppsala, Sweden

More Related