380 likes | 448 Views
Modified Distortion Matrices for Phrase-Based SMT. Arianna Bisazza & Marcello Federico – FBK (Italy). PSMT decoding overview. E ' necessario incoraggiare tale mobilità garantendo la sicurezza dei percorsi professionali. 2.
E N D
Modified Distortion Matrices for Phrase-Based SMT Arianna Bisazza & Marcello Federico – FBK (Italy)
PSMT decoding overview E' necessario incoraggiare tale mobilità garantendo la sicurezza dei percorsi professionali 2 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
PSMT decoding overview ReoM scores ReoM scores E' necessario incoraggiare tale mobilità garantendo la sicurezza dei percorsi professionali TM scores TM scores Freedom of movement must be encouraged LM scores LM scores 3 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
PSMT decoding overview ReoM scores ReoM scores ReoM scores ReoM scores E' necessario incoraggiare tale mobilità garantendo la sicurezza dei percorsi professionali TM scores TM scores TM scores TM scores career paths … while ensuring that Freedom of movement must be encouraged LM scores LM scores LM scores LM scores 4 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
PSMT decoding overview E' necessario incoraggiare tale mobilità garantendo la sicurezza dei percorsi professionali ReoM scores ReoM scores ReoM scores ReoM scores Freedom of movement must be encouraged while ensuring that career paths … TM scores TM scores TM scores TM scores LM scores LM scores LM scores LM scores 5 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
Reordering Models Tillman 04,Zens & Ney 06 AlOnaizan & Papineni 06 Galley & Manning 08 Green & al.10, Feng & al.10 … Many solutions have been proposed with different reo. classes, features, train modes etc. E' necessario incoraggiare tale mobilità garantendo la sicurezza dei percorsi professionali ReoM scores ReoM scores ReoM scores ReoM scores 6 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
Reordering Models Tillman04, Zens&Ney06 AlOnaizan & Papineni06 Galley & Manning08 Green &al.10, Feng &al.10 … Tillman 04,Zens & Ney 06 AlOnaizan & Papineni 06 Galley & Manning 08 Green & al.10, Feng & al.10 … Many solutions have been proposed with different reo. classes, features, train modes etc. E' necessario incoraggiare tale mobilità garantendo la sicurezza dei percorsi professionali ReoM scores ReoM scores ReoM scores ReoM scores No matter what reordering model is used, permutation search space must be limited! The power of all reordering models is bound to the reordering constraints in use 7 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
E' necessario incoraggiare tale mobilità garantendo la sicurezza dei percorsi professionali ReoM scores ReoM scores ReoM scores ReoM scores 8 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
ReoM scores ReoM scores ReoM scores ReoM scores E' necessario incoraggiare tale mobilità garantendo la sicurezza dei percorsi professionali Reordering Constraints #perm.=11!≈40,000,000 9 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
E' necessario incoraggiare tale mobilità garantendo la sicurezza dei percorsi professionali Reordering Constraints #perm.=11!≈40,000,000 D(x,y)=|y-x-1| Source-to-Source distortion 10 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
E' necessario incoraggiare tale mobilità garantendo la sicurezza dei percorsi professionali DL: distortion limit Reordering Constraints #perm.=11!≈40,000,000 D(x,y)=|y-x-1| DL=3 #perm.≈7,000 Source-to-Source distortion 11 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
The problem with DL… Arabic-English EN EN AR AR 12 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
The problem with DL… German-English EN EN DE DE 13 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
#perm.=11! ≈40,000,000 D(x,y)=|y-x-1| DL=3 #perm.≈7,000 Current solution: increase the DLimit Source-to-Source distortion 14 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
#perm.=11! ≈40,000,000 D(x,y)=|y-x-1| DL=3 #perm.≈7,000 DL=7 #perm.≈7,000,000 Current solution: increase the DLimit Generally leads to worse translations! Source-to-Source distortion 15 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
Our solution: #perm.=11! ≈40,000,000 D(x,y)=|y-x-1| DL=3 #perm.≈7,000 DL=7 #perm.≈7,000,000 Source-to-Source distortion 16 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
Our solution: modify distortion for each test sentence #perm.=11! ≈40,000,000 D(x,y)=|y-x-1| DL=3 #perm.≈7,000 DL=7 #perm.≈7,000,000 DL=3 & modif(D) #perm.≈20,000 Simplifies the task of reordering models! Source-to-Source distortion 17 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT Rest of the talk: How to modify the distortion matrix? What effect on translation quality? What effect on baseline runtimes?
Chunk-based fuzzy reordering rules • Shallow syntax chunking: • cheaper and easier than deep parsing • constrains reorderings in a softer way • Fuzzy(non-determinisic) reordering rules: • generate N permutations for each matching sequence • final reordering decision is taken during translation, • guided by all SMT models (reoM, LM...) • Few rules for language pair, to only capture long reordering 19 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
Chunk-based fuzzy reordering rules Arabic-English “Move verb chunk (and following chunk) to the right by 1 to N chunks” w- $ArkfyAltZAhrpE$rAtAlmslHynmnAlktA}b . and took part in the march dozens of militants from the Brigades CC1 VC2 PC3 NC4 PC5 Pct6 20 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
Chunk-based fuzzy reordering rules Arabic-English “Move verb chunk (and following chunk) to the right by 1 to N chunks” w- $ArkfyAltZAhrpE$rAtAlmslHynmnAlktA}b . and took part in the march dozens of militants from the Brigades CC1 VC2 PC3 NC4 PC5 Pct6 Pct6 CC1 PC3 VC2 NC4 PC5 NC4 Pct6 PC5 CC1 PC3 VC2 PC3 NC4 PC5 VC2 CC1 Pct6 21 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
Chunk-based fuzzy reordering rules Arabic-English “Move verb chunk (and following chunk) to the right by 1 to N chunks” w- $ArkfyAltZAhrpE$rAtAlmslHynmnAlktA}b . and took part in the march dozens of militants from the Brigades CC1 VC2 PC3 NC4 PC5 Pct6 Pct6 CC1 PC3 VC2 NC4 PC5 NC4 Pct6 PC5 CC1 PC3 VC2 PC3 NC4 PC5 VC2 CC1 Pct6 CC1 NC4 VC2 PC3 PC5 Pct6 CC1 NC4 PC5 VC2 PC3 Pct6 22 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
Chunk-based fuzzy reordering rules Reordered source LM Reordering selection w- $ArkfyAltZAhrpE$rAtAlmslHynmnAlktA}b . and took part in the march dozens of militants from the Brigades CC1 VC2 PC3 NC4 PC5 Pct6 0.7 Pct6 CC1 PC3 VC2 NC4 PC5 NC4 Pct6 PC5 CC1 PC3 VC2 0.1 PC3 NC4 PC5 VC2 CC1 Pct6 0.1 CC1 NC4 VC2 PC3 PC5 Pct6 0.4 0.9 CC1 NC4 PC5 VC2 PC3 Pct6 23 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
Chunk-based fuzzy reordering rules Reordered source LM Reordering selection w- $ArkfyAltZAhrpE$rAtAlmslHynmnAlktA}b . and took part in the march dozens of militants from the Brigades CC1 VC2 PC3 NC4 PC5 Pct6 0.7 Pct6 CC1 PC3 VC2 NC4 PC5 0.1 0.1 Reorderings to encode in the distortion matrix 0.4 0.9 CC1 NC4 PC5 VC2 PC3 Pct6 24 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
Modifying the distortion matrix Pct6 CC1 PC3 VC2 NC4 PC5 Reorderings to encode in the distortion matrix CC1 NC4 PC5 VC2 PC3 Pct6 25 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
Modifying the distortion matrix Pct6 CC1 PC3 VC2 NC4 PC5 Reorderings to encode in the distortion matrix CC1 NC4 PC5 VC2 PC3 Pct6 26 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
Modifying the distortion matrix Pct6 CC1 PC3 VC2 NC4 PC5 Reorderings to encode in the distortion matrix CC1 NC4 PC5 VC2 PC3 Pct6 27 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
Modifying the distortion matrix Pct6 CC1 PC3 VC2 NC4 PC5 Reorderings to encode in the distortion matrix CC1 NC4 PC5 VC2 PC3 Pct6 28 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
Modifying the distortion matrix Pct6 CC1 PC3 VC2 NC4 PC5 Reorderings to encode in the distortion matrix CC1 NC4 PC5 VC2 PC3 Pct6 29 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
Modifying the distortion matrix Pct6 CC1 PC3 VC2 NC4 PC5 Reorderings to encode in the distortion matrix CC1 NC4 PC5 VC2 PC3 Pct6 30 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
Modifying the distortion matrix Pct6 CC1 PC3 VC2 NC4 PC5 Reorderings to encode in the distortion matrix CC1 NC4 PC5 VC2 PC3 Pct6 31 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
Modifying the distortion matrix Pct6 CC1 PC3 VC2 NC4 PC5 Reorderings to encode in the distortion matrix CC1 NC4 PC5 VC2 PC3 Pct6 32 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
Experiments • Tasks: NIST-MT09 for Ar-En, WMT10 for De-En • Systems based on Moses, include state-of-the-art hierarchical lexicalized reordering models [Tillmann 04; Koehn & al 05; Galley & Manning 08] • Baseline Distortion Limits: 5 in Ar-En, 10 in De-En • Evaluation by: • - BLEU for lexical match & local order • - KRS for global order 33 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
Arabic-English: +0.9 BLEU +0.6 KRS (signif.) Translation Quality Translation Time Test set: eval09-nw Distortion modified with 3-best reorderings per rule-matching sequence 35 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
German-English: +0.4 BLEU +0.7 KRS (signif.) Translation Quality Translation Time Test set: newstest10 Distortion modified with 3-best reorderings per rule-matching sequence 37 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
Conclusions • Modified distortion allows for finer & linguistically motivated definition of search space • We achieve better translation & faster decoding in language pairs where long reordering concentrates on few patterns • Our method is complementary to reordering modeling • For now, few reordering rules are needed to modify distortion • We are currently working on a fully data-driven approach to replace the rules 38 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
39 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT
40 A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT A. Bisazza & M. Federico – Modified Distortion Matrices for PSMT