1 / 34

Faster is not always better, but what if it’s not worse?

Faster is not always better, but what if it’s not worse?. Joke Daems & Lieve Macken http://lt3.hogent.be/en/projects/robot/. Main questions. Is MT+PE more productive than HT? Is there a difference in quality between MT+PE and HT?. Overview. Experiment set-up Analysis Productivity

kiara
Download Presentation

Faster is not always better, but what if it’s not worse?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Faster is not always better,but what if it’s not worse? Joke Daems & Lieve Macken http://lt3.hogent.be/en/projects/robot/

  2. Main questions • Is MT+PE more productive than HT? • Is there a difference in quality between MT+PE and HT? Joke Daems: ROBOT

  3. Overview • Experiment set-up • Analysis • Productivity • TQA approach • Results • Productivity • Quality • Conclusion & Future work Joke Daems: ROBOT

  4. Overview • Experiment set-up • Analysis • Productivity • TQA approach • Results • Productivity • Quality • Conclusion & Futurework Joke Daems: ROBOT

  5. Experiment set-up • Participants: Master’s students of translation • 17% never used MT • 61% sometimes used MT • Task: HT & MT+PE (Google Translate) EN->NL • Texts: 4 newspaper articles ± 270 words • Tools: PET & external resources Joke Daems: ROBOT

  6. The PET-tool Joke Daems: ROBOT

  7. Overview • Experiment set-up • Analysis • Productivity • TQA approach • Results • Productivity • Quality • Conclusion & Futurework Joke Daems: ROBOT

  8. Analysis: TQA-approach Joke Daems: ROBOT

  9. TQA-approach: Acceptability Joke Daems: ROBOT

  10. TQA-approach: Adequacy Joke Daems: ROBOT

  11. TQA: Annotation (brat-tool) 1) Acceptability 2) Adequacy Joke Daems: ROBOT

  12. Inter-annotator agreement • 2 annotators • Assessing quality = highly subjective • Acceptability: agreement = 38%; κ = 0.31 • Adequacy: agreement = 41%; κ = 0.30 Joke Daems: ROBOT

  13. Inter-annotator agreement • Correlation between annotators • Acceptability: r = 0.70 • Adequacy: r = 0.89 • Agreement on categories • Acceptability: agreement = 89%; κ = 0.88 • Adequacy: agreement = 89%; κ = 0.87 • Increase after consolidation phase • Acceptability: agreement = 69%; κ = 0.67 • Adequacy: agreement = 82%; κ = 0.79 Joke Daems: ROBOT

  14. Overview • Experiment set-up • Analysis • Productivity • TQA approach • Results • Productivity • Quality • Conclusion & Futurework Joke Daems: ROBOT

  15. Results: Productivity Joke Daems: ROBOT

  16. Overview • Experiment set-up • Analysis • Productivity • TQA approach • Results • Productivity • Quality • Conclusion & Futurework Joke Daems: ROBOT

  17. Results: Quality Joke Daems: ROBOT

  18. Results: Quality Joke Daems: ROBOT

  19. Results: Quality Joke Daems: ROBOT

  20. Results: Quality Joke Daems: ROBOT

  21. Results: Quality Proportion of total per subcategory: focus on acceptability Joke Daems: ROBOT

  22. Overview • Experiment set-up • Analysis • Productivity • TQA approach • Results • Productivity • Quality • Conclusion & Futurework Joke Daems: ROBOT

  23. Conclusions • 2-step TQA approach works • Quality is text-dependent • PE is faster than HT • PE quality often higher than HT quality • Different types of errors between PE and HT Joke Daems: ROBOT

  24. Future work • Link translation problems to source text and MT • Better understanding of PE • Better understanding of translatability • Train post-editors • Repeat experiment with professional translators Joke Daems: ROBOT

  25. Thank you for listening For more information, contact: joke.daems@ugent.be Suggestions? Questions? Joke Daems: ROBOT

  26. Extra information

  27. Extra information Readability scores per text 1722177117811802 (288 words) (274 words) (260 words) (263 words)

  28. Extra information

  29. Extra information

  30. Extra information

  31. Extra information

  32. Extra information

  33. Extra information Proportion of total per subcategory: focus on adequacy

More Related