1 / 17

Applying Machine Translation Metrics to Student-Written Translations

Applying Machine Translation Metrics to Student-Written Translations. Lisa N. Michaud Computer Science Department Merrimack College North Andover, Massachusetts, USA Patricia Ann McCoy Language Department Universidad de las Americas Puebla Puebla , Mexico. Criteria for judging translations.

triage
Download Presentation

Applying Machine Translation Metrics to Student-Written Translations

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Applying Machine Translation Metrics to Student-Written Translations Lisa N. MichaudComputer Science DepartmentMerrimack CollegeNorth Andover, Massachusetts, USA Patricia Ann McCoyLanguage DepartmentUniversidad de las AmericasPueblaPuebla, Mexico

  2. Criteria for judging translations fluency(is it well-formed?) fidelity (does it convey original meaning?) (Hovy et al., 2002) Michaud and McCoy

  3. Multiplicity of translations • In each one of these jobs the professor could have agreed to work 6 hours a day and therefore would not be surpassing the working day hour limit. • In each one of these jobs the teacher could have agreed to work 6 hours per day and therefore he wouldn't be bound by the limits of the working day. • In each of these examples the teaching could have been arranged so that he/she works six hours a day and would not be affected by any workday limitations. • In both of these jobs the professor could have agreed to work six hours daily and therefore he wouldn't be affecting his work shift limit. Michaud and McCoy

  4. Multiplicity of translations • In each one of these jobs the professor could have agreed to work 6 hours a day and therefore would not be surpassing the working day hour limit. • In each one of these jobs the teacher could have agreed to work 6 hours per day and therefore he wouldn't be bound by the limits of the working day. • In each of these examples the teaching could have been arranged so that he/she works six hours a day and would not be affected by any workday limitations. • In both of these jobs the professor could have agreed to work six hours daily and therefore he wouldn't be affecting his work shift limit. Michaud and McCoy

  5. BLEU Hypothesis Multiple References Michaud and McCoy

  6. TERp Hypothesis INSERTION PHRASALEQUIVALENCE or SYNONYM SAMESTEM SUBSTITUTION SHIFT Single Reference Michaud and McCoy

  7. TERp alignment and tags Michaud and McCoy

  8. Student translation corpus Michaud and McCoy

  9. Does TERp agree with an expert? Instructor Scores vs Inverted TERp 650 sentences (22%) Pearson Correlation r = 0.232236 Michaud and McCoy

  10. Score distribution Michaud and McCoy

  11. Instructor rubric (original) 10 Excellent 9 Good 8Satisfactory 0-7Deficient Michaud and McCoy

  12. Evaluating TERp tags (pilot) Michaud and McCoy

  13. Future work Michaud and McCoy

  14. Instructor rubric (revised) 100 Excellent 90 Good 80 Satisfactory 0-70 Deficient Michaud and McCoy

  15. Modifying the TERp Score Hypothesis INSERTION PHRASALEQUIVALENCE or SYNONYM SAMESTEM SUBSTITUTION SHIFT Single Reference Michaud and McCoy

  16. Recognizing false cognates Hypothesis cynical Source cínico SUBSTITUTION brazen Single Reference Michaud and McCoy

  17. Extracting mistranslation pairs SPANISHDICTIONARY ENGLISHDICTIONARY cynical cynicalbrazen cínico zona zone Michaud and McCoy

More Related