1 / 88

Translation Quality Measurement

Translation Quality Measurement. By Riccardo Schiaffino and Franco Zearo. Biographical Notes on the Authors. Riccardo Schiaffino

Albert_Lan
Download Presentation

Translation Quality Measurement

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Translation Quality Measurement By Riccardo Schiaffino and Franco Zearo

  2. Biographical Notes on the Authors Riccardo Schiaffino Riccardo Schiaffino worked as translator, translation manager and special software translation project lead for a major software company. As a translation manager, Riccardo worked on the improvement of translation quality and on translation quality metrics and tools. He holds an MA degree in Translation, and has been working in translations for over 18 years, first in Italy and then in the U.S. Riccardo is ATA accredited. Contact: riccardo_schiaffino@aliquantum.biz. Franco Pietro Zearo Franco Pietro Zearo is a project manager with Lionbridge Technologies in Boulder, Colorado. He holds a degree in translation from the Advanced School of Modern Languages for Translators and Interpreters at the University of Trieste, Italy, and earned an MBA from the University of Phoenix. Before joining Lionbridge in 1996, he worked as a freelance technical translator in Italian, English, and Russian. At Lionbridge, he has held positions in translation, localization analysis, presales, and cultural and globalization consulting. He has been responsible for translation quality on numerous projects for many Fortune 500 clients. In his previous role as senior technical translator, he helped define best practices for the translation department. Contact: franco_zearo@lionbridge.com.

  3. Overview • Technical translation and quality • Translation quality initiatives • Quality Control vs. Quality Assurance • Our proposal for quality assurance • Checklists • Sampling techniques • Conclusions • Importance of cost/benefit factors

  4. Overview • Measuring Quality • Translation Quality Assessment • Quality Assurance Forms • Error Categories • Sampling • Translation Quality Index • Questions and Answers

  5. Overview • Why Is Quality Measurement Important? • How to Set Up a Quality Measurement System • Demo of a Translation Quality Measurement Tool Prototype • Practical Recommendations • Questions & Answers

  6. Our Definition of Quality • Functional approach to quality • Different views of translation lead to:ð Different concepts of quality ð Different assessments • Quality is defined as meeting the needs and expectations of the customer or user.

  7. Our Definition of Quality • Functional approach to quality • Quality is defined as consistently meeting the needs and expectations of the customer or user

  8. Correct Translation A correct translation is a translation with no errors or where total error points result in a Translation Quality Index above the desired threshold

  9. Customer-driven Considerations • Conformance to specifications Customer’s vs. One’s own • Fitness for use How well the translation performs its intended purpose • Value ( = quality & price) How well the translation performs its intended purposeat a price customers are willing to pay • Support E.g.: Printing, testing • Psychological impressions E.g.: In-country translators; certification

  10. Customer-driven Considerations

  11. Importance of Quality • Quality as a Competitive Weapon • Good Quality ð Higher ProfitsGood quality of translation(product) and service (process)can pay off in higher profits • Improving on quality can reduce costs andspeed up time-to-market

  12. Why is Quality Measurement Important? • You can’t manage what you can’t measure • It is difficult to improve something if you cannot measure it. • Such measurement should be repeatable and objective. • Different persons should arrive at similar assessment for the same piece of translation.

  13. Why is Quality Measurement Important? • It is difficult to improve something if you cannot measure it. • Such measurement should be repeatable and objective. • Different evaluators should arrive at similar assessment for the same piece of translation.

  14. Why is Quality Measurement Important? It is difficult to improve something if you cannot measure it Metrics provide: • A way to objectively quantify a process • A means to reduce the cost of poor quality • A means to increase customer satisfaction • An opportunity for benchmarking • Competitive advantages

  15. “You cannot measure quality” • This is not true: • There are certain components of translation quality that will always remain subjective. • However, • There are other elements that can be objectively measured. • By concentrating of these, we believe we can achieve a satisfactory measurement of translation quality.

  16. Who Benefits from Reliable Translation Quality Measurement? • Professional Translators • Translation Companies andIn-House Translation Departments • Translation Customers and Users

  17. Why Do We Make Errors? • The reasons behind the errors are separate from the measurement of the errors: Studying why errors happen is important, but it pertains more to quality control and improvement than to quality assurance • E.g., capitalization errors due to the "Autocorrect" (mis)feature of MS Word (e.g., HBsAg "corrected" to HbsAg)

  18. QC vs QA • Quality Control (QC) • Quality verification over the whole text. Example: editing. • Quality Assurance (QA) • Sampling techniques, control of quality over a (statistically significant) sample of the whole text. Example: quality measurement.

  19. QC vs QA • Quality Control (QC) • Quality verification over the whole text. Example: Editing. • Quality Assurance (QA) • Sampling techniques, control of quality over a (statistically significant) sample of the whole text. Appropriate use: Quality measurement.

  20. Translation Quality Factors

  21. Inspection Points Key Principle: Reject “defective material” at its lowest value Proof Edit SLContent Development(GIGO) $ Value of Service Translation Stages of Production

  22. Inspection Points Key Principle: Reject “defective material” at its lowest value Proof Edit SLContent Development(GIGO) $ Value of Service Translation Stages of Production

  23. Cost/Benefit Analysis • Quality measurements are a tool to determine the optimal level ofquality.They could help us identify a cut-off point.

  24. Ideas from other disciplines • Software project management techniques • W. Edwards Deming and other quality assurance experts

  25. When we study translation quality, we can focus on different things: The translated text(the “product”) The translator The translation process(the “process”)

  26. Product & Process Assessment Translation quality assessment must apply to both: • The translated text(the “product”) • The translation process(the “process”)

  27. Product & Process Assessment Translation quality assessment must apply to both: • The translated text(the “product”) • The translation process(the “process”)

  28. Translation Quality Initiatives The translated text ATA and other translators’ certification initiatives The translator The translation process DIN 2345ISO 900xUNI EN 10754 EUATCASTM SAE J2450 LISA QA

  29. Translation Quality Initiatives • ISO 9002 • EUATC Quality Standard • DIN 2345 • ASTM Standard for Language Translation • SAE J2450 • LISA QA Model • Academic translation theories and studies • Private sector methodologies

  30. Quality Measurement: Our Proposal • What Can Other Disciplines Teach Us? • Use checklists to collect the data • Identify types of errors, issues or problems • Determine relative importance of issues (may be different for different languages; e.g., spelling errors in English, French or Italian) • Use sampling techniques to assess your quality level • Determine percent thresholds for various levels of quality • Determine whether you have achieved your target quality or not

  31. Criteria for Successful Quality Measurements Translation quality measurements should be: • Repeatable (two assessments of the same sample yield similar results) • Reproducible (different evaluators should arrive at a similar assessment for the same piece of translation • Objective (void of subjective bias)

  32. Classification of Errors

  33. Measurement through Circumstantial Evidence • Errors are circumstantial evidence of quality • We believe that precise error measurement provides sufficient indication of good and bad translations • A good translation is a translation with very few errors or none at all

  34. Definition of Errors Deal with errors only when they violate agreed upon protocols of engagement whether implicit or explicit Examples of explicit and implicit criteria: • Non-compliance errors (e.g. not following instructions) • Violations of generally accepted language conventions

  35. Summary: Error Categorization • Select a (small) set of categories • CTQ: Critical-To-Quality categories • Provide clear definitions • Set tolerance limits • Min / Max # of errors per X words • Assign a weight • Critical, Major, Minor

  36. Summary: Error Categorization • Select a (small) set of categories • CTQ: Critical-To-Quality categories • Provide clear definitions • Assign a weight • Critical, Major, Minor

  37. Real Life Examples • Development of translation quality measurement at J.D. Edwards • Use of sampling techniques for quality assurance at Lionbridge

  38. The J.D. Edwards’ QA FormLanguage Customization • Weighting the major categories

  39. The J.D. Edwards’ QA FormLanguage Customization • Weighting the items within the major categories

  40. The J.D. Edwards’ QA FormLanguage Customization • Weighting the items within the major categories (detail)

  41. How We Worked to Develop Our Spreadsheet • Determine type of errors, issues or problems • Determine relative importance of issues (may be different for different languages; e.g., spelling errors in English, French or Italian) • Determine which are the responsibility of translation • Determine tolerance limits for various levels of quality

  42. Translation Quality Measurement Tool • The Translation Quality Measurement tool helps to measure process quality • It is NOT an editing tool, but it serves to measure whether a process is effective

  43. Use of the Tool • Use the tool to measure the effectiveness of quality control process • Analyze the results obtained through the tool (control charts) • If the process is NOT in statistical control • Discover special causes and deal with them appropriately • Remove them if they are negative • Incorporate them in process if they are positive • Improve the process when it is in statistical control

  44. A TQI Tool Prototype

  45. ATA Implementation

  46. ATA Implementation

  47. SAE Implementation (Modified)

  48. SAE Implementation (Modified)

  49. TQI Log

  50. Use of Checklists • There are several quality assessment methodologies that rely on the use of checklists – among these the LISA methodology.

More Related