Article Structure

Abstract

The automatic estimation of machine translation (MT) output quality is a hard task in which the selection of the appropriate algorithm and the most predictive features over reasonably sized training sets plays a crucial role.

Introduction

After two decades of steady progress, research in statistical machine translation (SMT) started to cross its path with translation industry with tangible mutual benefit.

Related work

QE is generally cast as a supervised machine learning task, where a model trained from a collection of (source, target, label) instances is used to predict labels1 for new, unseen test items (Specia et al., 2010).

Online QE for CAT environments

When operating with advanced CAT tools, translators are presented with suggestions (either matching fragments from a translation memory or automatic translations produced by an MT system) for each sentence of a source document.

Evaluation framework

To measure the adaptation capability of different QE models, we experiment with a range of conditions defined by variable degrees of similarity between training and test data.

Experiments with WMT12 data

The motivations for experiments with training and test data featuring homogeneous label distributions are twofold.

Experiments with CAT data

To experiment with adaptive QB in more realistic conditions we used a CAT tool12 to collect two datasets of (source, target, posLedited target) English-Italian tuples.The source sentences in the datasets come from two documents from different domains, respectively legal (L) and information technology (IT).

Conclusion

In the CAT scenario, each translation job can be seen as a complex situation where the user (his personal style and background), the source document (the language and the domain) and the underlying technology (the translation memory and the MT engine that generate translation suggestions) contribute to make the task unique.