Translation Assistance by Translation of L1 Fragments in an L2 Context
van Gompel, Maarten and van den Bosch, Antal

Article Structure

Abstract

In this paper we present new research in translation assistance.

Introduction

Whereas machine translation generally concerns the translation of whole sentences or texts from one language to the other, this study focusses on the translation of native language (henceforth L1) words and phrases, i.e.

Data preparation

Preparing the data to build training and test data for our intended translation assistance system is not trivial, as the type of interactive translation assistant we aim to develop does not exist yet.

System

We develop a classifier-based system composed of so-called “classifier experts”.

Evaluation

Several automated metrics exist for the evaluation of L2 system output against the L2 reference out-

Baselines

A context-insensitive yet informed baseline was constructed to assess the impact of L2 context information in translating Ll fragments.

Experiments & Results

The data for our experiments were drawn from the Europarl parallel corpus (Koehn, 2005) from which we extracted two sets of 200, 000 sentence pairs each for several language pairs.

Discussion and conclusion

In this study we have shown the feasibility of a classifier-based translation assistance system in which L1 fragments are translated in an L2 context, in which the classifier experts are built individually per word or phrase.

Topics

language model

Appears in 25 sentences as: Language Model (3) language model (20) language modelling (2) language models (2)
In Translation Assistance by Translation of L1 Fragments in an L2 Context
  1. We study the feasibility of exploiting cross-lingual context to obtain high-quality translation suggestions that improve over statistical language modelling and word-sense disambiguation baselines.
    Page 1, “Abstract”
  2. The main research question in this research is how to disambiguate an L1 word or phrase to its L2 translation based on an L2 context, and whether such cross-lingual contextual approaches provide added value compared to baseline models that are not context informed or compared to standard language models .
    Page 2, “Introduction”
  3. 3.1 Language Model
    Page 4, “System”
  4. We also implement a statistical language model as an optional component of our classifier-based system and also as a baseline to compare our system to.
    Page 4, “System”
  5. The language model is a trigram-based back-off language model with Kneser-Ney smoothing, computed using SRILM (Stolcke, 2002) and trained on the same training data as the translation model.
    Page 4, “System”
  6. For any given hypothesis H, results from the L1 to L2 classifier are combined with results from the L2 language model .
    Page 4, “System”
  7. We do so by normalising the class probability from the classifier (scoreT(H)), which is our translation model, and the language model (scorelm(H)), in such a way that the highest classifier score for the alternatives under consideration is always 1.0, and the highest language model score of the sentence is always 1.0.
    Page 4, “System”
  8. If desired, the search can be parametrised with variables A3 and A4, representing the weights we want to attach to the classifier-based translation model and the language model , respectively.
    Page 4, “System”
  9. In the current study we simply left both weights set to one, thereby assigning equal importance to translation model and language model .
    Page 4, “System”
  10. A second baseline was constructed by weighing the probabilities from the translation table directly with the L2 language model described earlier.
    Page 4, “Baselines”
  11. target language modelling ) which is also cus-
    Page 4, “Baselines”

See all papers in Proc. ACL 2014 that mention language model.

See all papers in Proc. ACL that mention language model.

Back to top.

LM

Appears in 11 sentences as: +LM (2) LM (9)
In Translation Assistance by Translation of L1 Fragments in an L2 Context
  1. It adds a LM component to the MLF baseline.
    Page 4, “Baselines”
  2. This LM baseline allows the comparison of classification through L1 fragments in an L2 context, with a more traditional L2 context modelling (i.e.
    Page 4, “Baselines”
  3. As expected, the LM baseline substantially outperforms the context-insensitive MLF baseline.
    Page 5, “Experiments & Results”
  4. Second, our classifier approach attains a substantially higher accuracy than the LM baseline.
    Page 5, “Experiments & Results”
  5. The same significance level was found when comparing llrl+LM against llrl, auto+LM against aut o, as well as the LM baseline against the MLF baseline.
    Page 6, “Experiments & Results”
  6. This result is in line with the positive effect of adding the LM to the l 1 r1.
    Page 8, “Experiments & Results”
  7. However, for English—>Dutch and English—>Chinese we find that the LM baseline actually performs slightly worse than baseline.
    Page 8, “Experiments & Results”
  8. LM baseline llrl
    Page 9, “Discussion and conclusion”
  9. llrl +LM auto
    Page 9, “Discussion and conclusion”
  10. LM baseline 11r1
    Page 9, “Discussion and conclusion”
  11. 11r1 +LM auto auto—I—LM
    Page 9, “Discussion and conclusion”

See all papers in Proc. ACL 2014 that mention LM.

See all papers in Proc. ACL that mention LM.

Back to top.

sentence pairs

Appears in 9 sentences as: sentence pair (4) sentence pairs (7)
In Translation Assistance by Translation of L1 Fragments in an L2 Context
  1. 2. for each aligned sentence pair (sentences E SS, sentencet E St) in the parallel corpus split (88,875):
    Page 2, “Data preparation”
  2. The output of the algorithm in Figure 1 is a modified set of sentence pairs (sentence; sentencet), in which the same sentence pair may be used multiple times with different Ll substitutions for different fragments.
    Page 3, “Data preparation”
  3. If output 0 is a subset of reference 7“ then a score of % is assigned for that sentence pair .
    Page 4, “Evaluation”
  4. The word accuracy for the entire set is then computed by taking the sum of the word accuracies per sentence pair, divided by the total number of sentence pairs .
    Page 4, “Evaluation”
  5. The data for our experiments were drawn from the Europarl parallel corpus (Koehn, 2005) from which we extracted two sets of 200, 000 sentence pairs each for several language pairs.
    Page 5, “Experiments & Results”
  6. The final test sets are a randomly sampled 5, 000 sentence pairs from the 200, 000-sentence test split for each language pair.
    Page 5, “Experiments & Results”
  7. English fallback in a Spanish context, consists of 5, 608, 015 sentence pairs .
    Page 5, “Experiments & Results”
  8. This number is much larger than the 200, 000 we mentioned before because single sentence pairs may be reused multiple times with different marked fragments.
    Page 5, “Experiments & Results”
  9. From this training set of sentence pairs over 100, 000 classifier experts are derived.
    Page 5, “Experiments & Results”

See all papers in Proc. ACL 2014 that mention sentence pairs.

See all papers in Proc. ACL that mention sentence pairs.

Back to top.

parallel corpus

Appears in 7 sentences as: parallel corpus (7)
In Translation Assistance by Translation of L1 Fragments in an L2 Context
  1. We start with a parallel corpus that is tokenised for both L1 and L2.
    Page 2, “Data preparation”
  2. The parallel corpus is randomly sampled into two large and equally-sized parts.
    Page 2, “Data preparation”
  3. 1. using phrase-translation table T and parallel corpus split 8
    Page 2, “Data preparation”
  4. 2. for each aligned sentence pair (sentences E SS, sentencet E St) in the parallel corpus split (88,875):
    Page 2, “Data preparation”
  5. Figure 1: Algorithm for extracting training and test data on the basis of a phrase-translation table (T) and subset/split from a parallel corpus (S).
    Page 2, “Data preparation”
  6. The fact that a phrase-translation table needs to be constructed for the test data is also the reason that the parallel corpus split from which the test data is derived has to be large enough, ensuring better quality.
    Page 3, “Data preparation”
  7. The data for our experiments were drawn from the Europarl parallel corpus (Koehn, 2005) from which we extracted two sets of 200, 000 sentence pairs each for several language pairs.
    Page 5, “Experiments & Results”

See all papers in Proc. ACL 2014 that mention parallel corpus.

See all papers in Proc. ACL that mention parallel corpus.

Back to top.

BLEU

Appears in 5 sentences as: BLEU (6)
In Translation Assistance by Translation of L1 Fragments in an L2 Context
  1. We report on BLEU , NIST, METEOR, and word error rate metrics WER and PER.
    Page 4, “Evaluation”
  2. The BLEU scores, not included in the figure but shown in Table 2, show a similar trend.
    Page 5, “Experiments & Results”
  3. Statistical significance on the BLEU scores was tested using pairwise bootstrap sampling (Koehn, 2004).
    Page 6, “Experiments & Results”
  4. Another discrepancy is found in the BLEU scores of the English—>Chinese experiments, where we measure an unexpected drop in BLEU score under baseline.
    Page 8, “Experiments & Results”
  5. We therefore attach low importance to this deviation in BLEU here.
    Page 8, “Experiments & Results”

See all papers in Proc. ACL 2014 that mention BLEU.

See all papers in Proc. ACL that mention BLEU.

Back to top.

cross-lingual

Appears in 5 sentences as: cross-lingual (5)
In Translation Assistance by Translation of L1 Fragments in an L2 Context
  1. We study the feasibility of exploiting cross-lingual context to obtain high-quality translation suggestions that improve over statistical language modelling and word-sense disambiguation baselines.
    Page 1, “Abstract”
  2. The cross-lingual context in our research question may at first seem artificial, but its design explicitly aims at applications related to computer-aided language learning (Laghos and Panayiotis, 2005; Levy, 1997) and computer-aided translation (Barrachina et al., 2009).
    Page 1, “Introduction”
  3. The main research question in this research is how to disambiguate an L1 word or phrase to its L2 translation based on an L2 context, and whether such cross-lingual contextual approaches provide added value compared to baseline models that are not context informed or compared to standard language models.
    Page 2, “Introduction”
  4. The choice for this algorithm is motivated by the fact that it handles multiple classes with ease, but first and foremost because it has been successfully employed for word sense disambiguation in other studies (Hoste et al., 2002; Decadt et al., 2004), in particular in cross-lingual word sense disambiguation, a task closely resembling our current task (van Gompel and van den Bosch, 2013).
    Page 3, “System”
  5. The latter study on cross-lingual WSD finds a positive impact when conducting feature selection per classifier.
    Page 6, “Experiments & Results”

See all papers in Proc. ACL 2014 that mention cross-lingual.

See all papers in Proc. ACL that mention cross-lingual.

Back to top.

feature vector

Appears in 5 sentences as: feature vector (5)
In Translation Assistance by Translation of L1 Fragments in an L2 Context
  1. The feature vector for the classifiers represents a local context of neighbouring words, and optionally also global context keywords in a binary-valued bag-of-words configuration.
    Page 3, “System”
  2. If not, we check for the presence of a classifier expert for the offered L1 fragment; only then we can proceed by extracting the desired number of L2 local context words to the immediate left and right of this fragment and adding those to the feature vector .
    Page 3, “System”
  3. For the classifier-based system, we tested various different feature vector configurations.
    Page 5, “Experiments & Results”
  4. The various erY configurations use the same feature vector setup for all classifier experts.
    Page 5, “Experiments & Results”
  5. The auto configuration does not uniformly apply the same feature vector setup to all classifier experts but instead seeks to find the optimal setup per classifier expert.
    Page 5, “Experiments & Results”

See all papers in Proc. ACL 2014 that mention feature vector.

See all papers in Proc. ACL that mention feature vector.

Back to top.

language pair

Appears in 5 sentences as: language pair (3) language pairs (2)
In Translation Assistance by Translation of L1 Fragments in an L2 Context
  1. The data for our experiments were drawn from the Europarl parallel corpus (Koehn, 2005) from which we extracted two sets of 200, 000 sentence pairs each for several language pairs .
    Page 5, “Experiments & Results”
  2. The final test sets are a randomly sampled 5, 000 sentence pairs from the 200, 000-sentence test split for each language pair .
    Page 5, “Experiments & Results”
  3. Let us first zoom in to convey a sense of scale on a specific language pair .
    Page 5, “Experiments & Results”
  4. In order to draw accurate conclusions, experiments on a single data set and language pair are not sufficient.
    Page 8, “Experiments & Results”
  5. We therefore conducted a number of experiments with other language pairs , and present the abridged results in Table 6.
    Page 8, “Experiments & Results”

See all papers in Proc. ACL 2014 that mention language pair.

See all papers in Proc. ACL that mention language pair.

Back to top.

Machine Translation

Appears in 5 sentences as: Machine Translation (3) machine translation (2)
In Translation Assistance by Translation of L1 Fragments in an L2 Context
  1. Whereas machine translation generally concerns the translation of whole sentences or texts from one language to the other, this study focusses on the translation of native language (henceforth L1) words and phrases, i.e.
    Page 1, “Introduction”
  2. the role of the translation model in Statistical Machine Translation (SMT).
    Page 1, “Introduction”
  3. This is done using the scripts provided by the Statistical Machine Translation system Moses (Koehn et al., 2007).
    Page 2, “Data preparation”
  4. It has also been used in machine translation studies in which local source context is used to classify source phrases into target phrases, rather than looking them up in a phrase table (Stroppa et al., 2007; Haque et al., 2011).
    Page 3, “System”
  5. In addition to these, the system’s output can be compared against the L2 reference translation(s) using established Machine Translation evaluation metrics.
    Page 4, “Evaluation”

See all papers in Proc. ACL 2014 that mention Machine Translation.

See all papers in Proc. ACL that mention Machine Translation.

Back to top.

translation model

Appears in 5 sentences as: translation model (5)
In Translation Assistance by Translation of L1 Fragments in an L2 Context
  1. the role of the translation model in Statistical Machine Translation (SMT).
    Page 1, “Introduction”
  2. The language model is a trigram-based back-off language model with Kneser-Ney smoothing, computed using SRILM (Stolcke, 2002) and trained on the same training data as the translation model .
    Page 4, “System”
  3. We do so by normalising the class probability from the classifier (scoreT(H)), which is our translation model , and the language model (scorelm(H)), in such a way that the highest classifier score for the alternatives under consideration is always 1.0, and the highest language model score of the sentence is always 1.0.
    Page 4, “System”
  4. If desired, the search can be parametrised with variables A3 and A4, representing the weights we want to attach to the classifier-based translation model and the language model, respectively.
    Page 4, “System”
  5. In the current study we simply left both weights set to one, thereby assigning equal importance to translation model and language model.
    Page 4, “System”

See all papers in Proc. ACL 2014 that mention translation model.

See all papers in Proc. ACL that mention translation model.

Back to top.

context information

Appears in 4 sentences as: context information (3) context informed (1)
In Translation Assistance by Translation of L1 Fragments in an L2 Context
  1. The main research question in this research is how to disambiguate an L1 word or phrase to its L2 translation based on an L2 context, and whether such cross-lingual contextual approaches provide added value compared to baseline models that are not context informed or compared to standard language models.
    Page 2, “Introduction”
  2. Nevertheless, we hope to show that our automated way of test set generation is sufficient to test the feasibility of our core hypothesis that L1 fragments can be translated to L2 using L2 context information .
    Page 3, “Data preparation”
  3. If so, we are done quickly and need not rely on context information .
    Page 3, “System”
  4. A context-insensitive yet informed baseline was constructed to assess the impact of L2 context information in translating Ll fragments.
    Page 4, “Baselines”

See all papers in Proc. ACL 2014 that mention context information.

See all papers in Proc. ACL that mention context information.

Back to top.

best results

Appears in 3 sentences as: best results (3)
In Translation Assistance by Translation of L1 Fragments in an L2 Context
  1. Here we observe that a context width of one yields the best results .
    Page 5, “Experiments & Results”
  2. This combination of a classifier with context size one and trigram-based language model proves to be most effective and reaches the best results so far.
    Page 6, “Experiments & Results”
  3. Automated configuration selection had positive results, yet the system with context size one and an L2 language model component often produces the best results .
    Page 8, “Discussion and conclusion”

See all papers in Proc. ACL 2014 that mention best results.

See all papers in Proc. ACL that mention best results.

Back to top.

BLEU scores

Appears in 3 sentences as: BLEU score (1) BLEU scores (3)
In Translation Assistance by Translation of L1 Fragments in an L2 Context
  1. The BLEU scores , not included in the figure but shown in Table 2, show a similar trend.
    Page 5, “Experiments & Results”
  2. Statistical significance on the BLEU scores was tested using pairwise bootstrap sampling (Koehn, 2004).
    Page 6, “Experiments & Results”
  3. Another discrepancy is found in the BLEU scores of the English—>Chinese experiments, where we measure an unexpected drop in BLEU score under baseline.
    Page 8, “Experiments & Results”

See all papers in Proc. ACL 2014 that mention BLEU scores.

See all papers in Proc. ACL that mention BLEU scores.

Back to top.

MT system

Appears in 3 sentences as: MT system (2) MT systems (1)
In Translation Assistance by Translation of L1 Fragments in an L2 Context
  1. These scores should generally be much better than the typical MT system performances as only local changes are made to otherwise “perfect” L2 sentences.
    Page 4, “Evaluation”
  2. An application of our idea outside the area of translation assistance is post-correction of the output of some MT systems that, as a last-resort heuristic, copy source words or phrases into their output, producing precisely the kind of input our system is trained on.
    Page 9, “Discussion and conclusion”
  3. Our classification-based approach may be able to resolve some of these cases operating as an add-on to a regular MT system —or as a independent post-correction system.
    Page 9, “Discussion and conclusion”

See all papers in Proc. ACL 2014 that mention MT system.

See all papers in Proc. ACL that mention MT system.

Back to top.

native language

Appears in 3 sentences as: native language (3)
In Translation Assistance by Translation of L1 Fragments in an L2 Context
  1. We describe a system capable of translating native language (L1) fragments to foreign language (L2) fragments in an L2 context.
    Page 1, “Abstract”
  2. The type of translation assistance system under investigation here encourages language learners to write in their target language while allowing them to fall back to their native language in case the correct word or expression is not known.
    Page 1, “Abstract”
  3. Whereas machine translation generally concerns the translation of whole sentences or texts from one language to the other, this study focusses on the translation of native language (henceforth L1) words and phrases, i.e.
    Page 1, “Introduction”

See all papers in Proc. ACL 2014 that mention native language.

See all papers in Proc. ACL that mention native language.

Back to top.

randomly sampled

Appears in 3 sentences as: randomly sampled (2) randomly sampling (1)
In Translation Assistance by Translation of L1 Fragments in an L2 Context
  1. The parallel corpus is randomly sampled into two large and equally-sized parts.
    Page 2, “Data preparation”
  2. The final test set is created by randomly sampling the desired number of test instances.
    Page 3, “Data preparation”
  3. The final test sets are a randomly sampled 5, 000 sentence pairs from the 200, 000-sentence test split for each language pair.
    Page 5, “Experiments & Results”

See all papers in Proc. ACL 2014 that mention randomly sampled.

See all papers in Proc. ACL that mention randomly sampled.

Back to top.