Cut the noise: Mutually reinforcing reordering and alignments for improved machine translation
Visweswariah, Karthik and Khapra, Mitesh M. and Ramanathan, Ananthakrishnan

Article Structure

Abstract

Preordering of a source language sentence to match target word order has proved to be useful for improving machine translation systems.

Introduction

Dealing with word order differences between source and target languages presents a significant challenge for machine translation systems.

Reordering issues in Urdu-English translation

In this section we describe the main sources of word order differences between Urdu and English since this is the language pair we experiment with in this paper.

Reordering model

In this section we briefly describe the reordering model (Visweswariah et al., 2011) that forms the basis of our work.

Generating reference reordering from parallel sentences

The main aim of our work is to improve the reordering model by using parallel sentences for which manual word alignments are not available.

Experimental setup

In this section we describe the experimental setup that we used to evaluate the models proposed in this paper.

Results and Discussions

We now discuss the results of our experiments.

Related work

Dealing with the problem of handling word order differences in machine translation has recently received much attention.

Conclusion

In the paper we showed that a reordering model can benefit from data beyond a relatively small corpus of manual word alignments.

Topics

word alignments

Appears in 44 sentences as: word aligned (4) word alignment (3) word alignments (42)
In Cut the noise: Mutually reinforcing reordering and alignments for improved machine translation
  1. Previous work has shown that a reordering model can be learned from high quality manual word alignments to improve machine translation performance.
    Page 1, “Abstract”
  2. In this paper, we focus on further improving the performance of the reordering model (and thereby machine translation) by using a larger corpus of sentence aligned data for which manual word alignments are not available but automatic machine generated alignments are available.
    Page 1, “Abstract”
  3. To mitigate the effect of noisy machine alignments, we propose a novel approach that improves reorderings produced given noisy alignments and also improves word alignments using information from the reordering model.
    Page 1, “Abstract”
  4. The data generated allows us to train a reordering model that gives an improvement of 1.8 BLEU points on the NIST MT—08 Urdu-English evaluation set over a reordering model that only uses manual word alignments , and a gain of 5.2 BLEU points over a standard phrase-based baseline.
    Page 1, “Abstract”
  5. These methods use a small corpus of manual word alignments (where the words in the source sentence are manually aligned to the words in the target sentence) to learn a model to preorder the source sentence to match target order.
    Page 1, “Introduction”
  6. In this paper, we build upon the approach in (Visweswariah et al., 2011) which uses manual word alignments for learning a reordering model.
    Page 1, “Introduction”
  7. Specifically, we show that we can significantly improve reordering performance by using a large number of sentence pairs for which manual word alignments are not available.
    Page 1, “Introduction”
  8. The motivation for going beyond manual word alignments is clear: the reordering model can have millions of features and estimating weights for the features on thousands of sentences of manual word alignments is
    Page 1, “Introduction”
  9. This will cut down on the number of features and perhaps the model would be leamable with a small set of manual word alignments .
    Page 2, “Introduction”
  10. Unfortunately, as we will see in the experimental section, leaving out lexical information from the models hurts performance even with a relatively small set of manual word alignments .
    Page 2, “Introduction”
  11. Another option would be to collect more manual word alignments but this is undesirable because it is time consuming and expensive.
    Page 2, “Introduction”

See all papers in Proc. ACL 2013 that mention word alignments.

See all papers in Proc. ACL that mention word alignments.

Back to top.

alignment model

Appears in 14 sentences as: (2) alignment model (12) alignment models (1)
In Cut the noise: Mutually reinforcing reordering and alignments for improved machine translation
  1. Complementing this model, we build an alignment model (P(a|ws,wt,7rs,7rt)) that scores alignments a given the source and target sentences and their predicted reorderings according to source and target reordering models.
    Page 4, “Generating reference reordering from parallel sentences”
  2. The model (C(773|ws, wt, a)) helps to produce better reference reorderings for training our final reordering model given fixed machine alignments and the alignment model (P (a|ws, Wt, 773, 79)) helps improve the machine alignments taking into account information from reordering models.
    Page 4, “Generating reference reordering from parallel sentences”
  3. St 2: Feed predictions the reordering models to the alignment model
    Page 4, “Generating reference reordering from parallel sentences”
  4. p 3: Feed predictions of e alignment model to the eordering models
    Page 4, “Generating reference reordering from parallel sentences”
  5. Figure 2: Overall approach: Building a sequence of reordering and alignment models .
    Page 4, “Generating reference reordering from parallel sentences”
  6. The basic idea is to chain together the two models, viz, reordering model and alignment model , as illustrated in Figure 2.
    Page 4, “Generating reference reordering from parallel sentences”
  7. Step 2: Next, we use the hand alignments to train an alignment model P(a|ws, wt, 773 , 7ft).
    Page 4, “Generating reference reordering from parallel sentences”
  8. In addition to the original source and target sentence, we also feed the predictions of the reordering model trained in Step 1 to this alignment model (see section 4.2 for details of the model itself).
    Page 4, “Generating reference reordering from parallel sentences”
  9. Step 3: Finally, we use the predictions of the alignment model trained in Step 2 to train reordering models C(773|ws,wt, a) (see section 4.3 for details on the reordering model itself).
    Page 4, “Generating reference reordering from parallel sentences”
  10. Consider the case when we are training an alignment model conditioned on reorderings (P(a|ws,wt, 71's, 79)).
    Page 5, “Generating reference reordering from parallel sentences”
  11. If the reordering model that generated these reorderings 71'3,71"5 were trained on the same data that we are using to train the alignment model, then the reorderings would be much better than we would expect on unseen test data, and hence the alignment model (P(a|ws,wt, 71's, 79)) may learn to make the alignment overly consistent with the reorderings 71'3 and 7ft.
    Page 5, “Generating reference reordering from parallel sentences”

See all papers in Proc. ACL 2013 that mention alignment model.

See all papers in Proc. ACL that mention alignment model.

Back to top.

machine translation

Appears in 13 sentences as: machine translation (14)
In Cut the noise: Mutually reinforcing reordering and alignments for improved machine translation
  1. Preordering of a source language sentence to match target word order has proved to be useful for improving machine translation systems.
    Page 1, “Abstract”
  2. Previous work has shown that a reordering model can be learned from high quality manual word alignments to improve machine translation performance.
    Page 1, “Abstract”
  3. In this paper, we focus on further improving the performance of the reordering model (and thereby machine translation ) by using a larger corpus of sentence aligned data for which manual word alignments are not available but automatic machine generated alignments are available.
    Page 1, “Abstract”
  4. Dealing with word order differences between source and target languages presents a significant challenge for machine translation systems.
    Page 1, “Introduction”
  5. in machine translation output that is not fluent and is often very hard to understand.
    Page 1, “Introduction”
  6. This results in a 1.8 BLEU point gain in machine translation performance on an Urdu-English machine translation task over a preordering model trained using only manual word alignments.
    Page 2, “Introduction”
  7. Section 5 presents the experimental setup used for evaluating the models proposed in this paper on an Urdu-English machine translation task.
    Page 2, “Introduction”
  8. Additionally, we evaluate the effect of reordering on our final systems for machine translation measured using BLEU.
    Page 6, “Experimental setup”
  9. The parallel corpus is used for building our phrased based machine translation system and to add training data for our reordering model.
    Page 6, “Experimental setup”
  10. For our machine translation experiments, we used a standard phrase based system (Al-Onaizan and Papineni, 2006) with a lexicalized distortion model with a window size of +/-4 words5.
    Page 6, “Experimental setup”
  11. We see a significant gain of 1.8 BLEU points in machine translation by going beyond manual word alignments using the best reordering model reported in Table 3.
    Page 8, “Results and Discussions”

See all papers in Proc. ACL 2013 that mention machine translation.

See all papers in Proc. ACL that mention machine translation.

Back to top.

model trained

Appears in 9 sentences as: model trained (7) model training (1) models trained (3)
In Cut the noise: Mutually reinforcing reordering and alignments for improved machine translation
  1. A reordering model trained on such incorrect reorderings would obviously perform poorly.
    Page 2, “Introduction”
  2. Our experiments show that reordering models trained using these improved machine alignments perform significantly better than models trained only on manual word alignments.
    Page 2, “Introduction”
  3. This results in a 1.8 BLEU point gain in machine translation performance on an Urdu-English machine translation task over a preordering model trained using only manual word alignments.
    Page 2, “Introduction”
  4. However, as we will see in the experimental results, the quality of a reordering model trained from automatic alignments is very sensitive to the quality of alignments.
    Page 4, “Generating reference reordering from parallel sentences”
  5. In addition to the original source and target sentence, we also feed the predictions of the reordering model trained in Step 1 to this alignment model (see section 4.2 for details of the model itself).
    Page 4, “Generating reference reordering from parallel sentences”
  6. Step 3: Finally, we use the predictions of the alignment model trained in Step 2 to train reordering models C(773|ws,wt, a) (see section 4.3 for details on the reordering model itself).
    Page 4, “Generating reference reordering from parallel sentences”
  7. Using fewer features: We compare the performance of a model trained using lexical features for all words (Column 2 of Table l) with a model trained using lexical features only for the 1000 most frequent words (Column 3 of Table l).
    Page 6, “Results and Discussions”
  8. Table 2: mBLEU scores for Urdu to English reordering using models trained on different data sources and tested on a development set of 8017 Urdu tokens.
    Page 7, “Results and Discussions”
  9. Table 3: mBLEU with different methods to generate reordering model training data from a machine aligned parallel corpus in addition to manual word alignments.
    Page 7, “Results and Discussions”

See all papers in Proc. ACL 2013 that mention model trained.

See all papers in Proc. ACL that mention model trained.

Back to top.

BLEU

Appears in 8 sentences as: BLEU (12)
In Cut the noise: Mutually reinforcing reordering and alignments for improved machine translation
  1. The data generated allows us to train a reordering model that gives an improvement of 1.8 BLEU points on the NIST MT—08 Urdu-English evaluation set over a reordering model that only uses manual word alignments, and a gain of 5.2 BLEU points over a standard phrase-based baseline.
    Page 1, “Abstract”
  2. This results in a 1.8 BLEU point gain in machine translation performance on an Urdu-English machine translation task over a preordering model trained using only manual word alignments.
    Page 2, “Introduction”
  3. In all, this increases the gain in performance by using the preordering model to 5.2 BLEU points over a standard phrase-based system with no preordering.
    Page 2, “Introduction”
  4. All experiments were done on Urdu-English and we evaluate reordering in two ways: Firstly, we evaluate reordering performance directly by comparing the reordered source sentence in Urdu with a reference reordering obtained from the manual word alignments using BLEU (Papineni et al., 2002) (we call this measure monolingual BLEU or mBLEU).
    Page 6, “Experimental setup”
  5. Additionally, we evaluate the effect of reordering on our final systems for machine translation measured using BLEU .
    Page 6, “Experimental setup”
  6. We see a significant gain of 1.8 BLEU points in machine translation by going beyond manual word alignments using the best reordering model reported in Table 3.
    Page 8, “Results and Discussions”
  7. We also note a gain of 2.0 BLEU points over a hierarchical phrase based system.
    Page 8, “Results and Discussions”
  8. Cumulatively, we see a gain of 1.8 BLEU points over a baseline reordering model that only uses manual word alignments, a gain of 2.0 BLEU points over a hierarchical phrase based system, and a gain of 5.2 BLEU points over a phrase based
    Page 8, “Conclusion”

See all papers in Proc. ACL 2013 that mention BLEU.

See all papers in Proc. ACL that mention BLEU.

Back to top.

BLEU points

Appears in 6 sentences as: BLEU point (1) BLEU points (8)
In Cut the noise: Mutually reinforcing reordering and alignments for improved machine translation
  1. The data generated allows us to train a reordering model that gives an improvement of 1.8 BLEU points on the NIST MT—08 Urdu-English evaluation set over a reordering model that only uses manual word alignments, and a gain of 5.2 BLEU points over a standard phrase-based baseline.
    Page 1, “Abstract”
  2. This results in a 1.8 BLEU point gain in machine translation performance on an Urdu-English machine translation task over a preordering model trained using only manual word alignments.
    Page 2, “Introduction”
  3. In all, this increases the gain in performance by using the preordering model to 5.2 BLEU points over a standard phrase-based system with no preordering.
    Page 2, “Introduction”
  4. We see a significant gain of 1.8 BLEU points in machine translation by going beyond manual word alignments using the best reordering model reported in Table 3.
    Page 8, “Results and Discussions”
  5. We also note a gain of 2.0 BLEU points over a hierarchical phrase based system.
    Page 8, “Results and Discussions”
  6. Cumulatively, we see a gain of 1.8 BLEU points over a baseline reordering model that only uses manual word alignments, a gain of 2.0 BLEU points over a hierarchical phrase based system, and a gain of 5.2 BLEU points over a phrase based
    Page 8, “Conclusion”

See all papers in Proc. ACL 2013 that mention BLEU points.

See all papers in Proc. ACL that mention BLEU points.

Back to top.

parallel corpus

Appears in 6 sentences as: parallel corpus (6)
In Cut the noise: Mutually reinforcing reordering and alignments for improved machine translation
  1. This model allows us to combine features from the original reordering model along with information coming from the alignments to find source reorderings given a parallel corpus and alignments.
    Page 6, “Generating reference reordering from parallel sentences”
  2. We use a parallel corpus of 3.9M words consisting of 1.7M words from the NIST MT—08 training data set and 2.2M words extracted from parallel news stories on the
    Page 6, “Experimental setup”
  3. The parallel corpus is used for building our phrased based machine translation system and to add training data for our reordering model.
    Page 6, “Experimental setup”
  4. For our English language model, we use the Gigaword English corpus in addition to the English side of our parallel corpus .
    Page 6, “Experimental setup”
  5. Table 3: mBLEU with different methods to generate reordering model training data from a machine aligned parallel corpus in addition to manual word alignments.
    Page 7, “Results and Discussions”
  6. (DeNero and Uszkoreit, 2011; Visweswariah et al., 2011; Neubig et al., 2012) focus on the use of manual word alignments to learn preordering models and in both cases no benefit was obtained by using the parallel corpus in addition to manual word alignments.
    Page 8, “Related work”

See all papers in Proc. ACL 2013 that mention parallel corpus.

See all papers in Proc. ACL that mention parallel corpus.

Back to top.

word order

Appears in 6 sentences as: word order (6)
In Cut the noise: Mutually reinforcing reordering and alignments for improved machine translation
  1. Preordering of a source language sentence to match target word order has proved to be useful for improving machine translation systems.
    Page 1, “Abstract”
  2. Dealing with word order differences between source and target languages presents a significant challenge for machine translation systems.
    Page 1, “Introduction”
  3. Recently, approaches that address the problem of word order differences between the source and target language without requiring a high quality source or target parser have been proposed (DeNero and Uszkoreit, 2011; Visweswariah et al., 2011; Neubig et al., 2012).
    Page 1, “Introduction”
  4. In this section we describe the main sources of word order differences between Urdu and English since this is the language pair we experiment with in this paper.
    Page 2, “Reordering issues in Urdu-English translation”
  5. The typical word order in Urdu is Subject-Object-Verb unlike English in which the order is Subject-Verb-Object.
    Page 2, “Reordering issues in Urdu-English translation”
  6. Dealing with the problem of handling word order differences in machine translation has recently received much attention.
    Page 8, “Related work”

See all papers in Proc. ACL 2013 that mention word order.

See all papers in Proc. ACL that mention word order.

Back to top.

f-Measure

Appears in 5 sentences as: f-Measure (5)
In Cut the noise: Mutually reinforcing reordering and alignments for improved machine translation
  1. This approach generates alignments that are 2.6 f-Measure points better than a baseline supervised aligner.
    Page 1, “Abstract”
  2. Type f-Measure (words)
    Page 7, “Results and Discussions”
  3. The f-Measure of this aligner is 78.1% (see row 1, column 2).
    Page 7, “Results and Discussions”
  4. Method f-Measure mBLEU Base Correction model 78.1 55.1 Correction model, C(fl'la) 78.1 56.4 P(alfl'), C(fl'la) 80.7 57.6
    Page 7, “Results and Discussions”
  5. We also proposed a model that scores alignments given source and target sentence reorderings that improves a supervised alignment model by 2.6 points in f-Measure .
    Page 8, “Conclusion”

See all papers in Proc. ACL 2013 that mention f-Measure.

See all papers in Proc. ACL that mention f-Measure.

Back to top.

MaxEnt

Appears in 4 sentences as: MaxEnt (4)
In Cut the noise: Mutually reinforcing reordering and alignments for improved machine translation
  1. This model was significantly better than the MaxEnt aligner (Ittycheriah and Roukos, 2005) and is also flexible in the sense that it allows for arbitrary features to be introduced while still keeping training and decoding tractable by using a greedy decoding algorithm that explores potential alignments in a small neighborhood of the current alignment.
    Page 5, “Generating reference reordering from parallel sentences”
  2. The model thus needs a reasonably good initial alignment to start with for which we use the MaxEnt aligner (Ittycheriah and Roukos, 2005) as in McCarley et al.
    Page 5, “Generating reference reordering from parallel sentences”
  3. None - 35.5 Manual 180K 52.5 MaxEnt 70.0 3.9M 49.5
    Page 7, “Results and Discussions”
  4. We see that the quality of the alignments matter a great deal to the reordering model; using MaxEnt alignments cause a degradation in performance over just using a small set of manual word alignments.
    Page 7, “Results and Discussions”

See all papers in Proc. ACL 2013 that mention MaxEnt.

See all papers in Proc. ACL that mention MaxEnt.

Back to top.

NIST

Appears in 4 sentences as: NIST (4)
In Cut the noise: Mutually reinforcing reordering and alignments for improved machine translation
  1. The data generated allows us to train a reordering model that gives an improvement of 1.8 BLEU points on the NIST MT—08 Urdu-English evaluation set over a reordering model that only uses manual word alignments, and a gain of 5.2 BLEU points over a standard phrase-based baseline.
    Page 1, “Abstract”
  2. We use about 10K sentences (180K words) of manual word alignments which were created in house using part of the NIST MT—08 training data3 to train our baseline reordering model and to train our supervised machine aligners.
    Page 6, “Experimental setup”
  3. We use a parallel corpus of 3.9M words consisting of 1.7M words from the NIST MT—08 training data set and 2.2M words extracted from parallel news stories on the
    Page 6, “Experimental setup”
  4. We report results on the (four reference) NIST MT—08 evaluation set in Table 4 for the News and Web conditions.
    Page 6, “Experimental setup”

See all papers in Proc. ACL 2013 that mention NIST.

See all papers in Proc. ACL that mention NIST.

Back to top.

language pairs

Appears in 3 sentences as: language pair (1) language pairs (2)
In Cut the noise: Mutually reinforcing reordering and alignments for improved machine translation
  1. In this section we describe the main sources of word order differences between Urdu and English since this is the language pair we experiment with in this paper.
    Page 2, “Reordering issues in Urdu-English translation”
  2. The task of directly learning a reordering model for language pairs that are very different is closely related to the task of parsing and hence work on semi-supervised parsing (Koo et al., 2008; McClosky et al., 2006; Suzuki et al., 2009) is broadly related to our work.
    Page 8, “Related work”
  3. As future work we would like to evaluate our models on other language pairs .
    Page 9, “Conclusion”

See all papers in Proc. ACL 2013 that mention language pairs.

See all papers in Proc. ACL that mention language pairs.

Back to top.

parallel sentences

Appears in 3 sentences as: parallel sentences (3)
In Cut the noise: Mutually reinforcing reordering and alignments for improved machine translation
  1. The main aim of our work is to improve the reordering model by using parallel sentences for which manual word alignments are not available.
    Page 4, “Generating reference reordering from parallel sentences”
  2. In other words, we want to generate relatively clean reference reorderings from parallel sentences and use them for training a reordering model.
    Page 4, “Generating reference reordering from parallel sentences”
  3. word alignments (H) and a much larger corpus of parallel sentences (U) that are not word aligned.
    Page 4, “Generating reference reordering from parallel sentences”

See all papers in Proc. ACL 2013 that mention parallel sentences.

See all papers in Proc. ACL that mention parallel sentences.

Back to top.

POS tags

Appears in 3 sentences as: POS tags (4)
In Cut the noise: Mutually reinforcing reordering and alignments for improved machine translation
  1. where 6 is a learned vector of weights and (I) is a vector of binary feature functions that inspect the words and POS tags of the source sentence at and around positions m and n. We use the features ((1)) described in Visweswariah et al.
    Page 3, “Reordering model”
  2. the Model 1 probabilities between pairs of words linked in the alignment a, features that inspect source and target POS tags and parses (if available) and features that inspect the alignments of adjacent words in the source and target sentence.
    Page 5, “Generating reference reordering from parallel sentences”
  3. We conjoin the msd (minimum signed distance) with the POS tags to allow the model to capture the fact that the alignment error rate maybe higher for some POS tags than others (e.g., we have observed verbs have a higher error rate in Urdu-English alignments).
    Page 6, “Generating reference reordering from parallel sentences”

See all papers in Proc. ACL 2013 that mention POS tags.

See all papers in Proc. ACL that mention POS tags.

Back to top.

sentence pairs

Appears in 3 sentences as: sentence pairs (3)
In Cut the noise: Mutually reinforcing reordering and alignments for improved machine translation
  1. Specifically, we show that we can significantly improve reordering performance by using a large number of sentence pairs for which manual word alignments are not available.
    Page 1, “Introduction”
  2. In this paper we focus on the case where in addition to using a relatively small number of manual word aligned sentences to derive the reference permutations 77* used to train our model, we would like to use more abundant but noisier machine aligned sentence pairs .
    Page 3, “Reordering model”
  3. We use H to refer to the manually word aligned data and U to refer to the additional sentence pairs for which manual word alignments are not available.
    Page 7, “Results and Discussions”

See all papers in Proc. ACL 2013 that mention sentence pairs.

See all papers in Proc. ACL that mention sentence pairs.

Back to top.

translation systems

Appears in 3 sentences as: translation system (1) translation systems (2)
In Cut the noise: Mutually reinforcing reordering and alignments for improved machine translation
  1. Preordering of a source language sentence to match target word order has proved to be useful for improving machine translation systems .
    Page 1, “Abstract”
  2. Dealing with word order differences between source and target languages presents a significant challenge for machine translation systems .
    Page 1, “Introduction”
  3. The parallel corpus is used for building our phrased based machine translation system and to add training data for our reordering model.
    Page 6, “Experimental setup”

See all papers in Proc. ACL 2013 that mention translation systems.

See all papers in Proc. ACL that mention translation systems.

Back to top.

PDF available from www.aclweb.org.
Send us an email