Confidence Measure for Word Alignment
Huang, Fei

Article Structure

Abstract

In this paper we present a confidence measure for word alignment based on the posterior probability of alignment links.

Introduction

Data-driven approaches have been quite active in recent machine translation (MT) research.

Sentence Alignment Confidence Measure

2.1 Definition

Alignment Link Confidence Measure

3.1 Definition

Improved MaXEnt Aligner with Confidence-based Link Filtering

In addition to the alignment combination, we also improve the performance of the MaXEnt aligner through confidence-based alignment link filtering.

Translation

We evaluate the improved alignment on several Chinese-English and Arabic-English machine translation tasks.

Related Work

In the machine translation area, most research on confidence measure focus on the confidence of MT output: how accurate a translated sentence is.

Conclusion

In this paper we presented two alignment confidence measures for word alignment.

Topics

word alignment

Appears in 22 sentences as: word alignment (21) word alignments (3) word alignment’s (1)
In Confidence Measure for Word Alignment
  1. In this paper we present a confidence measure for word alignment based on the posterior probability of alignment links.
    Page 1, “Abstract”
  2. Based on these measures, we improve the alignment quality by selecting high confidence sentence alignments and alignment links from multiple word alignments of the same sentence pair.
    Page 1, “Abstract”
  3. Additionally, we remove low confidence alignment links from the word alignment of a bilingual training corpus, which increases the alignment F-score, improves Chinese-English and Arabic-English translation quality and significantly reduces the phrase translation table size.
    Page 1, “Abstract”
  4. Many MT systems, such as statistical phrase-based and syntax-based systems, learn phrase translation pairs or translation rules from large amount of bilingual data with word alignment .
    Page 1, “Introduction”
  5. The quality of the parallel data and the word alignment have significant impacts on the learned translation models and ultimately the quality of translation output.
    Page 1, “Introduction”
  6. Given the huge amount of bilingual training data, word alignments are automatically generated using various algorithms ((Brown et al., 1994), (Vogel et al., 1996)
    Page 1, “Introduction”
  7. Figure 1: An example of inaccurate translation and word alignment .
    Page 1, “Introduction”
  8. and (Ittycheriah and Roukos, 2005)), which also introduce many word alignment errors.
    Page 1, “Introduction”
  9. The example in Figure 1 shows the word alignment of the given Chinese and English sentence pair, where the English words following each Chinese word is its literal translation.
    Page 1, “Introduction”
  10. These spurious words cause significant word alignment errors (as shown with dash lines), which in turn directly affect the quality of phrase translation tables or translation rules that are learned based on word alignment .
    Page 1, “Introduction”
  11. In this paper we introduce a confidence measure for word alignment, which is robust to extra or missing words in the bilingual sentence pairs, as well as word alignment errors.
    Page 1, “Introduction”

See all papers in Proc. ACL 2009 that mention word alignment.

See all papers in Proc. ACL that mention word alignment.

Back to top.

confidence scores

Appears in 19 sentences as: confidence score (9) confidence scores (11)
In Confidence Measure for Word Alignment
  1. For each sentence pair, we also calculate the sentence alignment confidence score — log 0 (A|S, T).
    Page 2, “Sentence Alignment Confidence Measure”
  2. measure suggests the possibility of selecting the alignment with the highest confidence score to obtain better alignments.
    Page 3, “Sentence Alignment Confidence Measure”
  3. For each sentence pair in the CE test set, we calculate the confidence scores of the HMM alignment, the Block Model alignment and the MaXEnt alignment, then select the alignment with the highest confidence score .
    Page 3, “Sentence Alignment Confidence Measure”
  4. This indicates that the confidence score of any link connecting 253- to any source word is at most 1 /N .
    Page 3, “Alignment Link Confidence Measure”
  5. From multiple alignments of the same sentence pair, we select high confidence links from different alignments based on their link confidence scores and alignment agreement ratio.
    Page 3, “Alignment Link Confidence Measure”
  6. Where C (A) is the confidence score of the alignment A as defined in formula 1.
    Page 3, “Alignment Link Confidence Measure”
  7. This formula computes the sum of the alignment confidence scores for the alignments containing aij, which is
    Page 3, “Alignment Link Confidence Measure”
  8. normalized by the sum of all alignments’ confidence scores .
    Page 4, “Alignment Link Confidence Measure”
  9. For each link we calculate the link confidence score C(aij) and the alignment agreement ratio New).
    Page 4, “Alignment Link Confidence Measure”
  10. Select links whose confidence scores are higher than an empirically chosen threshold H as anchor links 1.
    Page 5, “Improved MaXEnt Aligner with Confidence-based Link Filtering”
  11. 2When two equally close alignment links have the same confidence score ), we randomly select one of the tied links as the anchor link.
    Page 5, “Improved MaXEnt Aligner with Confidence-based Link Filtering”

See all papers in Proc. ACL 2009 that mention confidence scores.

See all papers in Proc. ACL that mention confidence scores.

Back to top.

sentence pairs

Appears in 15 sentences as: sentence pair (8) sentence pairs (10)
In Confidence Measure for Word Alignment
  1. Based on these measures, we improve the alignment quality by selecting high confidence sentence alignments and alignment links from multiple word alignments of the same sentence pair .
    Page 1, “Abstract”
  2. The example in Figure 1 shows the word alignment of the given Chinese and English sentence pair , where the English words following each Chinese word is its literal translation.
    Page 1, “Introduction”
  3. In this paper we introduce a confidence measure for word alignment, which is robust to extra or missing words in the bilingual sentence pairs , as well as word alignment errors.
    Page 1, “Introduction”
  4. Given a bilingual sentence pair (S,T) where S={31,. .
    Page 2, “Sentence Alignment Confidence Measure”
  5. We randomly selected 512 Chinese-English (CE) sentence pairs and generated word alignment using the MaxEnt aligner (Ittycheriah and Roukos, 2005).
    Page 2, “Sentence Alignment Confidence Measure”
  6. For each sentence pair , we also calculate the sentence alignment confidence score — log 0 (A|S, T).
    Page 2, “Sentence Alignment Confidence Measure”
  7. For each sentence pair in the CE test set, we calculate the confidence scores of the HMM alignment, the Block Model alignment and the MaXEnt alignment, then select the alignment with the highest confidence score.
    Page 3, “Sentence Alignment Confidence Measure”
  8. Similar to the sentence alignment confidence measure, the confidence of an alignment link aij in the sentence pair (S, T) is defined as
    Page 3, “Alignment Link Confidence Measure”
  9. From multiple alignments of the same sentence pair , we select high confidence links from different alignments based on their link confidence scores and alignment agreement ratio.
    Page 3, “Alignment Link Confidence Measure”
  10. Suppose the sentence pair (8, T) have alignments A1,.
    Page 3, “Alignment Link Confidence Measure”
  11. 512 sentence pairs, and the A-E alignment test set is the 200 Arabic-English sentence pairs from NIST MT03 test set.
    Page 6, “Improved MaXEnt Aligner with Confidence-based Link Filtering”

See all papers in Proc. ACL 2009 that mention sentence pairs.

See all papers in Proc. ACL that mention sentence pairs.

Back to top.

MaXEnt

Appears in 12 sentences as: MaXEnt (9) MaxEnt (3)
In Confidence Measure for Word Alignment
  1. In section 4 we show how to improve a MaXEnt word alignment quality by removing low confidence alignment links, which also leads to improved translation quality as shown in section 5.
    Page 2, “Introduction”
  2. HMM 54.72 -0.710 BM 62.53 -0.699 MaxEnt 69.26 -0.699
    Page 2, “Sentence Alignment Confidence Measure”
  3. We randomly selected 512 Chinese-English (CE) sentence pairs and generated word alignment using the MaxEnt aligner (Ittycheriah and Roukos, 2005).
    Page 2, “Sentence Alignment Confidence Measure”
  4. For each sentence pair in the CE test set, we calculate the confidence scores of the HMM alignment, the Block Model alignment and the MaXEnt alignment, then select the alignment with the highest confidence score.
    Page 3, “Sentence Alignment Confidence Measure”
  5. As a result, 82% of selected alignments have higher F-scores, and the F—measure of the combined alignments is increased over the best aligner (the MaxEnt aligner) by 0.8.
    Page 3, “Sentence Alignment Confidence Measure”
  6. We combine the HMM alignment, the BM alignment and the MaXEnt alignment (ME) using the above link selection algorithm.
    Page 4, “Alignment Link Confidence Measure”
  7. Figure 3 shows such an example, where alignment errors in the MaXEnt alignment are shown with dotted lines.
    Page 4, “Alignment Link Confidence Measure”
  8. In addition to the alignment combination, we also improve the performance of the MaXEnt aligner through confidence-based alignment link filtering.
    Page 4, “Improved MaXEnt Aligner with Confidence-based Link Filtering”
  9. Here we select the MaXEnt aligner because it has
    Page 4, “Improved MaXEnt Aligner with Confidence-based Link Filtering”
  10. We extract phrase translation tables from the baseline MaXEnt word alignment as well as the alignment with confidence-based link filtering, then translate the test set with each phrase translation table.
    Page 6, “Translation”
  11. Regarding word alignment combination, in addition to the commonly used ”intersection-union-refine” approach (Och and Ney, 2003), (Ayan and Dorr, 2006b) and (Ayan et al., 2005) combined alignment links from multiple word alignment based on a set of linguistic and alignment features within the MaXEnt framework or a neural net model.
    Page 7, “Related Work”

See all papers in Proc. ACL 2009 that mention MaXEnt.

See all papers in Proc. ACL that mention MaXEnt.

Back to top.

Chinese-English

Appears in 7 sentences as: Chinese-English (7)
In Confidence Measure for Word Alignment
  1. Additionally, we remove low confidence alignment links from the word alignment of a bilingual training corpus, which increases the alignment F-score, improves Chinese-English and Arabic-English translation quality and significantly reduces the phrase translation table size.
    Page 1, “Abstract”
  2. We randomly selected 512 Chinese-English (CE) sentence pairs and generated word alignment using the MaxEnt aligner (Ittycheriah and Roukos, 2005).
    Page 2, “Sentence Alignment Confidence Measure”
  3. We applied the confidence-based link filtering on Chinese-English and Arabic-English word alignment.
    Page 5, “Improved MaXEnt Aligner with Confidence-based Link Filtering”
  4. We evaluate the improved alignment on several Chinese-English and Arabic-English machine translation tasks.
    Page 6, “Translation”
  5. In the Chinese-English MT experiment, we selected 40 NW documents, 41 WE documents as the test set, which includes 623 sentences with 16667 words.
    Page 6, “Translation”
  6. lected among multiple alignments and it obtained 0.8 F-measure improvement over the single best Chinese-English aligner.
    Page 8, “Conclusion”
  7. When we removed low confidence links from the MaXEnt aligner, we reduced the Chinese-English alignment error by 5% and the Arabic-English alignment error by 10%.
    Page 8, “Conclusion”

See all papers in Proc. ACL 2009 that mention Chinese-English.

See all papers in Proc. ACL that mention Chinese-English.

Back to top.

F-score

Appears in 7 sentences as: F-score (7)
In Confidence Measure for Word Alignment
  1. Additionally, we remove low confidence alignment links from the word alignment of a bilingual training corpus, which increases the alignment F-score , improves Chinese-English and Arabic-English translation quality and significantly reduces the phrase translation table size.
    Page 1, “Abstract”
  2. Aligner F-score Cor.
    Page 2, “Sentence Alignment Confidence Measure”
  3. The results in Figure 2 shows strong correlation between the confidence measure and the alignment F-score , with the correlation coefficients equals to -0.69.
    Page 2, “Sentence Alignment Confidence Measure”
  4. Table 2 shows the precision, recall and F-score of individual alignments and the combined align-
    Page 4, “Alignment Link Confidence Measure”
  5. Overall it improves the F-score by 1.5 points (from 69.3 to 70.8), 1.8 point improvement for content words and 1.0 point for function words.
    Page 4, “Alignment Link Confidence Measure”
  6. Precision Recall F-score Baseline 72.66 66.17 69.26 +ALF 78.14 64.36 70.59
    Page 6, “Improved MaXEnt Aligner with Confidence-based Link Filtering”
  7. Precision Recall F-score Baseline 84.43 83.64 84.04 +ALF 88.29 83.14 85.64
    Page 6, “Improved MaXEnt Aligner with Confidence-based Link Filtering”

See all papers in Proc. ACL 2009 that mention F-score.

See all papers in Proc. ACL that mention F-score.

Back to top.

translation quality

Appears in 6 sentences as: translation quality (6)
In Confidence Measure for Word Alignment
  1. Additionally, we remove low confidence alignment links from the word alignment of a bilingual training corpus, which increases the alignment F-score, improves Chinese-English and Arabic-English translation quality and significantly reduces the phrase translation table size.
    Page 1, “Abstract”
  2. In section 4 we show how to improve a MaXEnt word alignment quality by removing low confidence alignment links, which also leads to improved translation quality as shown in section 5.
    Page 2, “Introduction”
  3. We measure the translation quality with automatic metrics including BLEU (Papineni et al., 2001) and TER (Snover et al., 2006).
    Page 6, “Translation”
  4. The higher the BLEU score is, or the lower the TER score is, the better the translation quality is.
    Page 6, “Translation”
  5. For newswire, the translation quality is improved by 0.44 on the whole test set and 1.1 on the tail documents, as measured by (TER-BLEU)/2.
    Page 6, “Translation”
  6. This is similar to the ”loose phrases” described in (Ayan and Dorr, 2006a), which increased the number of correct phrase translations and improved the translation quality .
    Page 7, “Related Work”

See all papers in Proc. ACL 2009 that mention translation quality.

See all papers in Proc. ACL that mention translation quality.

Back to top.

word pair

Appears in 6 sentences as: word pair (4) word pairs (2)
In Confidence Measure for Word Alignment
  1. It is the product of lexical translation probabilities for the aligned word pairs .
    Page 2, “Sentence Alignment Confidence Measure”
  2. which is defined as the word translation probability of the aligned word pair divided by the sum of the translation probabilities over all the target words in the sentence.
    Page 3, “Alignment Link Confidence Measure”
  3. Intuitively, the above link confidence definition compares the lexical translation probability of the aligned word pair with the translation probabilities of all the target words given the source word.
    Page 3, “Alignment Link Confidence Measure”
  4. On the other hand, additional information (such as the distance of the word pair , the alignment of neighbor words) could indicate higher likelihood for the alignment link.
    Page 3, “Alignment Link Confidence Measure”
  5. We link the word pair (51-, 253-) if either C(aij) > hl or Maij) > 71, where hl and T1 are empirically chosen thresholds.
    Page 4, “Alignment Link Confidence Measure”
  6. Furthermore, it is possible to create new links by relinking unaligned source and target word pairs within the context window if their context-dependent link posterior probability is high.
    Page 5, “Improved MaXEnt Aligner with Confidence-based Link Filtering”

See all papers in Proc. ACL 2009 that mention word pair.

See all papers in Proc. ACL that mention word pair.

Back to top.

F-measure

Appears in 5 sentences as: F-measure (5)
In Confidence Measure for Word Alignment
  1. the highest F-measure among the three aligners, although the algorithm described below can be applied to any aligner.
    Page 5, “Improved MaXEnt Aligner with Confidence-based Link Filtering”
  2. For CE alignment, removing low confidence alignment links increased alignment precision by 5.5 point, while decreased recall by 1.8 point, and the overall alignment F-measure is increased by 1.3 point.
    Page 6, “Improved MaXEnt Aligner with Confidence-based Link Filtering”
  3. When looking into the alignment links which are removed during the alignment link filtering process, we found that 80% of the removed links (1320 out of 1661 links) are incorrect alignments, For A-E alignment, it increased the precision by 3 points while reducing recall by 0.5 points, and the alignment F-measure is increased by about 1.5 points absolute, a 10% relative alignment error rate reduction.
    Page 6, “Improved MaXEnt Aligner with Confidence-based Link Filtering”
  4. lected among multiple alignments and it obtained 0.8 F-measure improvement over the single best Chinese-English aligner.
    Page 8, “Conclusion”
  5. The second is the alignment link confidence measure, which selects the most reliable links from multiple alignments and obtained 1.5 F-measure improvement.
    Page 8, “Conclusion”

See all papers in Proc. ACL 2009 that mention F-measure.

See all papers in Proc. ACL that mention F-measure.

Back to top.

translation probability

Appears in 5 sentences as: translation probabilities (3) translation probability (4)
In Confidence Measure for Word Alignment
  1. 2 82“ ' > 2.1/mans) ( ) When computing the source-to-target alignment posterior probability, the numerator is the sentence translation probability calculated according to the given alignment A:
    Page 2, “Sentence Alignment Confidence Measure”
  2. It is the product of lexical translation probabilities for the aligned word pairs.
    Page 2, “Sentence Alignment Confidence Measure”
  3. The denominator is the sentence translation probability summing over all possible alignments, which can be calculated similar to IBM Model 1 in (Brown et al., 1994)
    Page 2, “Sentence Alignment Confidence Measure”
  4. which is defined as the word translation probability of the aligned word pair divided by the sum of the translation probabilities over all the target words in the sentence.
    Page 3, “Alignment Link Confidence Measure”
  5. Intuitively, the above link confidence definition compares the lexical translation probability of the aligned word pair with the translation probabilities of all the target words given the source word.
    Page 3, “Alignment Link Confidence Measure”

See all papers in Proc. ACL 2009 that mention translation probability.

See all papers in Proc. ACL that mention translation probability.

Back to top.

machine translation

Appears in 4 sentences as: machine translation (4)
In Confidence Measure for Word Alignment
  1. Data-driven approaches have been quite active in recent machine translation (MT) research.
    Page 1, “Introduction”
  2. We evaluate the improved alignment on several Chinese-English and Arabic-English machine translation tasks.
    Page 6, “Translation”
  3. In the machine translation area, most research on confidence measure focus on the confidence of MT output: how accurate a translated sentence is.
    Page 7, “Related Work”
  4. (Ueff-ing et al., 2003) presented several word-level confidence measures for machine translation based on word posterior probabilities.
    Page 7, “Related Work”

See all papers in Proc. ACL 2009 that mention machine translation.

See all papers in Proc. ACL that mention machine translation.

Back to top.

content words

Appears in 3 sentences as: content word (1) content words (2)
In Confidence Measure for Word Alignment
  1. F-content and F-function are the F-scores for content words and function words, respectively.
    Page 4, “Alignment Link Confidence Measure”
  2. Overall it improves the F-score by 1.5 points (from 69.3 to 70.8), 1.8 point improvement for content words and 1.0 point for function words.
    Page 4, “Alignment Link Confidence Measure”
  3. On the other hand, removing incorrect content word links produced cleaner phrase translation tables.
    Page 7, “Related Work”

See all papers in Proc. ACL 2009 that mention content words.

See all papers in Proc. ACL that mention content words.

Back to top.

translation model

Appears in 3 sentences as: translation model (2) translation models (1)
In Confidence Measure for Word Alignment
  1. The quality of the parallel data and the word alignment have significant impacts on the learned translation models and ultimately the quality of translation output.
    Page 1, “Introduction”
  2. The source-to-target lexical translation model p(t|s) and target-to-source model p(s|t) can be obtained through IBM Model-l or HMM training.
    Page 2, “Sentence Alignment Confidence Measure”
  3. For the efficient computation of the denominator, we use the lexical translation model .
    Page 2, “Sentence Alignment Confidence Measure”

See all papers in Proc. ACL 2009 that mention translation model.

See all papers in Proc. ACL that mention translation model.

Back to top.

PDF available from www.aclweb.org.
Send us an email