Improve SMT Quality with Automatically Extracted Paraphrase Rules
He, Wei and Wu, Hua and Wang, Haifeng and Liu, Ting

Article Structure

Abstract

We propose a novel approach to improve SMT via paraphrase rules which are automatically extracted from the bilingual training data.

Introduction

The translation quality of the SMT system is highly related to the coverage of translation models.

Forward-Translation vs. Back-Translation

The Back-Translation method is mainly used for automatic MT evaluation (Rapp 2009).

Extraction of Paraphrase Rules

3.1 Definition of Paraphrase Rules

Paraphrasing the Input Sentences

The extracted paraphrase rules aim to rewrite the input sentences to an MT-favored form which may lead to a better translation.

Experiments

5.1 Experimental Data

Discussion

We make a detailed analysis on the Chinese-English translation results that are affected by our paraphrase rules.

Related Work

Previous studies on improving SMT using paraphrase rules focus on handcrafted rules.

Conclusion

In this paper, we propose a novel method for extracting paraphrase rules by comparing the source side of bilingual corpus to the target-to-source translation of the target side.

Topics

BLEU

Appears in 9 sentences as: BLEU (14)
In Improve SMT Quality with Automatically Extracted Paraphrase Rules
  1. The experimental results show that our proposed approach achieves significant improvements of l.6~3.6 points of BLEU in the oral domain and 0.5~l points in the news domain.
    Page 1, “Abstract”
  2. The experimental results show that our proposed approach achieves significant improvements of l.6~3.6 points of BLEU in the oral domain and 0.5~l points in the news domain.
    Page 2, “Introduction”
  3. As mentioned above, the detailed procedure is: T1 = S1 = T2 = Finally we compute BLEU (Papineni et al.
    Page 3, “Extraction of Paraphrase Rules”
  4. If the sentence in T 2 has a higher BLEU score than the aligned sentence in T1, the corresponding sentences in S0 and S1 are selected as candidate paraphrase sentence pairs, which are used in the following steps of paraphrase extractions.
    Page 3, “Extraction of Paraphrase Rules”
  5. The metrics for automatic evaluation were BLEU 3 and TER 4 (Snover et al., 2005).
    Page 5, “Experiments”
  6. (00,-, 01,-) are selected for the extraction of paraphrase rules if two conditions are satisfied: (1) BLEU(eZi) — BLEU(eli) > 61, and (2) BLEU(eZi) > 62, where BLEU(-) is a function for computing BLEU score; 61 and 62 are thresholds for balancing the rules number and the quality of paraphrase rules.
    Page 6, “Experiments”
  7. Our system gains significant improvements of 1.6~3.6 points of BLEU in the oral domain, and 0.5~1 points of BLEU in the news domain.
    Page 6, “Experiments”
  8. IWSLT 2005 BLEU TER baseline 0.4644 0.4164 para.
    Page 7, “Experiments”
  9. on BLEU score
    Page 7, “Discussion”

See all papers in Proc. ACL 2012 that mention BLEU.

See all papers in Proc. ACL that mention BLEU.

Back to top.

Chinese-English

Appears in 9 sentences as: Chinese-English (11)
In Improve SMT Quality with Automatically Extracted Paraphrase Rules
  1. The experiments were conducted in both Chinese-English and English-Chinese directions for the oral group, and Chinese-English direction for the news group.
    Page 5, “Experiments”
  2. corpora, including the Chinese-English Sentence Aligned Bilingual Corpus (CLDC-LAC-2003-004) and the Chinese-English Parallel Corpora (CLDC-LAC-2003-006).
    Page 6, “Experiments”
  3. For testing and developing, we used six Chinese-English development corpora of IWSLT 2008.
    Page 6, “Experiments”
  4. The system was tested using the Chinese-English MT evaluation sets of NIST 2004, NIST 2006 and NIST 2008.
    Page 6, “Experiments”
  5. For development, we used the Chinese-English MT evaluation sets of NIST 2002 and NIST 2005.
    Page 6, “Experiments”
  6. A Chinese-English and an English-Chinese MT system are trained on (C0, E0).
    Page 6, “Experiments”
  7. The Chinese-English experimental results of Goral and Gnews are shown in Table 5 and Table 6, respectively.
    Page 6, “Experiments”
  8. We make a detailed analysis on the Chinese-English translation results that are affected by our paraphrase rules.
    Page 7, “Discussion”
  9. The analysis is carried out on the IWSLT 2007 Chinese-English test set, 84 out of 489 input sentences have been affected by paraphrases, and the statistic of human evaluation is shown in Table 8.
    Page 7, “Discussion”

See all papers in Proc. ACL 2012 that mention Chinese-English.

See all papers in Proc. ACL that mention Chinese-English.

Back to top.

word alignment

Appears in 8 sentences as: Word Alignment (1) word alignment (4) Word Alignments (1) word alignments (2)
In Improve SMT Quality with Automatically Extracted Paraphrase Rules
  1. 3.3 Word Alignments Filtering
    Page 3, “Extraction of Paraphrase Rules”
  2. We can construct word alignment between S0 and S1 through T 0.
    Page 3, “Extraction of Paraphrase Rules”
  3. On the initial corpus of (S0, T 0), we conduct word alignment with Giza++ (Och and Ney, 2000) in both directions and then apply the grow-diag-fmal heuristic (Koehn et al., 2005) for symmetrization.
    Page 3, “Extraction of Paraphrase Rules”
  4. Because S1 is generated by feeding T 0 into the PBMT system SYS_T S, the word alignment between T 0 and S1 can be acquired from the verbose information of the decoder.
    Page 3, “Extraction of Paraphrase Rules”
  5. The word alignments of S0 and S1 contain noises which are produced by either wrong alignment of GIZA++ or translation errors of SYS_T S. To ensure the alignment quality, we use some heuristics to filter the alignment between S0 and S1:
    Page 3, “Extraction of Paraphrase Rules”
  6. >_\<‘flz 3% 4E@ iii/EL o I very feel interest that N/A blue handbag Figure 2: Example for Word Alignment Filtration
    Page 4, “Extraction of Paraphrase Rules”
  7. From the word-aligned sentence pairs, we then extract a set of rules that are consistent with the word alignments .
    Page 4, “Extraction of Paraphrase Rules”
  8. The alignment was obtained using GIZA++ (Och and Ney, 2003) and then we symmetrized the word alignment using the grow-diag-fmal heuristic.
    Page 5, “Experiments”

See all papers in Proc. ACL 2012 that mention word alignment.

See all papers in Proc. ACL that mention word alignment.

Back to top.

sentence pairs

Appears in 6 sentences as: sentence pair (1) Sentence Pairs (1) sentence pairs (4)
In Improve SMT Quality with Automatically Extracted Paraphrase Rules
  1. The aligned sentence pairs in (S0, S1) can be considered as paraphrases.
    Page 3, “Forward-Translation vs. Back-Translation”
  2. 3.2 Selecting Paraphrase Sentence Pairs
    Page 3, “Extraction of Paraphrase Rules”
  3. If the sentence in T 2 has a higher BLEU score than the aligned sentence in T1, the corresponding sentences in S0 and S1 are selected as candidate paraphrase sentence pairs , which are used in the following steps of paraphrase extractions.
    Page 3, “Extraction of Paraphrase Rules”
  4. From the word-aligned sentence pairs , we then extract a set of rules that are consistent with the word alignments.
    Page 4, “Extraction of Paraphrase Rules”
  5. Take the sentence pair in Figure 2 as an example, two initial phrase pairs PP1 = “f R EEE il-EE ||| £3 4‘ EEE il-EE” and PP2 = “W FIE R E@ $l§fi E féfll III E E?
    Page 4, “Extraction of Paraphrase Rules”
  6. After tokenization and filtering, this bilingual corpus contained 319,694 sentence pairs (7.9M tokens on
    Page 6, “Experiments”

See all papers in Proc. ACL 2012 that mention sentence pairs.

See all papers in Proc. ACL that mention sentence pairs.

Back to top.

MT system

Appears in 5 sentences as: MT system (6)
In Improve SMT Quality with Automatically Extracted Paraphrase Rules
  1. The only requirement is that the MT system needs to be bidirectional.
    Page 2, “Forward-Translation vs. Back-Translation”
  2. The procedure includes translating a text into certain foreign language with the MT system (Forward-Translation), and translating it back into the original language with the same system (Back-Translation).
    Page 2, “Forward-Translation vs. Back-Translation”
  3. Two possible reasons may explain this phenomenon: (l) in the first round of translation T 0 9 S1, some target word orders are reserved due to the reordering failure, and these reserved orders lead to a better result in the second round of translation; (2) the text generated by an MT system is more likely to be matched by the reversed but homologous MT system .
    Page 3, “Forward-Translation vs. Back-Translation”
  4. A Chinese-English and an English-Chinese MT system are trained on (C0, E0).
    Page 6, “Experiments”
  5. (2010) captures the structures implicitly by training an MT system on (SO, $1) and “translates” the SMT input to an MT-favored expression.
    Page 8, “Related Work”

See all papers in Proc. ACL 2012 that mention MT system.

See all papers in Proc. ACL that mention MT system.

Back to top.

translation models

Appears in 5 sentences as: translation models (6)
In Improve SMT Quality with Automatically Extracted Paraphrase Rules
  1. The translation quality of the SMT system is highly related to the coverage of translation models .
    Page 1, “Introduction”
  2. Naturally, a solution to the coverage problem is to bridge the gaps between the input sentences and the translation models , either from the input side, which targets on rewriting the input sentences to the MT-favored expressions, or from
    Page 1, “Introduction”
  3. the side of translation models, which tries to enrich the translation models to cover more expressions.
    Page 1, “Introduction”
  4. The proposed methods can be classified into two categories according to the paraphrase targets: (1) enrich translation models to cover more bilingual expressions; (2) paraphrase the input sentences to reduce OOVs or generate multiple inputs.
    Page 1, “Introduction”
  5. (2010) and Max (2010) used paraphrases to smooth translation models .
    Page 1, “Introduction”

See all papers in Proc. ACL 2012 that mention translation models.

See all papers in Proc. ACL that mention translation models.

Back to top.

translation quality

Appears in 5 sentences as: translation quality (5)
In Improve SMT Quality with Automatically Extracted Paraphrase Rules
  1. These rules are employed to enrich the SMT inputs for translation quality improvement.
    Page 1, “Abstract”
  2. The translation quality of the SMT system is highly related to the coverage of translation models.
    Page 1, “Introduction”
  3. Finally the translation quality of Back-Translation is evaluated by using the original source texts as references.
    Page 2, “Forward-Translation vs. Back-Translation”
  4. investigate What kinds of transformation finally lead to the translation quality improvement.
    Page 7, “Discussion”
  5. The manual investigation on oral translation results indicate that the paraphrase rules capture four kinds of MT-favored transformation to ensure translation quality improvement.
    Page 8, “Conclusion”

See all papers in Proc. ACL 2012 that mention translation quality.

See all papers in Proc. ACL that mention translation quality.

Back to top.

language model

Appears in 4 sentences as: language model (3) language models (1)
In Improve SMT Quality with Automatically Extracted Paraphrase Rules
  1. We used SRILM2 for the training of language models (S-gram in all the experiments).
    Page 5, “Experiments”
  2. We trained a Chinese language model for the EC translation on the Chinese part of the bi-text.
    Page 6, “Experiments”
  3. For the English language model of CE translation, an extra corpus named Tanaka was used besides the English part of the bilingual corpora.
    Page 6, “Experiments”
  4. We trained a 5-gram language model on the English side of the bi-text.
    Page 6, “Experiments”

See all papers in Proc. ACL 2012 that mention language model.

See all papers in Proc. ACL that mention language model.

Back to top.

SMT system

Appears in 4 sentences as: SMT system (3) SMT systems (1)
In Improve SMT Quality with Automatically Extracted Paraphrase Rules
  1. The translation quality of the SMT system is highly related to the coverage of translation models.
    Page 1, “Introduction”
  2. This problem is more serious for online SMT systems in real-world applications.
    Page 1, “Introduction”
  3. the input sentences of the SMT system using automatically extracted paraphrase rules which can capture structures on sentence level in addition to paraphrases on the word or phrase level.
    Page 2, “Introduction”
  4. Removing redundant words: Mostly, translating redundant words may confuse the SMT system and would be unnecessary.
    Page 7, “Discussion”

See all papers in Proc. ACL 2012 that mention SMT system.

See all papers in Proc. ACL that mention SMT system.

Back to top.

BLEU score

Appears in 3 sentences as: BLEU score (3)
In Improve SMT Quality with Automatically Extracted Paraphrase Rules
  1. If the sentence in T 2 has a higher BLEU score than the aligned sentence in T1, the corresponding sentences in S0 and S1 are selected as candidate paraphrase sentence pairs, which are used in the following steps of paraphrase extractions.
    Page 3, “Extraction of Paraphrase Rules”
  2. (00,-, 01,-) are selected for the extraction of paraphrase rules if two conditions are satisfied: (1) BLEU(eZi) — BLEU(eli) > 61, and (2) BLEU(eZi) > 62, where BLEU(-) is a function for computing BLEU score ; 61 and 62 are thresholds for balancing the rules number and the quality of paraphrase rules.
    Page 6, “Experiments”
  3. on BLEU score
    Page 7, “Discussion”

See all papers in Proc. ACL 2012 that mention BLEU score.

See all papers in Proc. ACL that mention BLEU score.

Back to top.

NIST

Appears in 3 sentences as: NIST (10)
In Improve SMT Quality with Automatically Extracted Paraphrase Rules
  1. develop NIST 2002 878 10 NIST 2005 1,082 4 NIST 2004 1,788 5 test NIST 2006 1,664 4 NIST 2008 1,357 4
    Page 6, “Experiments”
  2. The system was tested using the Chinese-English MT evaluation sets of NIST 2004, NIST 2006 and NIST 2008.
    Page 6, “Experiments”
  3. For development, we used the Chinese-English MT evaluation sets of NIST 2002 and NIST 2005.
    Page 6, “Experiments”

See all papers in Proc. ACL 2012 that mention NIST.

See all papers in Proc. ACL that mention NIST.

Back to top.

parallel corpus

Appears in 3 sentences as: parallel corpus (3)
In Improve SMT Quality with Automatically Extracted Paraphrase Rules
  1. Without using extra paraphrase resources, we acquire the rules by comparing the source side of the parallel corpus with the target-to-source translations of the target side.
    Page 1, “Abstract”
  2. Note that all the texts of S0, S1, S2, T 0 and T1 are sentence aligned because the initial parallel corpus (S0, T 0) is aligned in the sentence level.
    Page 3, “Forward-Translation vs. Back-Translation”
  3. We train a source-to-target PBMT system (SYS_ST) and a target-to-source PBMT system (SYS_TS) on the parallel corpus .
    Page 3, “Extraction of Paraphrase Rules”

See all papers in Proc. ACL 2012 that mention parallel corpus.

See all papers in Proc. ACL that mention parallel corpus.

Back to top.

significant improvements

Appears in 3 sentences as: significant improvements (3)
In Improve SMT Quality with Automatically Extracted Paraphrase Rules
  1. The experimental results show that our proposed approach achieves significant improvements of l.6~3.6 points of BLEU in the oral domain and 0.5~l points in the news domain.
    Page 1, “Abstract”
  2. The experimental results show that our proposed approach achieves significant improvements of l.6~3.6 points of BLEU in the oral domain and 0.5~l points in the news domain.
    Page 2, “Introduction”
  3. Our system gains significant improvements of 1.6~3.6 points of BLEU in the oral domain, and 0.5~1 points of BLEU in the news domain.
    Page 6, “Experiments”

See all papers in Proc. ACL 2012 that mention significant improvements.

See all papers in Proc. ACL that mention significant improvements.

Back to top.

word order

Appears in 3 sentences as: word order (2) word orders (1)
In Improve SMT Quality with Automatically Extracted Paraphrase Rules
  1. Two possible reasons may explain this phenomenon: (l) in the first round of translation T 0 9 S1, some target word orders are reserved due to the reordering failure, and these reserved orders lead to a better result in the second round of translation; (2) the text generated by an MT system is more likely to be matched by the reversed but homologous MT system.
    Page 3, “Forward-Translation vs. Back-Translation”
  2. Take the first line of Table 9 as an example, the paraphrased sentence “gob/How many fii/cigarettes Elwx/can fiEfiE/duty-free #7 /take 3Z/NULL” is not a fluent Chinese sentence, however, the rearranged word order is closer to English, which finally results in a much better translation.
    Page 8, “Discussion”
  3. (2008) use grammars to paraphrase the source side of training data, covering aspects like word order and minor lexical variations (tenses etc.)
    Page 8, “Related Work”

See all papers in Proc. ACL 2012 that mention word order.

See all papers in Proc. ACL that mention word order.

Back to top.

PDF available from www.aclweb.org.
Send us an email