A Syntax-Free Approach to Japanese Sentence Compression
Hirao, Tsutomu and Suzuki, Jun and Isozaki, Hideki

Article Structure

Abstract

Conventional sentence compression methods employ a syntactic parser to compress a sentence without changing its meaning.

Introduction

In order to compress a sentence while retaining its original meaning, the subject-predicate relationship of the original sentence should be preserved after compression.

Analysis of reference compressions

Syntactic information does not always yield improved compression performance because humans usually ignore the syntactic structures when they compress sentences.

A Syntax Free Sequence-oriented Sentence Compression Method

As an alternative to syntactic parsing, we propose two novel features, intra-sentence positional term weighting (IPTW) and the patched language model (PLM) for our syntax-free sentence compressor.

Experimental Evaluation

4.1 Corpus and Evaluation Measures

Results and Discussion

5.1 Results: automatic evaluation

Related work

Conventional sentence compression methods employ the tree trimming approach to compress a sentence without changing its meaning.

Conclusions

We proposed a syntax free sequence-oriented Japanese sentence compression method with two novel features: IPTW and PLM.

Topics

sentence compression

Appears in 25 sentences as: Sentence Compression (2) sentence compression (21) sentence compressor (2)
In A Syntax-Free Approach to Japanese Sentence Compression
  1. Conventional sentence compression methods employ a syntactic parser to compress a sentence without changing its meaning.
    Page 1, “Abstract”
  2. Moreover, for the goal of on-demand sentence compression , the time spent in the parsing stage is not negligible.
    Page 1, “Abstract”
  3. In accordance with this idea, conventional sentence compression methods employ syntactic parsers.
    Page 1, “Introduction”
  4. Moreover, on-demand sentence compression is made problematic by the time spent in the parsing stage.
    Page 1, “Introduction”
  5. This paper proposes a syntax-free sequence-oriented sentence compression method.
    Page 1, “Introduction”
  6. This statistic supports the view that sentence compression that strongly depends on syntax is not useful in reproducing reference compressions.
    Page 2, “Analysis of reference compressions”
  7. We need a sentence compression method that can drop intermediate nodes in the syntactic tree aggressively beyond the tree-scoped boundary.
    Page 2, “Analysis of reference compressions”
  8. In addition, sentence compression methods that strongly depend on syntactic parsers have two problems: ‘parse error’ and ‘decoding speed.’ 44% of sentences output by a state-of-the-art Japanese dependency parser contain at least one error (Kudo and Matsumoto, 2005).
    Page 2, “Analysis of reference compressions”
  9. This critically degrades the overall performance of sentence compression .
    Page 2, “Analysis of reference compressions”
  10. As an alternative to syntactic parsing, we propose two novel features, intra-sentence positional term weighting (IPTW) and the patched language model (PLM) for our syntax-free sentence compressor .
    Page 3, “A Syntax Free Sequence-oriented Sentence Compression Method”
  11. 3.1 Sentence Compression as a Combinatorial Optimization Problem
    Page 3, “A Syntax Free Sequence-oriented Sentence Compression Method”

See all papers in Proc. ACL 2009 that mention sentence compression.

See all papers in Proc. ACL that mention sentence compression.

Back to top.

language model

Appears in 11 sentences as: Language Model (2) language model (9)
In A Syntax-Free Approach to Japanese Sentence Compression
  1. As an alternative to syntactic parsing, we propose a novel term weighting technique based on the positional information within the original sentence and a novel language model that combines statistics from the original sentence and a general corpus.
    Page 1, “Abstract”
  2. To maintain the subject-predicate relationship in the compressed sentence and retain fluency without using syntactic parsers, we propose two novel features: intra-sentence positional term weighting (IPTW) and the patched language model (PLM).
    Page 1, “Introduction”
  3. PLM is a form of summarization-oriented fluency statistics derived from the original sentence and the general language model .
    Page 1, “Introduction”
  4. As an alternative to syntactic parsing, we propose two novel features, intra-sentence positional term weighting (IPTW) and the patched language model (PLM) for our syntax-free sentence compressor.
    Page 3, “A Syntax Free Sequence-oriented Sentence Compression Method”
  5. 3.2.2 Patched Language Model
    Page 4, “A Syntax Free Sequence-oriented Sentence Compression Method”
  6. Many studies on sentence compression employ the n-gram language model to evaluate the linguistic likelihood of a compressed sentence.
    Page 4, “A Syntax Free Sequence-oriented Sentence Compression Method”
  7. PLM stands for Patched Language Model .
    Page 4, “A Syntax Free Sequence-oriented Sentence Compression Method”
  8. We developed the n-gram language model from a 9 year set of Mainichi Newspaper articles.
    Page 5, “Experimental Evaluation”
  9. Replacing PLM with the bigram language model (w/o PLM) degrades the performance significantly.
    Page 6, “Results and Discussion”
  10. This result shows that the n-gram language model is improper for sentence compression because the n-gram probability is computed by using a corpus that includes both short and long sentences.
    Page 6, “Results and Discussion”
  11. 0 As an alternative to the syntactic parser, we proposed two novel features, Intra-sentence positional term weighting (IPTW) and the Patched language model (PLM), and showed their effectiveness by conducting automatic and human evaluations,
    Page 7, “Conclusions”

See all papers in Proc. ACL 2009 that mention language model.

See all papers in Proc. ACL that mention language model.

Back to top.

syntactic parsers

Appears in 11 sentences as: syntactic parser (3) syntactic parsers (6) syntactic parsing (2)
In A Syntax-Free Approach to Japanese Sentence Compression
  1. Conventional sentence compression methods employ a syntactic parser to compress a sentence without changing its meaning.
    Page 1, “Abstract”
  2. As an alternative to syntactic parsing , we propose a novel term weighting technique based on the positional information within the original sentence and a novel language model that combines statistics from the original sentence and a general corpus.
    Page 1, “Abstract”
  3. Because our method does not use a syntactic parser , it is 4.3 times faster than Hori’s method.
    Page 1, “Abstract”
  4. In accordance with this idea, conventional sentence compression methods employ syntactic parsers .
    Page 1, “Introduction”
  5. To maintain the subject-predicate relationship in the compressed sentence and retain fluency without using syntactic parsers , we propose two novel features: intra-sentence positional term weighting (IPTW) and the patched language model (PLM).
    Page 1, “Introduction”
  6. superior to conventional sequence-oriented methods that employ syntactic parsers while being about 4.3 times faster.
    Page 2, “Introduction”
  7. In addition, sentence compression methods that strongly depend on syntactic parsers have two problems: ‘parse error’ and ‘decoding speed.’ 44% of sentences output by a state-of-the-art Japanese dependency parser contain at least one error (Kudo and Matsumoto, 2005).
    Page 2, “Analysis of reference compressions”
  8. As an alternative to syntactic parsing , we propose two novel features, intra-sentence positional term weighting (IPTW) and the patched language model (PLM) for our syntax-free sentence compressor.
    Page 3, “A Syntax Free Sequence-oriented Sentence Compression Method”
  9. Moreover, their use of syntactic parsers seriously degrades the decoding speed.
    Page 7, “Related work”
  10. It is significantly superior to the methods that employ syntactic parsers .
    Page 7, “Conclusions”
  11. 0 As an alternative to the syntactic parser , we proposed two novel features, Intra-sentence positional term weighting (IPTW) and the Patched language model (PLM), and showed their effectiveness by conducting automatic and human evaluations,
    Page 7, “Conclusions”

See all papers in Proc. ACL 2009 that mention syntactic parsers.

See all papers in Proc. ACL that mention syntactic parsers.

Back to top.

bigram

Appears in 9 sentences as: Bigram (2) bigram (5) bigrams (4)
In A Syntax-Free Approach to Japanese Sentence Compression
  1. 1 if 1(yj) = (yj—1)+ 1 APLM Bigram (w71(yj)71(yj—1)) (5) otherwise
    Page 4, “A Syntax Free Sequence-oriented Sentence Compression Method”
  2. Here, 0 g APLM g l, Bigram(-) indicates word bigram probability.
    Page 4, “A Syntax Free Sequence-oriented Sentence Compression Method”
  3. The first line of equation (5) agrees with Jing’s observation on sentence alignment tasks (Jing and McKeown, 1999); that is, most (or almost all) bigrams in a compressed sentence appear in the original sentence as they are.
    Page 4, “A Syntax Free Sequence-oriented Sentence Compression Method”
  4. 3.2.3 POS bigram
    Page 4, “A Syntax Free Sequence-oriented Sentence Compression Method”
  5. Since POS bigrams are useful for rejecting ungrammatical sentences, we adopt them as follows:
    Page 4, “A Syntax Free Sequence-oriented Sentence Compression Method”
  6. For example, label ‘w/o IPTW + Dep’ employs IDF term weighting as function and word bigram, part-of-speech bigram and dependency probability between words as function in equation (1).
    Page 5, “Experimental Evaluation”
  7. Replacing PLM with the bigram language model (w/o PLM) degrades the performance significantly.
    Page 6, “Results and Discussion”
  8. Most bigrams in a compressed sentence followed those in the source sentence.
    Page 6, “Results and Discussion”
  9. PLM is similar to dependency probability in that both features emphasize word pairs that occurred as bigrams in the source sentence.
    Page 6, “Results and Discussion”

See all papers in Proc. ACL 2009 that mention bigram.

See all papers in Proc. ACL that mention bigram.

Back to top.

dependency trees

Appears in 8 sentences as: dependency tree (3) dependency trees (5)
In A Syntax-Free Approach to Japanese Sentence Compression
  1. For Japanese, dependency trees are trimmed instead of full parse trees (Takeuchi and Matsumoto, 2001; Oguro et al., 2002; Nomoto, 2008)1 This parsing approach is reasonable because the compressed output is grammatical if the
    Page 1, “Introduction”
  2. It treats a sentence as a sequence of words and structural information, such as a syntactic or dependency tree , is encoded in the sequence as features.
    Page 1, “Introduction”
  3. However, they still rely on syntactic information derived from fully parsed syntactic or dependency trees .
    Page 1, “Introduction”
  4. Human usually compress sentences by dropping the intermediate nodes in the dependency tree .
    Page 2, “Analysis of reference compressions”
  5. For Japanese sentences, instead of using full parse trees, existing sentence compression methods trim dependency trees by the discrim-inative model (Takeuchi and Matsumoto, 2001; Nomoto, 2008) through the use of simple linear combined features (Oguro et a1., 2002).
    Page 7, “Related work”
  6. They simply regard a sentence as a word sequence and structural information, such as full parse tree or dependency trees , are encoded in the sequence as features.
    Page 7, “Related work”
  7. However, they still rely on syntactic information derived from full parsed trees or dependency trees .
    Page 7, “Related work”
  8. 0 We revealed that in compressing Japanese sentences, humans usually ignore syntactic structures; they drop intermediate nodes of the dependency tree and drop words within bansetsa,
    Page 7, “Conclusions”

See all papers in Proc. ACL 2009 that mention dependency trees.

See all papers in Proc. ACL that mention dependency trees.

Back to top.

BLEU

Appears in 6 sentences as: BLEU (6)
In A Syntax-Free Approach to Japanese Sentence Compression
  1. For MCE learning, we selected the reference compression that maximize the BLEU score (Pap-ineni et al., 2002) (2 argmaxreRBLEUO‘, R\7“)) from the set of reference compressions and used it as correct data for training.
    Page 5, “Experimental Evaluation”
  2. For automatic evaluation, we employed BLEU (Papineni et al., 2002) by following (Unno et al., 2006).
    Page 5, “Experimental Evaluation”
  3. Label BLEU Proposed .679 w/o PLM .617 w/o IPTW .635 Hori— .493
    Page 5, “Experimental Evaluation”
  4. Our method achieved the highest BLEU score.
    Page 5, “Results and Discussion”
  5. For example, ‘w/o PLM + Dep’ achieved the second highest BLEU score.
    Page 6, “Results and Discussion”
  6. Compared to ‘Hori—’, ‘Hori’ achieved a significantly higher BLEU score.
    Page 6, “Results and Discussion”

See all papers in Proc. ACL 2009 that mention BLEU.

See all papers in Proc. ACL that mention BLEU.

Back to top.

parse trees

Appears in 6 sentences as: parse tree (1) parse trees (4) parsed trees (1)
In A Syntax-Free Approach to Japanese Sentence Compression
  1. English sentences are usually analyzed by a full parser to make parse trees , and the trees are then trimmed (Knight and Marcu, 2002; Turner and Chamiak, 2005; Unno et al., 2006).
    Page 1, “Introduction”
  2. For Japanese, dependency trees are trimmed instead of full parse trees (Takeuchi and Matsumoto, 2001; Oguro et al., 2002; Nomoto, 2008)1 This parsing approach is reasonable because the compressed output is grammatical if the
    Page 1, “Introduction”
  3. For instance, most English sentence compression methods make full parse trees and trim them by applying the generative model (Knight and Marcu, 2002; Turner and Charniak, 2005), discrimina-tive model (Knight and Marcu, 2002; Unno et a1., 2006).
    Page 7, “Related work”
  4. For Japanese sentences, instead of using full parse trees , existing sentence compression methods trim dependency trees by the discrim-inative model (Takeuchi and Matsumoto, 2001; Nomoto, 2008) through the use of simple linear combined features (Oguro et a1., 2002).
    Page 7, “Related work”
  5. They simply regard a sentence as a word sequence and structural information, such as full parse tree or dependency trees, are encoded in the sequence as features.
    Page 7, “Related work”
  6. However, they still rely on syntactic information derived from full parsed trees or dependency trees.
    Page 7, “Related work”

See all papers in Proc. ACL 2009 that mention parse trees.

See all papers in Proc. ACL that mention parse trees.

Back to top.

n-gram

Appears in 5 sentences as: N-gram (1) n-gram (5)
In A Syntax-Free Approach to Japanese Sentence Compression
  1. Many studies on sentence compression employ the n-gram language model to evaluate the linguistic likelihood of a compressed sentence.
    Page 4, “A Syntax Free Sequence-oriented Sentence Compression Method”
  2. N-gram distribution of short sentences may different from that of long sentences.
    Page 4, “A Syntax Free Sequence-oriented Sentence Compression Method”
  3. Therefore, the n-gram probability sometimes disagrees with our intuition in terms of sentence compression.
    Page 4, “A Syntax Free Sequence-oriented Sentence Compression Method”
  4. We developed the n-gram language model from a 9 year set of Mainichi Newspaper articles.
    Page 5, “Experimental Evaluation”
  5. This result shows that the n-gram language model is improper for sentence compression because the n-gram probability is computed by using a corpus that includes both short and long sentences.
    Page 6, “Results and Discussion”

See all papers in Proc. ACL 2009 that mention n-gram.

See all papers in Proc. ACL that mention n-gram.

Back to top.

BLEU score

Appears in 4 sentences as: BLEU score (4)
In A Syntax-Free Approach to Japanese Sentence Compression
  1. For MCE learning, we selected the reference compression that maximize the BLEU score (Pap-ineni et al., 2002) (2 argmaxreRBLEUO‘, R\7“)) from the set of reference compressions and used it as correct data for training.
    Page 5, “Experimental Evaluation”
  2. Our method achieved the highest BLEU score .
    Page 5, “Results and Discussion”
  3. For example, ‘w/o PLM + Dep’ achieved the second highest BLEU score .
    Page 6, “Results and Discussion”
  4. Compared to ‘Hori—’, ‘Hori’ achieved a significantly higher BLEU score .
    Page 6, “Results and Discussion”

See all papers in Proc. ACL 2009 that mention BLEU score.

See all papers in Proc. ACL that mention BLEU score.

Back to top.

dependency parser

Appears in 4 sentences as: dependency parser (4)
In A Syntax-Free Approach to Japanese Sentence Compression
  1. In addition, sentence compression methods that strongly depend on syntactic parsers have two problems: ‘parse error’ and ‘decoding speed.’ 44% of sentences output by a state-of-the-art Japanese dependency parser contain at least one error (Kudo and Matsumoto, 2005).
    Page 2, “Analysis of reference compressions”
  2. sion of Hori’s method which does not require the dependency parser .
    Page 5, “Experimental Evaluation”
  3. Our method was about 4.3 times faster than Hori’s method due to the latter’s use of dependency parser .
    Page 7, “Results and Discussion”
  4. 0 We showed that our method is about 4.3 times faster than Hori’s method which employs a dependency parser .
    Page 7, “Conclusions”

See all papers in Proc. ACL 2009 that mention dependency parser.

See all papers in Proc. ACL that mention dependency parser.

Back to top.

dependency relation

Appears in 4 sentences as: dependency relation (2) dependency relations (2)
In A Syntax-Free Approach to Japanese Sentence Compression
  1. Figure 1: An example of the dependency relation variant.
    Page 2, “Introduction”
  2. The solid arrows indicate dependency relations between words2.
    Page 2, “Analysis of reference compressions”
  3. We observe that the dependency relations are changed by compression; humans create compound nouns using the components derived from different portions of the original sentence without regard to syntactic constraints.
    Page 2, “Analysis of reference compressions”
  4. 2Generally, a dependency relation is defined between bun-setsu.
    Page 2, “Analysis of reference compressions”

See all papers in Proc. ACL 2009 that mention dependency relation.

See all papers in Proc. ACL that mention dependency relation.

Back to top.

PDF available from www.aclweb.org.
Send us an email