Concept-to-text Generation via Discriminative Reranking
Konstas, Ioannis and Lapata, Mirella

Article Structure

Abstract

This paper proposes a data-driven method for concept-to-text generation, the task of automatically producing textual output from nonlinguistic input.

Introduction

Concept-to-text generation broadly refers to the task of automatically producing textual output from nonlinguistic input such as databases of records, logical form, and expert system knowledge bases (Reiter and Dale, 2000).

Related Work

Early discriminative approaches to text generation were introduced in spoken dialogue systems, and usually tackled content selection and surface realization separately.

Problem Formulation

We assume our generator takes as input a set of database records (1 and produces text W that verbalizes some of these records.

Experimental Design

In this section we present our experimental setup for assessing the performance of our model.

Results

Table 2 summarizes our results.

Conclusions

We presented a discriminative reranking framework for an end-to-end generation system that performs both content selection and surface realization.

Topics

reranking

Appears in 16 sentences as: rerank (2) reranked (1) Reranking (2) reranking (10) reranks (1)
In Concept-to-text Generation via Discriminative Reranking
  1. The hypergraph structure encodes exponentially many derivations, which we rerank discriminatively using local and global features.
    Page 1, “Abstract”
  2. ia Discriminative Reranking
    Page 1, “Introduction”
  3. The performance of this baseline system could be potentially further improved using discriminative reranking (Collins, 2000).
    Page 1, “Introduction”
  4. Typically, this method first creates a list of n-best candidates from a generative model, and then reranks them with arbitrary features (both local and global) that are either not computable or intractable to compute within the
    Page 1, “Introduction”
  5. An appealing alternative is to rerank the hypergraph directly (Huang, 2008).
    Page 2, “Introduction”
  6. Our contributions in this paper are threefold: we recast concept-to-text generation in a probabilistic parsing framework that allows to jointly optimize content selection and surface realization; we represent parse derivations compactly using hypergraphs and illustrate the use of an algorithm for generating (rather than parsing) in this framework; finally, the application of discriminative reranking to concept-to-text generation is novel to our knowledge and as our experiments show beneficial.
    Page 2, “Introduction”
  7. Discriminative reranking has been employed in many NLP tasks such as syntactic parsing (Char-niak and Johnson, 2005; Huang, 2008), machine translation (Shen et al., 2004; Li and Khudanpur, 2009) and semantic parsing (Ge and Mooney, 2006).
    Page 2, “Related Work”
  8. Our model is closest to Huang (2008) who also performs forest reranking on a hypergraph, using both local and nonlocal features, whose weights are tuned with the averaged perceptron algorithm (Collins, 2002).
    Page 2, “Related Work”
  9. We adapt forest reranking to generation and introduce several task-specific features that boost performance.
    Page 2, “Related Work”
  10. We have a single reranking component that applies
    Page 2, “Related Work”
  11. The hypergraph representation allows us to decompose the feature functions and compute them piecemeal at each hyperarc (or sub-derivation), rather than at the root node as in conventional n-best list reranking .
    Page 3, “Problem Formulation”

See all papers in Proc. ACL 2012 that mention reranking.

See all papers in Proc. ACL that mention reranking.

Back to top.

perceptron

Appears in 6 sentences as: Perceptron (1) perceptron (5)
In Concept-to-text Generation via Discriminative Reranking
  1. Our model is closest to Huang (2008) who also performs forest reranking on a hypergraph, using both local and nonlocal features, whose weights are tuned with the averaged perceptron algorithm (Collins, 2002).
    Page 2, “Related Work”
  2. Algorithm 1: Averaged Structured Perceptron N i=1
    Page 4, “Problem Formulation”
  3. We estimate the weights oc using the averaged structured perceptron algorithm (Collins, 2002), which is well known for its speed and good performance in similar large-parameter NLP tasks (Liang et al., 2006; Huang, 2008).
    Page 4, “Problem Formulation”
  4. As shown in Algorithm l, the perceptron makes several passes over the training scenarios, and in each iteration it computes the best scoring (w,h) among the candidate derivations, given the current weights oc.
    Page 4, “Problem Formulation”
  5. We find the best scoring derivation via forest reranking using both local and nonlocal features, that we train using the perceptron algorithm.
    Page 8, “Conclusions”
  6. Finally, distributed training strategies have been developed for the perceptron algorithm (McDonald et al., 2010), which would allow our generator to scale to even larger datasets.
    Page 8, “Conclusions”

See all papers in Proc. ACL 2012 that mention perceptron.

See all papers in Proc. ACL that mention perceptron.

Back to top.

BLEU

Appears in 4 sentences as: BLEU (4)
In Concept-to-text Generation via Discriminative Reranking
  1. Experimental evaluation on the ATIS domain shows that our model outperforms a competitive discriminative system both using BLEU and in a judgment elicitation study.
    Page 1, “Abstract”
  2. As can be seen, inclusion of lexical features gives our decoder an absolute increase of 6.73% in BLEU over the l-BEST system.
    Page 7, “Results”
  3. System BLEU METEOR l-BEST+BASE+ALIGN 21.93 34.01 k-BEST+BASE+ALIGN+LEX 28.66 45.18 k-BEST+BASE+ALIGN+LEX+STR 30.62 46.07 ANGELI 26.77 42.41
    Page 8, “Results”
  4. over the l-BEST system and 3.85% over ANGELI in terms of BLEU .
    Page 8, “Results”

See all papers in Proc. ACL 2012 that mention BLEU.

See all papers in Proc. ACL that mention BLEU.

Back to top.

gold-standard

Appears in 4 sentences as: gold-standard (4)
In Concept-to-text Generation via Discriminative Reranking
  1. Here, w is the estimated text, W* the gold-standard text, h is the estimated latent configuration of the model and h+ the oracle latent configuration.
    Page 4, “Problem Formulation”
  2. In other NLP tasks such as syntactic parsing, there is a gold-standard parse, that can be used as the oracle.
    Page 5, “Problem Formulation”
  3. They broadly convey similar meaning with the gold-standard ; ANGELI exhibits some long-range repetition, probably due to reiteration of the same record patterns.
    Page 8, “Results”
  4. It is worth noting that both our system and ANGELI produce output that is semantically compatible with but lexically different from the gold-standard (compare please list the flights and show me the flights against give me the flights).
    Page 8, “Results”

See all papers in Proc. ACL 2012 that mention gold-standard.

See all papers in Proc. ACL that mention gold-standard.

Back to top.

structural features

Appears in 4 sentences as: Structural Features (1) structural features (3)
In Concept-to-text Generation via Discriminative Reranking
  1. 2We also store field information to compute structural features , described in Section 4.2.
    Page 5, “Problem Formulation”
  2. Structural Features Features in this category target primarily content selection and influence appropriate choice at the field level:
    Page 6, “Experimental Design”
  3. Addition of the structural features further boosts performance.
    Page 7, “Results”
  4. We tackle this issue with the inclusion of nonlocal structural features .
    Page 8, “Results”

See all papers in Proc. ACL 2012 that mention structural features.

See all papers in Proc. ACL that mention structural features.

Back to top.

baseline system

Appears in 3 sentences as: baseline system (3)
In Concept-to-text Generation via Discriminative Reranking
  1. The performance of this baseline system could be potentially further improved using discriminative reranking (Collins, 2000).
    Page 1, “Introduction”
  2. baseline system .
    Page 2, “Introduction”
  3. 5Since the addition of these features, essentially incurs reranking, it follows that the systems would exhibit the exact same performance as the baseline system with l—best lists.
    Page 7, “Experimental Design”

See all papers in Proc. ACL 2012 that mention baseline system.

See all papers in Proc. ACL that mention baseline system.

Back to top.

bigram

Appears in 3 sentences as: bigram (3)
In Concept-to-text Generation via Discriminative Reranking
  1. The weight of this rule is the bigram probability of two records conditioned on their type, multiplied with a normalization factor 7».
    Page 3, “Problem Formulation”
  2. Rule (6) defines the expansion of field F to a sequence of (binarized) words W, with a weight equal to the bigram probability of the current word given the previous word, the current record, and field.
    Page 4, “Problem Formulation”
  3. Consecutive Word/Bigram/Trigram This feature family targets adjacent repetitions of the same word, bigram or trigram, e.g., ‘show me the show me the
    Page 6, “Experimental Design”

See all papers in Proc. ACL 2012 that mention bigram.

See all papers in Proc. ACL that mention bigram.

Back to top.

language model

Appears in 3 sentences as: language model (3)
In Concept-to-text Generation via Discriminative Reranking
  1. In machine translation, a decoder that implements forest rescoring (Huang and Chiang, 2007) uses the language model as an external criterion of the goodness of sub-translations on account of their grammaticality.
    Page 5, “Problem Formulation”
  2. imately via cube pruning (Chiang, 2007), by integrating a trigram language model extracted from the training set (see Konstas and Lapata (2012) for details).
    Page 6, “Experimental Design”
  3. Lexical Features These features encourage grammatical coherence and inform lexical selection over and above the limited horizon of the language model captured by Rules (6)—(9).
    Page 6, “Experimental Design”

See all papers in Proc. ACL 2012 that mention language model.

See all papers in Proc. ACL that mention language model.

Back to top.

machine translation

Appears in 3 sentences as: machine translation (3)
In Concept-to-text Generation via Discriminative Reranking
  1. Discriminative reranking has been employed in many NLP tasks such as syntactic parsing (Char-niak and Johnson, 2005; Huang, 2008), machine translation (Shen et al., 2004; Li and Khudanpur, 2009) and semantic parsing (Ge and Mooney, 2006).
    Page 2, “Related Work”
  2. In machine translation , a decoder that implements forest rescoring (Huang and Chiang, 2007) uses the language model as an external criterion of the goodness of sub-translations on account of their grammaticality.
    Page 5, “Problem Formulation”
  3. 3In machine translation , Huang (2008) provides a soft algorithm that finds the forest oracle, i.e., the parse among the reranked candidates with the highest Parseval F—score.
    Page 5, “Experimental Design”

See all papers in Proc. ACL 2012 that mention machine translation.

See all papers in Proc. ACL that mention machine translation.

Back to top.

n-grams

Appears in 3 sentences as: n-grams (3)
In Concept-to-text Generation via Discriminative Reranking
  1. Local and nonlocal information (e.g., word n-grams , long-
    Page 2, “Related Work”
  2. Field bigrams/trigrams Analogously to the lexical features mentioned above, we introduce a series of nonlocal features that capture field n-grams , given a specific record.
    Page 6, “Experimental Design”
  3. The l-BEST system has some grammaticality issues, which we avoid by defining features over lexical n-grams and repeated words.
    Page 8, “Results”

See all papers in Proc. ACL 2012 that mention n-grams.

See all papers in Proc. ACL that mention n-grams.

Back to top.

natural language

Appears in 3 sentences as: natural language (3)
In Concept-to-text Generation via Discriminative Reranking
  1. Here, the records provide a structured representation of the flight details (e. g., departure and arrival time, location), and the text renders some of this information in natural language .
    Page 1, “Introduction”
  2. Specifically, we define a probabilistic context-free grammar (PCFG) that captures the structure of the database and its correspondence to natural language .
    Page 1, “Introduction”
  3. and Collins (2007) instead, which combines the utterances of a single user in one scenario and contains 5,426 scenarios in total; each scenario corresponds to a (manually annotated) formal meaning representation (it-expression) and its translation in natural language .
    Page 6, “Experimental Design”

See all papers in Proc. ACL 2012 that mention natural language.

See all papers in Proc. ACL that mention natural language.

Back to top.

PDF available from www.aclweb.org.
Send us an email