Unsupervised Argument Identification for Semantic Role Labeling
Abend, Omri and Reichart, Roi and Rappoport, Ari

Article Structure

Abstract

The task of Semantic Role Labeling (SRL) is often divided into two subtasks: verb argument identification, and argument classification.

Introduction

Semantic Role Labeling (SRL) is a major NLP task, providing a shallow sentence-level semantic analysis.

Related Work

The advance of machine learning based approaches in this field owes to the usage of large scale annotated corpora.

Algorithm

In this section we describe our algorithm.

Experimental Setup

Corpora.

Results

Our results are shown in Table l. The left section presents results on English and the right section presents results on Spanish.

Conclusion

In this work we presented the first algorithm for argument identification that uses neither supervised syntactic annotation nor SRL tagged data.

Topics

POS tags

Appears in 14 sentences as: POS tag (4) POS tagged (1) POS tagging (4) POS tags (6)
In Unsupervised Argument Identification for Semantic Role Labeling
  1. In this paper we present an unsupervised algorithm for identifying verb arguments, where the only type of annotation required is POS tagging .
    Page 1, “Abstract”
  2. A standard SRL algorithm requires thousands to dozens of thousands sentences annotated with POS tags , syntactic annotation and SRL annotation.
    Page 1, “Introduction”
  3. This parser is unique in that it is able to induce a bracketing (unlabeled parsing) from raw text (without even using POS tags ) achieving state-of-the-art results.
    Page 4, “Algorithm”
  4. The only type of supervised annotation we use is POS tagging .
    Page 4, “Algorithm”
  5. We use the taggers MX-POST (Ratnaparkhi, 1996) for English and Tree-Tagger (Schmid, 1994) for Spanish, to obtain POS tags for our model.
    Page 4, “Algorithm”
  6. Figure 1: An example of an unlabeled POS tagged parse tree.
    Page 4, “Algorithm”
  7. by an “IN” POS tag indicating that it is a subordinating conjunction.
    Page 5, “Algorithm”
  8. 1If while traversing the tree, we encounter an ancestor whose first word is preceded by a coordinating conjunction (marked by the POS tag “CC”), we refrain from performing any additional counter updates.
    Page 5, “Algorithm”
  9. We collect collocation statistics from a large corpus, which we annotate with parse trees and POS tags .
    Page 5, “Algorithm”
  10. We use two representations of an argument: one is the POS tag sequence of the terminals contained in the argument, the other is its head word4.
    Page 5, “Algorithm”
  11. The predicate is represented as the conjunction of its lemma with its POS tag .
    Page 5, “Algorithm”

See all papers in Proc. ACL 2009 that mention POS tags.

See all papers in Proc. ACL that mention POS tags.

Back to top.

F-score

Appears in 12 sentences as: F-score (11) F-score” (2)
In Unsupervised Argument Identification for Semantic Role Labeling
  1. We use two measures to evaluate the performance of our algorithm, precision and F-score .
    Page 2, “Introduction”
  2. Precision reflects the algorithm’s applicability for creating training data to be used by supervised SRL models, while the standard SRL F-score measures the model’s performance when used by itself.
    Page 2, “Introduction”
  3. The first stage of our algorithm is shown to outperform a strong baseline both in terms of F-score and of precision.
    Page 2, “Introduction”
  4. Better performance is achieved on the classification, where state-of-the-art supervised approaches achieve about 81% F-score on the in-domain identification task, of which about 95% are later labeled correctly (Marquez et al., 2008).
    Page 2, “Related Work”
  5. We report an F-score as well (the harmonic mean of precision and recall).
    Page 6, “Experimental Setup”
  6. We use the standard parsing F-score evaluation measure.
    Page 7, “Experimental Setup”
  7. In the “Collocation Maximum F-score” the collocation parameters were generally tuned such that the maximum possible F-score for the collocation algorithm is achieved.
    Page 7, “Results”
  8. The best or close to best F-score is achieved when using the clause detection algorithm alone (59.14% for English, 23.34% for Spanish).
    Page 7, “Results”
  9. Note that for both English and Spanish F-score improvements are achieved via a precision improvement that is more significant than the recall degradation.
    Page 7, “Results”
  10. F-score maximization would be the aim of a system that uses the output of our unsupervised ARGID by itself.
    Page 7, “Results”
  11. The “Collocation Maximum F-score” is an example of a model that provides a precision improvement (over both the
    Page 7, “Results”

See all papers in Proc. ACL 2009 that mention F-score.

See all papers in Proc. ACL that mention F-score.

Back to top.

parse tree

Appears in 11 sentences as: parse tree (10) parse trees (1)
In Unsupervised Argument Identification for Semantic Role Labeling
  1. Initially, the set of possible arguments for a given verb consists of all the constituents in the parse tree that do not contain that predicate.
    Page 2, “Introduction”
  2. Using this information, it further reduces the possible arguments only to those contained in the minimal clause, and further prunes them according to their position in the parse tree .
    Page 2, “Introduction”
  3. In addition, most models assume that a syntactic representation of the sentence is given, commonly in the form of a parse tree , a dependency structure or a shallow parse.
    Page 2, “Related Work”
  4. As preprocessing, we use an unsupervised parser that generates an unlabeled parse tree for each sen-
    Page 3, “Algorithm”
  5. Second, they should be k-th degree cousins of the predicate in the parse tree .
    Page 4, “Algorithm”
  6. Our algorithm attempts to find sub-trees within the parse tree , whose structure resembles the structure of a full sentence.
    Page 4, “Algorithm”
  7. Figure 1: An example of an unlabeled POS tagged parse tree .
    Page 4, “Algorithm”
  8. We collect collocation statistics from a large corpus, which we annotate with parse trees and POS tags.
    Page 5, “Algorithm”
  9. Recall that for each sentence, the arguments are a subset of the constituents in the parse tree .
    Page 5, “Algorithm”
  10. A minimal clause is the lowest ancestor of the verb in the parse tree that has a syntactic label of a clause according to the gold standard parse of the PTB.
    Page 7, “Experimental Setup”
  11. In our algorithm, the initial set of potential arguments consists of constituents in the Seginer parser’s parse tree .
    Page 7, “Results”

See all papers in Proc. ACL 2009 that mention parse tree.

See all papers in Proc. ACL that mention parse tree.

Back to top.

gold standard

Appears in 6 sentences as: gold standard (8)
In Unsupervised Argument Identification for Semantic Role Labeling
  1. We tested this on our development data — section 24 of the WSJ PTB, where we saw that 86% of the arguments that are also constituents (in the gold standard parse) were indeed contained in that minimal clause (as defined by the tree label types in the gold standard parse that denote a clause, e.g., S, SBAR).
    Page 4, “Algorithm”
  2. A match is considered each time an argument in the gold standard data matches a marked argument in our model’s output.
    Page 6, “Experimental Setup”
  3. An unmatched argument is an argument which appears in the gold standard data, and fails to appear in our model’s output, and an excessive argument is an argument which appears in our model’s output but does not appear in the gold standard .
    Page 6, “Experimental Setup”
  4. As a gold standard in this evaluation, we mark for each of the verbs in our development data the minimal clause containing it.
    Page 7, “Experimental Setup”
  5. A minimal clause is the lowest ancestor of the verb in the parse tree that has a syntactic label of a clause according to the gold standard parse of the PTB.
    Page 7, “Experimental Setup”
  6. A verb is any terminal marked by one of the POS tags of type verb according to the gold standard POS tags of the PTB.
    Page 7, “Experimental Setup”

See all papers in Proc. ACL 2009 that mention gold standard.

See all papers in Proc. ACL that mention gold standard.

Back to top.

Semantic Role

Appears in 4 sentences as: Semantic Role (2) semantic roles (2)
In Unsupervised Argument Identification for Semantic Role Labeling
  1. The task of Semantic Role Labeling (SRL) is often divided into two subtasks: verb argument identification, and argument classification.
    Page 1, “Abstract”
  2. Semantic Role Labeling (SRL) is a major NLP task, providing a shallow sentence-level semantic analysis.
    Page 1, “Introduction”
  3. They then use the unlabeled argument structure (without the semantic roles ) as training data for the ARGID stage and the entire data (perhaps with other features) for the classification stage.
    Page 2, “Related Work”
  4. They then use these as seed for a bootstrapping algorithm, which consequently identifies the verb arguments in the corpus and assigns their semantic roles .
    Page 3, “Related Work”

See all papers in Proc. ACL 2009 that mention Semantic Role.

See all papers in Proc. ACL that mention Semantic Role.

Back to top.

syntactic parser

Appears in 4 sentences as: syntactic parser (3) syntactic parsers (1)
In Unsupervised Argument Identification for Semantic Role Labeling
  1. The algorithm makes use of a fully unsupervised syntactic parser , using its output in order to detect clauses and gather candidate argument collocation statistics.
    Page 1, “Abstract”
  2. Using VerbNet along with the output of a rule-based chunker (in 2004) and a supervised syntactic parser (in 2005), they spot instances in the corpus that are very similar to the syntactic patterns listed in VerbNet.
    Page 3, “Related Work”
  3. Clause information has been applied to accelerating a syntactic parser (Glaysher and Moldovan, 2006).
    Page 3, “Related Work”
  4. The recent availability of unsupervised syntactic parsers has offered an opportunity to conduct research on SRL, without reliance on supervised syntactic annotation.
    Page 8, “Conclusion”

See all papers in Proc. ACL 2009 that mention syntactic parser.

See all papers in Proc. ACL that mention syntactic parser.

Back to top.

CoNLL

Appears in 3 sentences as: CoNLL (3)
In Unsupervised Argument Identification for Semantic Role Labeling
  1. PB is a standard corpus for SRL evaluation and was used in the CoNLL SRL shared tasks of 2004 (Carreras and Marquez, 2004) and 2005 (Carreras and Marquez, 2005).
    Page 2, “Related Work”
  2. The CoNLL shared tasks of 2004 and 2005 were devoted to SRL, and studied the influence of different syntactic annotations and domain changes on SRL results.
    Page 2, “Related Work”
  3. Supervised clause detection was also tackled as a separate task, notably in the CoNLL 2001 shared task (Tjong Kim Sang and Dejean, 2001).
    Page 3, “Related Work”

See all papers in Proc. ACL 2009 that mention CoNLL.

See all papers in Proc. ACL that mention CoNLL.

Back to top.

shared tasks

Appears in 3 sentences as: shared task (1) shared tasks (2)
In Unsupervised Argument Identification for Semantic Role Labeling
  1. PB is a standard corpus for SRL evaluation and was used in the CoNLL SRL shared tasks of 2004 (Carreras and Marquez, 2004) and 2005 (Carreras and Marquez, 2005).
    Page 2, “Related Work”
  2. The CoNLL shared tasks of 2004 and 2005 were devoted to SRL, and studied the influence of different syntactic annotations and domain changes on SRL results.
    Page 2, “Related Work”
  3. Supervised clause detection was also tackled as a separate task, notably in the CoNLL 2001 shared task (Tjong Kim Sang and Dejean, 2001).
    Page 3, “Related Work”

See all papers in Proc. ACL 2009 that mention shared tasks.

See all papers in Proc. ACL that mention shared tasks.

Back to top.

PDF available from www.aclweb.org.
Send us an email