Abstraction and Generalisation in Semantic Role Labels: PropBank, VerbNet or both?
Merlo, Paola and van der Plas, Lonneke

Article Structure

Abstract

Semantic role labels are the representation of the grammatically relevant aspects of a sentence meaning.

Introduction

Most current approaches to language analysis assume that the structure of a sentence depends on the lexical semantics of the verb and of other predicates in the sentence.

Materials and Method

In data analysis and inferential statistics, careful preparation of the data and choice of the appropriate statistical measures are key.

Amount of Information in Semantic Roles Inventory

Most proposals of semantic role inventories agree on the fact that the number of roles should be small to be valid generally.

Equivalence Classes of Semantic Roles

An observation that holds for all semantic role labelling schemes is that certain labels seem to be more similar than others, based on their ability to occur in the same syntactic environment and to be expressed by the same function words.

The Combinatorics of Semantic Roles

Semantic roles exhibit paradigmatic generalisations — generalisations across similar semantic roles in the inventory — (which we saw in section 4.)

Topics

semantic role

Appears in 46 sentences as: Semantic Role (1) Semantic role (2) semantic role (28) Semantic Roles (2) Semantic roles (3) semantic roles (16)
In Abstraction and Generalisation in Semantic Role Labels: PropBank, VerbNet or both?
  1. Semantic role labels are the representation of the grammatically relevant aspects of a sentence meaning.
    Page 1, “Abstract”
  2. Capturing the nature and the number of semantic roles in a sentence is therefore fundamental to correctly describing the interface between grammar and meaning.
    Page 1, “Abstract”
  3. In this paper, we compare two annotation schemes, PropBank and VerbNet, in a task-independent, general way, analysing how well they fare in capturing the linguistic generalisations that are known to hold for semantic role labels, and consequently how well they grammaticalise aspects of meaning.
    Page 1, “Abstract”
  4. We show that VerbNet is more verb-specific and better able to generalise to new semantic role instances, while PropBank better captures some of the structural constraints among roles.
    Page 1, “Abstract”
  5. Semantic role labels are the representation of the grammatically relevant aspects of a sentence meaning.
    Page 1, “Introduction”
  6. Capturing the nature and the number of semantic roles in a sentence is therefore fundamental to correctly describe the interface between grammar and meaning, and it is of paramount importance for all natural language processing (NLP) applications that attempt to extract meaning representations from analysed text, such as question-answering systems or even machine translation.
    Page 1, “Introduction”
  7. The role of theories of semantic role lists is to obtain a set of semantic roles that can apply to any argument of any verb, to provide an unambiguous identifier of the grammatical roles of the participants in the event described by the sentence (Dowty, 1991).
    Page 1, “Introduction”
  8. (2007) investigated the ability of the PropBank role inventory to generalise compared to the annotation in another semantic role list, proposed in the electronic dictionary VerbNet.
    Page 2, “Introduction”
  9. (2007) show that augmenting PropB ank labels with VerbNet labels increases generalisation of the less frequent labels, such as ARG2, to new verbs and new domains, they also show that PropBank labels perform better overall, in a semantic role labelling task.
    Page 2, “Introduction”
  10. First, the argument labels for which the VerbNet improvement was found are infrequent, and might therefore not have influenced the overall results enough to counterbalance new errors introduced by the finer-grained annotation scheme; second, the learning methods in both these experimental settings are largely based on syntactic information, thereby confounding learning and generalisation due to syntax — which would favour the more syntactically-driven PropBank annotation — with learning due to greater generality of the semantic role annotation; finally, task-specific learning-based experiments do not guarantee that the learners be sufficiently powerful to make use of the full generality of the semantic role labels.
    Page 2, “Introduction”
  11. In this paper, we compare the two annotation schemes, analysing how well they fare in capturing the linguistic generalisations that are known to hold for semantic role labels, and consequently how well they grammaticalise aspects of mean-
    Page 2, “Introduction”

See all papers in Proc. ACL 2009 that mention semantic role.

See all papers in Proc. ACL that mention semantic role.

Back to top.

role labels

Appears in 22 sentences as: role label (4) role labelling (7) role labels (12)
In Abstraction and Generalisation in Semantic Role Labels: PropBank, VerbNet or both?
  1. Semantic role labels are the representation of the grammatically relevant aspects of a sentence meaning.
    Page 1, “Abstract”
  2. In this paper, we compare two annotation schemes, PropBank and VerbNet, in a task-independent, general way, analysing how well they fare in capturing the linguistic generalisations that are known to hold for semantic role labels , and consequently how well they grammaticalise aspects of meaning.
    Page 1, “Abstract”
  3. Semantic role labels are the representation of the grammatically relevant aspects of a sentence meaning.
    Page 1, “Introduction”
  4. The annotated PropBank corpus, and therefore implicitly its role labels inventory, has been largely adopted in NLP because of its exhaustiveness and because it is coupled with syntactic annotation, properties that make it very attractive for the automatic leam-ing of these roles and their further applications to NLP tasks.
    Page 1, “Introduction”
  5. (2007) show that augmenting PropB ank labels with VerbNet labels increases generalisation of the less frequent labels, such as ARG2, to new verbs and new domains, they also show that PropBank labels perform better overall, in a semantic role labelling task.
    Page 2, “Introduction”
  6. (2008) find that PropBank role labels are more robust than VerbNet labels in predicting new verb usages, unseen verbs, and they port better to new domains.
    Page 2, “Introduction”
  7. First, the argument labels for which the VerbNet improvement was found are infrequent, and might therefore not have influenced the overall results enough to counterbalance new errors introduced by the finer-grained annotation scheme; second, the learning methods in both these experimental settings are largely based on syntactic information, thereby confounding learning and generalisation due to syntax — which would favour the more syntactically-driven PropBank annotation — with learning due to greater generality of the semantic role annotation; finally, task-specific learning-based experiments do not guarantee that the learners be sufficiently powerful to make use of the full generality of the semantic role labels .
    Page 2, “Introduction”
  8. In this paper, we compare the two annotation schemes, analysing how well they fare in capturing the linguistic generalisations that are known to hold for semantic role labels , and consequently how well they grammaticalise aspects of mean-
    Page 2, “Introduction”
  9. Because the well-attested strong correlation between syntactic structure and semantic role labels (Levin and Rappaport Hovav, 2005; Merlo and Stevenson, 2001) could intervene as a confounding factor in this analysis, we expressly limit our investigation to data analyses and statistical measures that do not exploit syntactic properties or parsing techniques.
    Page 2, “Introduction”
  10. In section 6, we show that PropBank more closely captures the thematic hierarchy and is more correlated to grammatical functions, hence potentially more useful for semantic role labelling , for leam-ers whose features are based on the syntactic tree.
    Page 2, “Introduction”
  11. Verbal predicates in the Penn Treebank (PTB) receive a label REL and their arguments are annotated with abstract semantic role labels A0-A5 or AA for those complements of the predicative verb that are considered arguments, while those complements of the verb labelled with a semantic functional label in the original PTB receive the composite semantic role label AM-X, where X stands for labels such as LOC, TMP or ADV, for locative, temporal and adverbial modifiers respectively.
    Page 3, “Materials and Method”

See all papers in Proc. ACL 2009 that mention role labels.

See all papers in Proc. ACL that mention role labels.

Back to top.

semantic role labels

Appears in 17 sentences as: semantic role label (2) semantic role labelling (6) Semantic role labels (2) semantic role labels (8)
In Abstraction and Generalisation in Semantic Role Labels: PropBank, VerbNet or both?
  1. Semantic role labels are the representation of the grammatically relevant aspects of a sentence meaning.
    Page 1, “Abstract”
  2. In this paper, we compare two annotation schemes, PropBank and VerbNet, in a task-independent, general way, analysing how well they fare in capturing the linguistic generalisations that are known to hold for semantic role labels , and consequently how well they grammaticalise aspects of meaning.
    Page 1, “Abstract”
  3. Semantic role labels are the representation of the grammatically relevant aspects of a sentence meaning.
    Page 1, “Introduction”
  4. (2007) show that augmenting PropB ank labels with VerbNet labels increases generalisation of the less frequent labels, such as ARG2, to new verbs and new domains, they also show that PropBank labels perform better overall, in a semantic role labelling task.
    Page 2, “Introduction”
  5. First, the argument labels for which the VerbNet improvement was found are infrequent, and might therefore not have influenced the overall results enough to counterbalance new errors introduced by the finer-grained annotation scheme; second, the learning methods in both these experimental settings are largely based on syntactic information, thereby confounding learning and generalisation due to syntax — which would favour the more syntactically-driven PropBank annotation — with learning due to greater generality of the semantic role annotation; finally, task-specific learning-based experiments do not guarantee that the learners be sufficiently powerful to make use of the full generality of the semantic role labels .
    Page 2, “Introduction”
  6. In this paper, we compare the two annotation schemes, analysing how well they fare in capturing the linguistic generalisations that are known to hold for semantic role labels , and consequently how well they grammaticalise aspects of mean-
    Page 2, “Introduction”
  7. Because the well-attested strong correlation between syntactic structure and semantic role labels (Levin and Rappaport Hovav, 2005; Merlo and Stevenson, 2001) could intervene as a confounding factor in this analysis, we expressly limit our investigation to data analyses and statistical measures that do not exploit syntactic properties or parsing techniques.
    Page 2, “Introduction”
  8. In section 6, we show that PropBank more closely captures the thematic hierarchy and is more correlated to grammatical functions, hence potentially more useful for semantic role labelling , for leam-ers whose features are based on the syntactic tree.
    Page 2, “Introduction”
  9. Verbal predicates in the Penn Treebank (PTB) receive a label REL and their arguments are annotated with abstract semantic role labels A0-A5 or AA for those complements of the predicative verb that are considered arguments, while those complements of the verb labelled with a semantic functional label in the original PTB receive the composite semantic role label AM-X, where X stands for labels such as LOC, TMP or ADV, for locative, temporal and adverbial modifiers respectively.
    Page 3, “Materials and Method”
  10. An observation that holds for all semantic role labelling schemes is that certain labels seem to be more similar than others, based on their ability to occur in the same syntactic environment and to be expressed by the same function words.
    Page 5, “Equivalence Classes of Semantic Roles”
  11. However, most annotation schemes in NLP and linguistics assume that semantic role labels are atomic.
    Page 5, “Equivalence Classes of Semantic Roles”

See all papers in Proc. ACL 2009 that mention semantic role labels.

See all papers in Proc. ACL that mention semantic role labels.

Back to top.

error rate

Appears in 3 sentences as: Error rate (1) error rate (2)
In Abstraction and Generalisation in Semantic Role Labels: PropBank, VerbNet or both?
  1. Table 2: Percent Error rate reduction (ERR) across role labelling sets in three tasks in Zapirain et al.
    Page 4, “Amount of Information in Semantic Roles Inventory”
  2. (2008) and calculate the reduction in error rate based on this differential baseline for the two annotation schemes.
    Page 4, “Amount of Information in Semantic Roles Inventory”
  3. VerbNet has better role generalising ability overall as its reduction in error rate is greater than PropBank (first line of Table 2), but it is more degraded by lack of verb information (second and third lines of Table 2).
    Page 5, “Amount of Information in Semantic Roles Inventory”

See all papers in Proc. ACL 2009 that mention error rate.

See all papers in Proc. ACL that mention error rate.

Back to top.

Penn Treebank

Appears in 3 sentences as: Penn Treebank (3)
In Abstraction and Generalisation in Semantic Role Labels: PropBank, VerbNet or both?
  1. Proposition Bank (Palmer et al., 2005) adds Levin’s style predicate-argument annotation and indication of verbs’ alternations to the syntactic structures of the Penn Treebank (Marcus et al.,
    Page 2, “Materials and Method”
  2. Verbal predicates in the Penn Treebank (PTB) receive a label REL and their arguments are annotated with abstract semantic role labels A0-A5 or AA for those complements of the predicative verb that are considered arguments, while those complements of the verb labelled with a semantic functional label in the original PTB receive the composite semantic role label AM-X, where X stands for labels such as LOC, TMP or ADV, for locative, temporal and adverbial modifiers respectively.
    Page 3, “Materials and Method”
  3. SemLink1 provides mappings from PropB ank to VerbNet for the WSJ portion of the Penn Treebank .
    Page 3, “Materials and Method”

See all papers in Proc. ACL 2009 that mention Penn Treebank.

See all papers in Proc. ACL that mention Penn Treebank.

Back to top.

Treebank

Appears in 3 sentences as: Treebank (3)
In Abstraction and Generalisation in Semantic Role Labels: PropBank, VerbNet or both?
  1. Proposition Bank (Palmer et al., 2005) adds Levin’s style predicate-argument annotation and indication of verbs’ alternations to the syntactic structures of the Penn Treebank (Marcus et al.,
    Page 2, “Materials and Method”
  2. Verbal predicates in the Penn Treebank (PTB) receive a label REL and their arguments are annotated with abstract semantic role labels A0-A5 or AA for those complements of the predicative verb that are considered arguments, while those complements of the verb labelled with a semantic functional label in the original PTB receive the composite semantic role label AM-X, where X stands for labels such as LOC, TMP or ADV, for locative, temporal and adverbial modifiers respectively.
    Page 3, “Materials and Method”
  3. SemLink1 provides mappings from PropB ank to VerbNet for the WSJ portion of the Penn Treebank .
    Page 3, “Materials and Method”

See all papers in Proc. ACL 2009 that mention Treebank.

See all papers in Proc. ACL that mention Treebank.

Back to top.

PDF available from www.aclweb.org.
Send us an email