Crosslingual Induction of Semantic Roles
Article Structure
Abstract
We argue that multilingual parallel data provides a valuable source of indirect supervision for induction of shallow semantic representations.
Introduction
Learning in the context of multiple languages simultaneously has been shown to be beneficial to a number of NLP tasks from morphological analysis to syntactic parsing (Kuhn, 2004; Snyder and Barzilay, 2010; McDonald et al., 2011).
Problem Definition
As we mentioned in the introduction, in this work we focus on the labeling stage of semantic role labeling.
Monolingual Model
In this section we describe one of the Bayesian models for semantic role induction proposed in (Titov and Klementiev, 2012).
Multilingual Extension
As we argued in Section 1, our goal is to penalize for disagreement in semantic structures predicted for each language on parallel data.
Inference
An inference algorithm for an unsupervised model should be efficient enough to handle vast amounts of unlabeled data, as it can easily be obtained and is likely to improve results.
Empirical Evaluation
We begin by describing the data and evaluation metrics we use before discussing results.
Related Work
Unsupervised learning in crosslingual setting has been an active area of research in recent years.
Conclusions
This work adds unsupervised semantic role labeling to the list of NLP tasks benefiting from the crosslingual induction setting.
Topics
semantic role
Appears in 22 sentences as: Semantic role (1) semantic role (14) semantic roles (7)
In Crosslingual Induction of Semantic Roles
- Specifically, we consider unsupervised induction of semantic roles from sentences annotated with automatically-predicted syntactic dependency representations and use a state-of-the-art generative Bayesian nonparametric model.Page 1, “Abstract”
- Semantic role labeling (SRL) (Gildea and Juraf-sky, 2002) involves predicting predicate argument structure, i.e.Page 1, “Introduction”
- and their assignment to underlying semantic roles .Page 1, “Introduction”
- Though syntactic representations are often predictive of semantic roles (Levin, 1993), the interface between syntactic and semantic representations is far from trivial.Page 1, “Introduction”
- For example, in our sentences (a) and (b) representing so-called blame alternation (Levin, 1993), the same information is conveyed in two different ways and a successful model of semantic role labeling needs to learn the corresponding linkings from the data.Page 2, “Introduction”
- Section 2 begins with a definition of the crosslingual semantic role induction task we address in this paper.Page 2, “Introduction”
- As we mentioned in the introduction, in this work we focus on the labeling stage of semantic role labeling.Page 2, “Problem Definition”
- In the labeling stage, semantic roles are represented by clusters of arguments, and labeling a particular argument corresponds to deciding on its role cluster.Page 2, “Problem Definition”
- In sum, we treat the unsupervised semantic role labeling task as clustering of argument keys.Page 3, “Problem Definition”
- Thus, argument occurrences in the corpus whose keys are clustered together are assigned the same semantic role .Page 3, “Problem Definition”
- In this section we describe one of the Bayesian models for semantic role induction proposed in (Titov and Klementiev, 2012).Page 3, “Monolingual Model”
See all papers in Proc. ACL 2012 that mention semantic role.
See all papers in Proc. ACL that mention semantic role.
Back to top.
CoNLL
Appears in 11 sentences as: CONLL (1) CoNLL (10)
In Crosslingual Induction of Semantic Roles
- Our model admits efficient inference: the estimation time on CoNLL 2009 data (Hajic et al., 2009) and Europarl v.6 bitext (Koehn, 2005) does not exceed 5 hours on a single processor and the inference algorithm is highly parallelizable, reducing in-Page 2, “Introduction”
- (Koehn, 2005) and the CoNLL 2009 distributions of the Penn Treebank WSJ corpus (Marcus et al., 1993) for English and the SALSA corpus (Burchardt et al., 2006) for German.Page 6, “Empirical Evaluation”
- As standard for unsupervised SRL, we use the entire CoNLL training sets for evaluation, and use held-out sets for model selection and parameter tuning.Page 6, “Empirical Evaluation”
- Although the CoNLL 2009 dataset already has predicted dependency structures, we could not reproduce them so that we could use the same parser to annotate Europarl.Page 6, “Empirical Evaluation”
- Accuracy of argument identification on CoNLL 2009 using predicted syntactic analyses was 80.7% and 86.5% for English and German, respectively.Page 6, “Empirical Evaluation”
- Domains of Europarl (parliamentary proceedings) and German/English CoNLL data (newswire) are substantially different.Page 7, “Empirical Evaluation”
- Since the influence of domain shift is not the focus of work, we try to minimize its effect by computing the likelihood part of the objective on CoNLL data alone.Page 7, “Empirical Evaluation”
- Since we do not have access to the systems, we compare on the marginally different English CoNLL 2008 (Surdeanu et al.,Page 7, “Empirical Evaluation”
- Table 1: Argument clustering performance with gold argument identification and gold syntactic parses on CoNLL 2008 shared-task dataset.Page 7, “Empirical Evaluation”
- Following (Lang and Lapata, 2010), we allocate a cluster for each of 20 most frequent relations in the CoNLL dataset and one cluster for all other relations.Page 7, “Empirical Evaluation”
- Table 2: Results on CONLL 2009 with automatic argument identification and automatic syntactic parses.Page 8, “Empirical Evaluation”
See all papers in Proc. ACL 2012 that mention CoNLL.
See all papers in Proc. ACL that mention CoNLL.
Back to top.
parallel data
Appears in 8 sentences as: parallel data (7) parallel data: (1)
In Crosslingual Induction of Semantic Roles
- We argue that multilingual parallel data provides a valuable source of indirect supervision for induction of shallow semantic representations.Page 1, “Abstract”
- When applied to German—English parallel data , our method obtains a substantial improvement over a model trained without using the agreement signal, when both are tested on nonparallel sentences.Page 1, “Abstract”
- The goal of this work is to show that parallel data is useful in unsupervised induction of shallow semantic representations.Page 1, “Introduction”
- As we argued in Section 1, our goal is to penalize for disagreement in semantic structures predicted for each language on parallel data .Page 4, “Multilingual Extension”
- Intuitively, when two arguments are aligned in parallel data , we expect them to be labeled with the same semantic role in both languages.Page 4, “Multilingual Extension”
- Specifically, we augment the joint probability with a penalty term computed on parallel data:Page 5, “Multilingual Extension”
- This objective corresponds to the classic maximum weighted bipartite matching problem with the weight for each edge 19(1) and 19(2) equal to the number of times the two predicates were aligned in parallel data .Page 5, “Multilingual Extension”
- We show that an agreement signal extracted from parallel data provides indirect supervision capable of substantially improving a state-of-the-art model for semantic role induction.Page 8, “Conclusions”
See all papers in Proc. ACL 2012 that mention parallel data.
See all papers in Proc. ACL that mention parallel data.
Back to top.
semantic representations
Appears in 8 sentences as: semantic representation (1) semantic representations (7)
In Crosslingual Induction of Semantic Roles
- We argue that multilingual parallel data provides a valuable source of indirect supervision for induction of shallow semantic representations .Page 1, “Abstract”
- The goal of this work is to show that parallel data is useful in unsupervised induction of shallow semantic representations .Page 1, “Introduction”
- Though syntactic representations are often predictive of semantic roles (Levin, 1993), the interface between syntactic and semantic representations is far from trivial.Page 1, “Introduction”
- 5This has been explored before for shallow semantic representations (Lang and Lapata, 2011a; Titov and Klementiev, 201 1).Page 5, “Inference”
- However, most of this research has focused on induction of syntactic structures (Kuhn, 2004; Snyder et al., 2009) or morphologic analysis (Snyder and Barzilay, 2008) and we are not aware of any previous work on induction of semantic representations in the crosslingual setting.Page 8, “Related Work”
- Learning of semantic representations in the context of monolingual weakly-parallel data was studied in Titov and Kozhevnikov (2010) but their setting was semi-supervised and they experimented only on a restricted domain.Page 8, “Related Work”
- Semi-supervised and weakly-supervised techniques have also been explored for other types of semantic representations but these studies again have mostly focused on restricted domains (Kate and Mooney, 2007; Liang et al., 2009; Goldwasser et al., 2011; Liang et al., 2011).Page 8, “Related Work”
- Although in this work we focused primarily on improving performance for each individual language, crosslingual semantic representation could be extracted by a simple postprocessing step.Page 8, “Conclusions”
See all papers in Proc. ACL 2012 that mention semantic representations.
See all papers in Proc. ACL that mention semantic representations.
Back to top.
role labeling
Appears in 5 sentences as: role labeling (5)
In Crosslingual Induction of Semantic Roles
- Semantic role labeling (SRL) (Gildea and Juraf-sky, 2002) involves predicting predicate argument structure, i.e.Page 1, “Introduction”
- For example, in our sentences (a) and (b) representing so-called blame alternation (Levin, 1993), the same information is conveyed in two different ways and a successful model of semantic role labeling needs to learn the corresponding linkings from the data.Page 2, “Introduction”
- As we mentioned in the introduction, in this work we focus on the labeling stage of semantic role labeling .Page 2, “Problem Definition”
- In sum, we treat the unsupervised semantic role labeling task as clustering of argument keys.Page 3, “Problem Definition”
- This work adds unsupervised semantic role labeling to the list of NLP tasks benefiting from the crosslingual induction setting.Page 8, “Conclusions”
See all papers in Proc. ACL 2012 that mention role labeling.
See all papers in Proc. ACL that mention role labeling.
Back to top.
semantic role labeling
Appears in 5 sentences as: Semantic role labeling (1) semantic role labeling (4)
In Crosslingual Induction of Semantic Roles
- Semantic role labeling (SRL) (Gildea and Juraf-sky, 2002) involves predicting predicate argument structure, i.e.Page 1, “Introduction”
- For example, in our sentences (a) and (b) representing so-called blame alternation (Levin, 1993), the same information is conveyed in two different ways and a successful model of semantic role labeling needs to learn the corresponding linkings from the data.Page 2, “Introduction”
- As we mentioned in the introduction, in this work we focus on the labeling stage of semantic role labeling .Page 2, “Problem Definition”
- In sum, we treat the unsupervised semantic role labeling task as clustering of argument keys.Page 3, “Problem Definition”
- This work adds unsupervised semantic role labeling to the list of NLP tasks benefiting from the crosslingual induction setting.Page 8, “Conclusions”
See all papers in Proc. ACL 2012 that mention semantic role labeling.
See all papers in Proc. ACL that mention semantic role labeling.
Back to top.
clusterings
Appears in 4 sentences as: clusterings (4)
In Crosslingual Induction of Semantic Roles
- For predicates present in both sides of a bitext, we guide models in both languages to prefer clusterings which maximize agreement between predicate argument structures predicted for each aligned predicate pair.Page 2, “Introduction”
- The objective of this work is to improve argument key clusterings by inducing them simultaneously in two languages.Page 3, “Problem Definition”
- Now, when parameters and argument key clusterings are chosen, we can summarize the remainder of the generative story as follows.Page 4, “Monolingual Model”
- For each pair of predicates, we search for clusterings to maximize the sum of the log-probability and the negated penalty term.Page 6, “Inference”
See all papers in Proc. ACL 2012 that mention clusterings.
See all papers in Proc. ACL that mention clusterings.
Back to top.
syntactic parses
Appears in 4 sentences as: syntactic parses (3) syntactic parsing (1)
In Crosslingual Induction of Semantic Roles
- Learning in the context of multiple languages simultaneously has been shown to be beneficial to a number of NLP tasks from morphological analysis to syntactic parsing (Kuhn, 2004; Snyder and Barzilay, 2010; McDonald et al., 2011).Page 1, “Introduction”
- Table 1: Argument clustering performance with gold argument identification and gold syntactic parses on CoNLL 2008 shared-task dataset.Page 7, “Empirical Evaluation”
- We report the results using gold argument identification and gold syntactic parses in order to focus the evaluation on the argument labeling stage and to minimize the noise due to automatic syntactic annotations.Page 7, “Empirical Evaluation”
- Table 2: Results on CONLL 2009 with automatic argument identification and automatic syntactic parses .Page 8, “Empirical Evaluation”
See all papers in Proc. ACL 2012 that mention syntactic parses.
See all papers in Proc. ACL that mention syntactic parses.
Back to top.
unlabeled data
Appears in 4 sentences as: unlabeled data (5)
In Crosslingual Induction of Semantic Roles
- This suggests that the models scale to much larger corpora, which is an important property for a successful unsupervised learning method, as unlabeled data is abundant.Page 2, “Introduction”
- An inference algorithm for an unsupervised model should be efficient enough to handle vast amounts of unlabeled data , as it can easily be obtained and is likely to improve results.Page 5, “Inference”
- Most of the SRL research has focused on the supervised setting, however, lack of annotated resources for most languages and insufficient coverage provided by the existing resources motivates the need for using unlabeled data or other forms of weak supervision.Page 8, “Related Work”
- This includes methods based on graph alignment between labeled and unlabeled data (Furstenau and Lapata, 2009), using unlabeled data to improve lexical generalization (Deschacht and Moens, 2009), and projection of annotation across languages (Pado and Lapata, 2009; van der Plas et al., 2011).Page 8, “Related Work”
See all papers in Proc. ACL 2012 that mention unlabeled data.
See all papers in Proc. ACL that mention unlabeled data.
Back to top.
word alignments
Appears in 4 sentences as: word alignment (2) word alignments (3)
In Crosslingual Induction of Semantic Roles
- In doing so, as in much of previous work on unsupervised induction of linguistic structures, we rely on automatically produced word alignments .Page 4, “Multilingual Extension”
- In Section 6, we describe how we use word alignment to decide if two arguments are aligned; for now, we assume that (noisy) argument alignments are given.Page 4, “Multilingual Extension”
- We use GIZA++ (Och and Ney, 2003) to produce word alignments in Europarl: we ran it in both directions and kept the intersection of the induced word alignments .Page 6, “Empirical Evaluation”
- We mark arguments in two languages as aligned if there is any word alignment between the corresponding sets and if they are arguments of aligned predicates.Page 6, “Empirical Evaluation”
See all papers in Proc. ACL 2012 that mention word alignments.
See all papers in Proc. ACL that mention word alignments.
Back to top.
F1 scores
Appears in 3 sentences as: F1 score (1) F1 scores (2)
In Crosslingual Induction of Semantic Roles
- We compute the aggregate PU, CO, and F1 scores over all predicates in the same way as (Lang and Lapata, 2011a) by weighting the scores of each predicate by the number of its argument occurrences.Page 7, “Empirical Evaluation”
- Our models are robust to parameter settings; the parameters were tuned (to an order of magnitude) to optimize the F1 score on the held-out development set and were as follows.Page 7, “Empirical Evaluation”
- Boldface is used to highlight the best F1 scores .Page 7, “Empirical Evaluation”
See all papers in Proc. ACL 2012 that mention F1 scores.
See all papers in Proc. ACL that mention F1 scores.
Back to top.
