Incremental Joint Extraction of Entity Mentions and Relations
Li, Qi and Ji, Heng

Article Structure

Abstract

We present an incremental joint framework to simultaneously extract entity mentions and relations using structured perceptron with efficient beam-search.

Introduction

The goal of end-to-end entity mention and relation extraction is to discover relational structures of entity mentions from unstructured texts.

Background

2.1 Task Definition

Algorithm 3.1 The Model

Our goal is to predict the hidden structure of each sentence based on arbitrary features and constraints.

Features

An advantage of our framework is that we can easily exploit arbitrary features across the two tasks.

Experiments

5.1 Data and Scoring Metric

Related Work

Entity mention extraction (e.g., (Florian et al., 2004; Florian et al., 2006; Florian et al., 2010; Zitouni and Florian, 2008; Ohta et al., 2012)) and relation extraction (e.g., (Reichartz et al., 2009; Sun et al., 2011; Jiang and Zhai, 2007; Bunescu and Mooney, 2005; Zhao and Grishman, 2005; Culotta and Sorensen, 2004; Zhou et al., 2007; Qian and Zhou, 2010; Qian et al., 2008; Chan and Roth, 2011; Plank and Moschitti, 2013)) have drawn much attention in recent years but were

Conclusions and Future Work

In this paper we introduced a new architecture for more powerful end-to-end entity mention and relation extraction.

Topics

entity mentions

Appears in 64 sentences as: Entity Mention (3) Entity mention (1) entity mention (28) Entity Mentions (1) entity mentions (34) entity mention’s (1)
In Incremental Joint Extraction of Entity Mentions and Relations
  1. We present an incremental joint framework to simultaneously extract entity mentions and relations using structured perceptron with efficient beam-search.
    Page 1, “Abstract”
  2. In addition, by virtue of the inexact search, we developed a number of new and effective global features as soft constraints to capture the interdependency among entity mentions and relations.
    Page 1, “Abstract”
  3. The goal of end-to-end entity mention and relation extraction is to discover relational structures of entity mentions from unstructured texts.
    Page 1, “Introduction”
  4. This problem has been artificially broken down into several components such as entity mention boundary identification, entity type classification and relation extraction.
    Page 1, “Introduction”
  5. For example, in Figure 1, the output structure of each sentence can be interpreted as a graph in which entity mentions are nodes and relations are directed arcs with relation types.
    Page 1, “Introduction”
  6. Various entity mentions and relations share linguistic and logical constraints.
    Page 1, “Introduction”
  7. For example, we can use the triangle feature in Figure lb to ensure that the relations between “forces”, and each of the entity mentions “Somalia/GPE”, “Haiti/GPE” and “Kosovo/GpE”, are of the same type (Physical (P HYS), in this case).
    Page 1, “Introduction”
  8. Following the above intuitions, we introduce a joint framework based on structured perceptron (Collins, 2002; Collins and Roark, 2004) with beam-search to extract entity mentions and relations simultaneously.
    Page 1, “Introduction”
  9. Most previous attempts on joint inference of entity mentions and relations (such as (Roth and Yih, 2004; Roth and Yih, 2007)) assumed that entity mention boundaries were given, and the classifiers of mentions and relations are separately learned.
    Page 2, “Introduction”
  10. As a key difference, we incrementally extract entity mentions together with relations using a single model.
    Page 2, “Introduction”
  11. This is the first work to incrementally predict entity mentions and relations using a single joint model (Section 3).
    Page 2, “Introduction”

See all papers in Proc. ACL 2014 that mention entity mentions.

See all papers in Proc. ACL that mention entity mentions.

Back to top.

entity type

Appears in 17 sentences as: Entity Type (1) Entity type (2) entity type (7) entity types (7)
In Incremental Joint Extraction of Entity Mentions and Relations
  1. This problem has been artificially broken down into several components such as entity mention boundary identification, entity type classification and relation extraction.
    Page 1, “Introduction”
  2. ACE defined 7 main entity types including Person (PER), Organization (ORG),
    Page 2, “Background”
  3. First, the algorithm enumerates all possible segments (i.e., subse-quences) of ac ending at the current token with various entity types .
    Page 4, “Algorithm 3.1 The Model”
  4. :c-axis and y-axis represent the input sentence and entity types , respectively.
    Page 4, “Algorithm 3.1 The Model”
  5. The rectangles denote segments with entity types , among which the shaded ones are three competing hypotheses ending at “1,400”.
    Page 4, “Algorithm 3.1 The Model”
  6. 3.3 Entity Type Constraints
    Page 5, “Algorithm 3.1 The Model”
  7. Entity type constraints have been shown effective in predicting relations (Roth and Yih, 2007; Chan and Roth, 2010).
    Page 5, “Algorithm 3.1 The Model”
  8. We automatically collect a mapping table of permissible entity types for each relation type from our training data.
    Page 5, “Algorithm 3.1 The Model”
  9. The entity segments of 3) can be expressed as a list of triples (61, ..., em), where each segment 6, = (ui, 2),, 75,-) is a triple of start index ui, end index 21,-, and entity type 25,-.
    Page 5, “Features”
  10. Gazetteer features Entity type of each segment based on matching a number of gazetteers including persons, countries, cities and organizations.
    Page 5, “Features”
  11. Coreference consistency Coreferential entity mentions should be assigned the same entity type .
    Page 6, “Features”

See all papers in Proc. ACL 2014 that mention entity type.

See all papers in Proc. ACL that mention entity type.

Back to top.

relation extraction

Appears in 16 sentences as: Relation Extraction (1) relation extraction (17)
In Incremental Joint Extraction of Entity Mentions and Relations
  1. The goal of end-to-end entity mention and relation extraction is to discover relational structures of entity mentions from unstructured texts.
    Page 1, “Introduction”
  2. This problem has been artificially broken down into several components such as entity mention boundary identification, entity type classification and relation extraction .
    Page 1, “Introduction”
  3. The entity mention extraction and relation extraction tasks we are addressing are those of the Automatic Content Extraction (ACE) program2.
    Page 2, “Background”
  4. Most previous research on relation extraction assumed that entity mentions were given In this work we aim to address the problem of end-to-end entity mention and relation extraction from raw texts.
    Page 2, “Background”
  5. In order to develop a baseline system representing state-of-the-art pipelined approaches, we trained a linear-chain Conditional Random Fields model (Lafferty et al., 2001) for entity mention extraction and a Maximum Entropy model for relation extraction .
    Page 2, “Background”
  6. Relation Extraction Model Given a sentence with entity mention annotations, the goal of baseline relation extraction is to classify each mention pair into one of the predefined relation types with direction or J_ (non-relation).
    Page 3, “Background”
  7. Most of our relation extraction features are based on the previous work of (Zhou et al., 2005) and (Kambhatla, 2004).
    Page 3, “Background”
  8. Most previous work on ACE relation extraction has reported results on ACE’04 data set.
    Page 6, “Experiments”
  9. We use the standard F1 measure to evaluate the performance of entity mention extraction and relation extraction .
    Page 7, “Experiments”
  10. Furthermore, we combine these two criteria to evaluate the performance of end-to-end entity mention and relation extraction .
    Page 7, “Experiments”
  11. The human F1 score on end-to-end relation extraction is only about 70%, which indicates it is a very challenging task.
    Page 8, “Experiments”

See all papers in Proc. ACL 2014 that mention relation extraction.

See all papers in Proc. ACL that mention relation extraction.

Back to top.

perceptron

Appears in 9 sentences as: Perceptron (1) perceptron (9)
In Incremental Joint Extraction of Entity Mentions and Relations
  1. We present an incremental joint framework to simultaneously extract entity mentions and relations using structured perceptron with efficient beam-search.
    Page 1, “Abstract”
  2. Following the above intuitions, we introduce a joint framework based on structured perceptron (Collins, 2002; Collins and Roark, 2004) with beam-search to extract entity mentions and relations simultaneously.
    Page 1, “Introduction”
  3. Our previous work (Li et al., 2013) used perceptron model with token-based tagging to jointly extract event triggers and arguments.
    Page 1, “Introduction”
  4. To estimate the feature weights, we use structured perceptron (Collins, 2002), an extension of the standard perceptron for structured prediction, as the learning framework.
    Page 5, “Algorithm 3.1 The Model”
  5. (2012) proved the convergency of structured perceptron when inexact search is applied with violation-fixing update methods such as early-update (Collins and Roark, 2004).
    Page 5, “Algorithm 3.1 The Model”
  6. Figure 4 shows the pseudocode for structured perceptron training with early-update.
    Page 5, “Algorithm 3.1 The Model”
  7. Figure 4: Perceptron algorithm with beam-search and early-update.
    Page 5, “Algorithm 3.1 The Model”
  8. Our previous work (Li et al., 2013) used structured perceptron with token-based decoder to jointly predict event triggers and arguments based on the assumption that entity mentions and other argument candidates are given as part of the input.
    Page 9, “Related Work”
  9. For the first time, we addressed this challenging task by an incremental beam-search algorithm in conjunction with structured perceptron .
    Page 9, “Conclusions and Future Work”

See all papers in Proc. ACL 2014 that mention perceptron.

See all papers in Proc. ACL that mention perceptron.

Back to top.

end-to-end

Appears in 8 sentences as: end-to-end (8)
In Incremental Joint Extraction of Entity Mentions and Relations
  1. Experiments on Automatic Content Extraction (ACE)1 corpora demonstrate that our joint model significantly outperforms a strong pipelined baseline, which attains better performance than the best-reported end-to-end system.
    Page 1, “Abstract”
  2. The goal of end-to-end entity mention and relation extraction is to discover relational structures of entity mentions from unstructured texts.
    Page 1, “Introduction”
  3. Most previous research on relation extraction assumed that entity mentions were given In this work we aim to address the problem of end-to-end entity mention and relation extraction from raw texts.
    Page 2, “Background”
  4. Furthermore, we combine these two criteria to evaluate the performance of end-to-end entity mention and relation extraction.
    Page 7, “Experiments”
  5. The human F1 score on end-to-end relation extraction is only about 70%, which indicates it is a very challenging task.
    Page 8, “Experiments”
  6. For end-to-end entity mention and relation extraction, both the joint approach and the pipelined baseline outperform the best results reported by (Chan and Roth, 2011) under the same setting.
    Page 8, “Experiments”
  7. We extended the similar idea to our end-to-end task by incrementally predicting relations along with entity mention segments.
    Page 9, “Related Work”
  8. In this paper we introduced a new architecture for more powerful end-to-end entity mention and relation extraction.
    Page 9, “Conclusions and Future Work”

See all papers in Proc. ACL 2014 that mention end-to-end.

See all papers in Proc. ACL that mention end-to-end.

Back to top.

joint model

Appears in 8 sentences as: joint model (8)
In Incremental Joint Extraction of Entity Mentions and Relations
  1. Experiments on Automatic Content Extraction (ACE)1 corpora demonstrate that our joint model significantly outperforms a strong pipelined baseline, which attains better performance than the best-reported end-to-end system.
    Page 1, “Abstract”
  2. This is the first work to incrementally predict entity mentions and relations using a single joint model (Section 3).
    Page 2, “Introduction”
  3. We compare our proposed method (Joint w/ Global) with the pipelined system (Pipeline), the joint model with only local features (Joint w/ Local), and two human annotators who annotated 73 documents in ACE’OS corpus.
    Page 7, “Experiments”
  4. Our joint model correctly identified the entity mentions and their relation.
    Page 7, “Experiments”
  5. Figure 7 shows the details when the joint model is applied to this sentence.
    Page 7, “Experiments”
  6. For entity mention extraction, our joint model achieved 79.7% on 5-fold cross-validation, which is comparable with the best F1 score 79.2% reported by (Florian et al., 2006) on single-fold.
    Page 8, “Experiments”
  7. Since these gazetteers, additional data sets and external IE models are all not publicly available, it is not fair to directly compare our joint model with their results.
    Page 8, “Experiments”
  8. In addition, we aim to incorporate other IE components such as event extraction into the joint model .
    Page 9, “Conclusions and Future Work”

See all papers in Proc. ACL 2014 that mention joint model.

See all papers in Proc. ACL that mention joint model.

Back to top.

coreference

Appears in 5 sentences as: Coreference (1) coreference (3) Coreferential (1) coreferential (1)
In Incremental Joint Extraction of Entity Mentions and Relations
  1. 3Throughout this paper we refer to relation mention as relation since we do not consider relation mention coreference .
    Page 2, “Background”
  2. Coreference consistency Coreferential entity mentions should be assigned the same entity type.
    Page 6, “Features”
  3. We determine high-recall coreference links between two segments in the same sentence using some simple heuristic rules:
    Page 6, “Features”
  4. Then we encode a global feature to check whether two coreferential segments share the same entity type.
    Page 6, “Features”
  5. Roth (2011), we excluded the D I SC relation type, and removed relations in the system output which are implicitly correct via coreference links for fair comparison.
    Page 7, “Experiments”

See all papers in Proc. ACL 2014 that mention coreference.

See all papers in Proc. ACL that mention coreference.

Back to top.

soft constraints

Appears in 4 sentences as: soft constraints (4)
In Incremental Joint Extraction of Entity Mentions and Relations
  1. In addition, by virtue of the inexact search, we developed a number of new and effective global features as soft constraints to capture the interdependency among entity mentions and relations.
    Page 1, “Abstract”
  2. We design a set of novel global features based on soft constraints over the entire output graph structure with low cost (Section 4).
    Page 2, “Introduction”
  3. Relation arcs can also share inter-dependencies or obey soft constraints .
    Page 6, “Features”
  4. As a key difference, our approach jointly extracts entity mentions and relations using a single model, in which arbitrary soft constraints can be easily incorporated.
    Page 9, “Related Work”

See all papers in Proc. ACL 2014 that mention soft constraints.

See all papers in Proc. ACL that mention soft constraints.

Back to top.

beam size

Appears in 3 sentences as: beam size (3)
In Incremental Joint Extraction of Entity Mentions and Relations
  1. k: beam size .
    Page 4, “Algorithm 3.1 The Model”
  2. In general a larger beam size can yield better performance but increase training and decoding time.
    Page 7, “Experiments”
  3. As a tradeoff, we set the beam size as 8 throughout the experiments.
    Page 7, “Experiments”

See all papers in Proc. ACL 2014 that mention beam size.

See all papers in Proc. ACL that mention beam size.

Back to top.

F1 score

Appears in 3 sentences as: F1 score (3)
In Incremental Joint Extraction of Entity Mentions and Relations
  1. The human F1 score on end-to-end relation extraction is only about 70%, which indicates it is a very challenging task.
    Page 8, “Experiments”
  2. Furthermore, the F1 score of the inter-annotator agreement is 51.9%, which is only 2.4% above that of our proposed method.
    Page 8, “Experiments”
  3. For entity mention extraction, our joint model achieved 79.7% on 5-fold cross-validation, which is comparable with the best F1 score 79.2% reported by (Florian et al., 2006) on single-fold.
    Page 8, “Experiments”

See all papers in Proc. ACL 2014 that mention F1 score.

See all papers in Proc. ACL that mention F1 score.

Back to top.

gold-standard

Appears in 3 sentences as: gold-standard (3)
In Incremental Joint Extraction of Entity Mentions and Relations
  1. It is worth noting that this can only happen if the gold-standard has a segment ending at the current token.
    Page 5, “Algorithm 3.1 The Model”
  2. y’ is the prefix of the gold-standard and z is the top assignment.
    Page 5, “Algorithm 3.1 The Model”
  3. In addition, (Singh et al., 2013) used gold-standard mention boundaries.
    Page 9, “Related Work”

See all papers in Proc. ACL 2014 that mention gold-standard.

See all papers in Proc. ACL that mention gold-standard.

Back to top.

significantly outperforms

Appears in 3 sentences as: significantly outperformed (1) significantly outperforms (2)
In Incremental Joint Extraction of Entity Mentions and Relations
  1. Experiments on Automatic Content Extraction (ACE)1 corpora demonstrate that our joint model significantly outperforms a strong pipelined baseline, which attains better performance than the best-reported end-to-end system.
    Page 1, “Abstract”
  2. We can see that our approach significantly outperforms the pipelined approach for both tasks.
    Page 7, “Experiments”
  3. Experiments demonstrated our approach significantly outperformed pipelined approaches for both tasks and dramatically advanced state-of-the-art.
    Page 9, “Conclusions and Future Work”

See all papers in Proc. ACL 2014 that mention significantly outperforms.

See all papers in Proc. ACL that mention significantly outperforms.

Back to top.

PDF available from www.aclweb.org.
Send us an email