Using Document Level Cross-Event Inference to Improve Event Extraction
Liao, Shasha and Grishman, Ralph

Article Structure

Abstract

Event extraction is a particularly challenging type of information extraction (IE).

Introduction

The goal of event extraction is to identify instances of a class of events in text.

Task Description

Automatic Content Extraction (ACE) defines an event as a specific occurrence involving participantsl, and it annotates 8 types and 33 subtypes of events.

Related Work

Almost all the current ACE event extraction systems focus on processing one sentence at a time (Grishman et al., 2005; Ahn, 2006; Hardy et al.

Motivation

We analyzed the sentence-level baseline event extraction, and found that many events are missing or spuriously tagged because the local information is not sufficient to make a confident decision.

Cross-event Approach

In this section we present our approach to using document-level event and role information to improve sentence-level ACE event extraction.

Experiments

We followed Ji and Grishman (2008)’s evaluation and randomly select 10 newswire texts from the ACE 2005 training corpora as our development set, which is used for parameter tuning, and then conduct a blind test on a separate set of 40 ACE 2005 newswire texts.

Conclusion and Future Work

We propose a document-level statistical model for event trigger and argument (role) classification to achieve document level within-event and cross-event consistency.

Topics

entity mentions

Appears in 8 sentences as: Entity mention (1) entity mention (1) entity mentions (6)
In Using Document Level Cross-Event Inference to Improve Event Extraction
  1. (coreferential) entity mentions .
    Page 2, “Task Description”
  2. Entity mention : a reference to an entity (typically, a noun phrase)
    Page 2, “Task Description”
  3. Event mention arguments (roles)2: the entity mentions that are involved in an event mention, and their relation to the event.
    Page 2, “Task Description”
  4. Event extraction depends on previous phases entity mention classification and coreference.
    Page 2, “Task Description”
  5. Note that entity mentions that share the same EntityID are coreferential and treated as the same object.
    Page 2, “Task Description”
  6. An example of entities and entity mentions and their types
    Page 2, “Task Description”
  7. For argument tagging, we only consider the entity mentions in the same sentence as the trigger word, because by the ACE event guidelines, the arguments of an event should appear within the same sentence as the trigger.
    Page 8, “Cross-event Approach”
  8. For a given event, we re-tag the entity mentions that have not already been assigned as arguments of that event by the confident-event or conflict table.
    Page 8, “Cross-event Approach”

See all papers in Proc. ACL 2010 that mention entity mentions.

See all papers in Proc. ACL that mention entity mentions.

Back to top.

extraction systems

Appears in 8 sentences as: extraction system (3) extraction systems (5)
In Using Document Level Cross-Event Inference to Improve Event Extraction
  1. Most current event extraction systems rely on local information at the phrase or sentence level.
    Page 1, “Abstract”
  2. Most current event extraction systems are based on phrase or sentence level extraction.
    Page 2, “Introduction”
  3. Several recent studies use high-level information to aid local event extraction systems .
    Page 2, “Introduction”
  4. We extend these approaches by introducing cross-event information to enhance the performance of multi-event-type extraction systems .
    Page 2, “Introduction”
  5. Almost all the current ACE event extraction systems focus on processing one sentence at a time (Grishman et al., 2005; Ahn, 2006; Hardy et al.
    Page 3, “Related Work”
  6. They used this technique to augment an information extraction system with long-distance dependency models, enforcing label consistency and extraction template consistency constraints.
    Page 3, “Related Work”
  7. Our event extraction system is a two-pass system where the sentence-level system is first applied to make decisions based on local information.
    Page 5, “Cross-event Approach”
  8. Experiments show that document-level information can improve the performance of a sentence-level baseline event extraction system .
    Page 8, “Conclusion and Future Work”

See all papers in Proc. ACL 2010 that mention extraction systems.

See all papers in Proc. ACL that mention extraction systems.

Back to top.

sentence-level

Appears in 8 sentences as: Sentence-level (1) sentence-level (7)
In Using Document Level Cross-Event Inference to Improve Event Extraction
  1. We analyzed the sentence-level baseline event extraction, and found that many events are missing or spuriously tagged because the local information is not sufficient to make a confident decision.
    Page 3, “Motivation”
  2. In this section we present our approach to using document-level event and role information to improve sentence-level ACE event extraction.
    Page 5, “Cross-event Approach”
  3. Our event extraction system is a two-pass system where the sentence-level system is first applied to make decisions based on local information.
    Page 5, “Cross-event Approach”
  4. 5.1 Sentence-level Baseline System
    Page 6, “Cross-event Approach”
  5. To use document-level information, we need to collect information based on the sentence-level baseline system.
    Page 6, “Cross-event Approach”
  6. We use the rest of the ACE training corpus (549 documents) as training data for both the sentence-level baseline event tagger and document-level event tagger.
    Page 8, “Experiments”
  7. Recall improved sharply, demonstrating that cross-event information could recover information that is difficult for the sentence-level baseline to extract; precision also improved over the baseline, although not as markedly.
    Page 8, “Experiments”
  8. Experiments show that document-level information can improve the performance of a sentence-level baseline event extraction system.
    Page 8, “Conclusion and Future Work”

See all papers in Proc. ACL 2010 that mention sentence-level.

See all papers in Proc. ACL that mention sentence-level.

Back to top.

baseline system

Appears in 6 sentences as: Baseline System (1) baseline system (5)
In Using Document Level Cross-Event Inference to Improve Event Extraction
  1. The sentence level baseline system finds event triggers like “founded” (trigger of Start-Org), “elected” (trigger of Elect), and “appointment” (trigger of Start-Position), which are easier to identify because these triggers have more specific meanings.
    Page 4, “Motivation”
  2. 5.1 Sentence-level Baseline System
    Page 6, “Cross-event Approach”
  3. To use document-level information, we need to collect information based on the sentence-level baseline system .
    Page 6, “Cross-event Approach”
  4. To this end, we set different thresholds from 0.1 to 1.0 in the baseline system output, and only evaluate triggers, arguments or roles whose confidence score is above the threshold.
    Page 6, “Cross-event Approach”
  5. The performance of different confidence thresholds in the baseline system on the development set
    Page 6, “Cross-event Approach”
  6. To achieve document consistency, in cases where the baseline system assigns a word to triggers for more than one event type, if the margin between the probability of the highest and the second highest scores is above a threshold m_threshold, we only keep the event type with highest score and record this in the confident-event table.
    Page 6, “Cross-event Approach”

See all papers in Proc. ACL 2010 that mention baseline system.

See all papers in Proc. ACL that mention baseline system.

Back to top.

coreference

Appears in 5 sentences as: coreference (3) coreferential (2)
In Using Document Level Cross-Event Inference to Improve Event Extraction
  1. ( coreferential ) entity mentions.
    Page 2, “Task Description”
  2. Event extraction depends on previous phases entity mention classification and coreference .
    Page 2, “Task Description”
  3. Note that entity mentions that share the same EntityID are coreferential and treated as the same object.
    Page 2, “Task Description”
  4. 2 Note that we do not deal with event mention coreference in this paper, so each event mention is treated as a separate
    Page 2, “Task Description”
  5. For every event, we collect its trigger and event type; for every argument, we use coreference information and record every entity and its role(s) in events of a certain type.
    Page 6, “Cross-event Approach”

See all papers in Proc. ACL 2010 that mention coreference.

See all papers in Proc. ACL that mention coreference.

Back to top.

rule-based

Appears in 3 sentences as: rule-based (3)
In Using Document Level Cross-Event Inference to Improve Event Extraction
  1. Ji and Grishman (2008) were inspired from the hypothesis of “One Sense Per Discourse” (Yarowsky, 1995); they extended the scope from a single document to a cluster of topic-related documents and employed a rule-based approach
    Page 3, “Related Work”
  2. Compared to the within-event-type rules, the cross-event model yields much more improvement for trigger classification: rule-based propagation gains 1.7% improvement while the cross-event model achieves a further 7.3% improvement.
    Page 8, “Experiments”
  3. For argument and role classification, the cross-event model also gains 3% and 2.3% above that obtained by the rule-based propagation process.
    Page 8, “Experiments”

See all papers in Proc. ACL 2010 that mention rule-based.

See all papers in Proc. ACL that mention rule-based.

Back to top.

PDF available from www.aclweb.org.
Send us an email