In-domain Relation Discovery with Meta-constraints via Posterior Regularization
Chen, Harr and Benson, Edward and Naseem, Tahira and Barzilay, Regina

Article Structure

Abstract

We present a novel approach to discovering relations and their instantiations from a collection of documents in a single domain.

Introduction

In this paper, we introduce a novel approach for the unsupervised learning of relations and their instantiations from a set of in-domain documents.

Related Work

Extraction with Reduced Supervision Recent research in information extraction has taken large steps toward reducing the need for labeled data.

Model

Our work performs in-domain relation discovery by leveraging regularities in relation expression at the lexical, syntactic, and discourse levels.

Inference with Constraints

The model presented above leverages relation regularities in local features and document placement.

Declarative Constraints

We now have the machinery to incorporate a variety of declarative constraints during inference.

Experimental Setup

Datasets and Metrics We evaluate on two datasets, financial market reports and newswire articles about earthquakes, previously used in work on high-level content analysis (Barzilay and Lee, 2004; Lap-ata, 2006).

Results

Table 3’s first two sections present the results of our main evaluation.

Conclusions

This paper has presented a constraint-based approach to in-domain relation discovery.

Topics

generative process

Appears in 6 sentences as: Generative Process (1) generative process (5)
In In-domain Relation Discovery with Meta-constraints via Posterior Regularization
  1. First, the model’s generative process encourages coherence in the local features and placement of relation instances.
    Page 2, “Introduction”
  2. This section describes the generative process , while Sections 4 and 5 discuss declarative constraints.
    Page 3, “Model”
  3. 3.2 Generative Process
    Page 3, “Model”
  4. There are three steps to the generative process .
    Page 3, “Model”
  5. Figure 3 presents a reference for the generative process .
    Page 3, “Model”
  6. Figure 3: The generative process for model parameters and features.
    Page 4, “Model”

See all papers in Proc. ACL 2011 that mention generative process.

See all papers in Proc. ACL that mention generative process.

Back to top.

relation instances

Appears in 6 sentences as: relation instance (1) relation instances (6)
In In-domain Relation Discovery with Meta-constraints via Posterior Regularization
  1. First, the model’s generative process encourages coherence in the local features and placement of relation instances .
    Page 2, “Introduction”
  2. diversity in the discovered relation types by restricting the number of times a single word can serve as either an indicator or part of the argument of a relation instance .
    Page 6, “Declarative Constraints”
  3. To incorporate training examples in our model, we simply treat annotated relation instances as observed variables.
    Page 9, “Results”
  4. For finance, it takes at least 10 annotated documents (corresponding to roughly 130 annotated relation instances ) for the CRF to match the semi-supervised model’s performance.
    Page 9, “Results”
  5. For earthquake, using even 10 annotated documents (about 71 relation instances ) is not sufficient to match our model’s performance.
    Page 9, “Results”
  6. Using a single labeled document (13 relation instances) yields superior performance to either of our model variants for finance, while four labeled documents (29 relation instances ) do the same for earthquake.
    Page 9, “Results”

See all papers in Proc. ACL 2011 that mention relation instances.

See all papers in Proc. ACL that mention relation instances.

Back to top.

CRF

Appears in 5 sentences as: CRF (6)
In In-domain Relation Discovery with Meta-constraints via Posterior Regularization
  1. Comparison against Supervised CRF Our final set of experiments compares a semi-supervised version of our model against a conditional random field ( CRF ) model.
    Page 9, “Results”
  2. The CRF model was trained using the same features as our model’s argument features.
    Page 9, “Results”
  3. At the sentence level, our model compares very favorably to the supervised CRF .
    Page 9, “Results”
  4. For finance, it takes at least 10 annotated documents (corresponding to roughly 130 annotated relation instances) for the CRF to match the semi-supervised model’s performance.
    Page 9, “Results”
  5. At the token level, the supervised CRF baseline is far more competitive.
    Page 9, “Results”

See all papers in Proc. ACL 2011 that mention CRF.

See all papers in Proc. ACL that mention CRF.

Back to top.

in-domain

Appears in 5 sentences as: in-domain (5)
In In-domain Relation Discovery with Meta-constraints via Posterior Regularization
  1. In this paper, we introduce a novel approach for the unsupervised learning of relations and their instantiations from a set of in-domain documents.
    Page 1, “Introduction”
  2. Clusters of similar in-domain documents are
    Page 1, “Introduction”
  3. Our work performs in-domain relation discovery by leveraging regularities in relation expression at the lexical, syntactic, and discourse levels.
    Page 2, “Model”
  4. methods ultimately aim to capture domain-specific relations expressed with varying verbalizations, and both operate over in-domain input corpora supplemented with syntactic information.
    Page 8, “Experimental Setup”
  5. This paper has presented a constraint-based approach to in-domain relation discovery.
    Page 9, “Conclusions”

See all papers in Proc. ACL 2011 that mention in-domain.

See all papers in Proc. ACL that mention in-domain.

Back to top.

F-score

Appears in 3 sentences as: F-Score (1) F-score (3)
In In-domain Relation Discovery with Meta-constraints via Posterior Regularization
  1. For these reasons, we evaluate on both sentence-level and token-level precision, recall, and F-score .
    Page 7, “Experimental Setup”
  2. However, the best F-Score corresponding to the optimal number of clusters is 42.2, still far below our model’s 66.0 F-score .
    Page 8, “Results”
  3. Our results show a large gap in F-score between the sentence and token-level evaluations for both the USP baseline and our model.
    Page 8, “Results”

See all papers in Proc. ACL 2011 that mention F-score.

See all papers in Proc. ACL that mention F-score.

Back to top.

hyperparameters

Appears in 3 sentences as: Hyperparameters (1) hyperparameters (2)
In In-domain Relation Discovery with Meta-constraints via Posterior Regularization
  1. Fixed hyperparameters are subscripted with zero.
    Page 4, “Model”
  2. Training Regimes and Hyperparameters For each run of our model we perform three random restarts to convergence and select the posterior with lowest final free energy.
    Page 8, “Experimental Setup”
  3. Dirichlet hyperparameters are set to 0.1.
    Page 8, “Experimental Setup”

See all papers in Proc. ACL 2011 that mention hyperparameters.

See all papers in Proc. ACL that mention hyperparameters.

Back to top.

part-of-speech

Appears in 3 sentences as: part-of-speech (3)
In In-domain Relation Discovery with Meta-constraints via Posterior Regularization
  1. These features can encode words, part-of-speech tags, context, and so on.
    Page 3, “Model”
  2. This mapping technique is based on the many-to-one scheme used for evaluating unsupervised part-of-speech induction (Johnson, 2007).
    Page 7, “Experimental Setup”
  3. We implement a clustering baseline using the CLUTO toolkit with word and part-of-speech features.
    Page 7, “Experimental Setup”

See all papers in Proc. ACL 2011 that mention part-of-speech.

See all papers in Proc. ACL that mention part-of-speech.

Back to top.

semi-supervised

Appears in 3 sentences as: semi-supervised (3)
In In-domain Relation Discovery with Meta-constraints via Posterior Regularization
  1. (2007) propose an objective function for semi-supervised extraction that balances likelihood of labeled instances and constraint violation on unlabeled instances.
    Page 2, “Related Work”
  2. Comparison against Supervised CRF Our final set of experiments compares a semi-supervised version of our model against a conditional random field (CRF) model.
    Page 9, “Results”
  3. For finance, it takes at least 10 annotated documents (corresponding to roughly 130 annotated relation instances) for the CRF to match the semi-supervised model’s performance.
    Page 9, “Results”

See all papers in Proc. ACL 2011 that mention semi-supervised.

See all papers in Proc. ACL that mention semi-supervised.

Back to top.

sentence-level

Appears in 3 sentences as: sentence-level (3)
In In-domain Relation Discovery with Meta-constraints via Posterior Regularization
  1. For these reasons, we evaluate on both sentence-level and token-level precision, recall, and F-score.
    Page 7, “Experimental Setup”
  2. Note that sentence-level scores are always at least as high as token-level scores, since it is possible to select a sentence correctly but none of its true relation tokens while the opposite is not possible.
    Page 7, “Experimental Setup”
  3. In light of our strong sentence-level performance, this suggests a possible human-assisted application: use our model to identify promising relation-bearing sentences in a new domain, then have a human annotate those sentences for use by a supervised approach to achieve optimal token-level extraction.
    Page 9, “Results”

See all papers in Proc. ACL 2011 that mention sentence-level.

See all papers in Proc. ACL that mention sentence-level.

Back to top.

PDF available from www.aclweb.org.
Send us an email