Joint Inference for Fine-grained Opinion Extraction
Yang, Bishan and Cardie, Claire

Article Structure

Abstract

This paper addresses the task of fine-grained opinion extraction — the identification of opinion-related entities: the opinion expressions, the opinion holders, and the targets of the opinions, and the relations between opinion expressions and their targets and holders.

Introduction

Fine-grained opinion analysis is concerned with identifying opinions in text at the expression level; this includes identifying the subjective (i.e., opinion) expression itself, the opinion holder and the target of the opinion (Wiebe et al., 2005).

Related Work

Significant research effort has been invested into fine-grained opinion extraction for open-domain text such as news articles (Wiebe et al., 2005; Wilson et al., 2009).

Model

As proposed in Section 1, we consider the task of jointly identifying opinion entities and opinion relations.

Experiments

For evaluation, we used version 2.0 of the MPQA corpus (Wiebe et al., 2005; Wilson, 2008), a widely used data set for fine-grained opinion analysis.6 We considered the subset of 482 documents7 that contain attitude and target annotations.

Results

Table 2 shows the results of opinion entity identification using both overlap and exact metrics.

Discussion

We note that the joint inference model yields a clear improvement on recall but not on precision compared to the CRF-based baselines.

Conclusion

In this paper we propose a joint inference approach for extracting opinion-related entities and opinion relations.

Topics

relation extraction

Appears in 20 sentences as: Relation Extraction (1) relation extraction (18) relation extractor (1)
In Joint Inference for Fine-grained Opinion Extraction
  1. 2007; Yang and Cardie, 2012)) and relation extraction techniques have been proposed to extract opinion holders and targets based on their linking relations to the opinion expressions (e. g. Kim and Hovy (2006), Kobayashi et al.
    Page 2, “Introduction”
  2. We model entity identification as a sequence tagging problem and relation extraction as binary classification.
    Page 2, “Introduction”
  3. In this section, we will describe how we model opinion entity identification and opinion relation extraction , and how we combine them in a joint inference model.
    Page 3, “Model”
  4. 3.2 Opinion Relation Extraction
    Page 3, “Model”
  5. In the following we will not distinguish these two relations, since they can both be characterized as relations between opinion expressions and opinion arguments, and the methods for relation extraction are the same.
    Page 3, “Model”
  6. We treat the relation extraction problem as a combination of two binary classification problems: opinion-arg classification, which decides whether a pair consisting of an opinion candidate 0 and an argument candidate a forms a relation; and opinion-implicit-arg classification, which decides whether an opinion candidate 0 is linked to an implicit argument, i.e.
    Page 3, “Model”
  7. The inference goal is to find the optimal prediction for both opinion entity identification and opinion relation extraction .
    Page 4, “Model”
  8. The objective function is defined as a linear combination of the potentials from different predictors with a parameter A to balance the contribution of two components: opinion entity identification and opinion relation extraction .
    Page 5, “Model”
  9. We adopted the evaluation metrics for entity and relation extraction from Choi et al.
    Page 6, “Experiments”
  10. We trained the classifiers for relation extraction using L1-regu1arized logistic regression with default parameters using the LIBLINEAR (Fan et al., 2008) package.
    Page 6, “Experiments”
  11. Three relation extraction techniques were used in the baselines:
    Page 6, “Experiments”

See all papers in Proc. ACL 2013 that mention relation extraction.

See all papers in Proc. ACL that mention relation extraction.

Back to top.

CRF

Appears in 14 sentences as: CRF (17)
In Joint Inference for Fine-grained Opinion Extraction
  1. 1We randomly split the training data into 10 parts and obtained the 50-best CRF predictions on each part for the generation of candidates.
    Page 3, “Model”
  2. We also experimented with candidates generated from more CRF predictions, but did not find any performance improvement for the task.
    Page 3, “Model”
  3. Each extracts opinion entities first using the same CRF employed in our approach, and then predicts opinion relations on the opinion entity candidates obtained from the CRF prediction.
    Page 6, “Experiments”
  4. We report results using opinion entity candidates from the best CRF output and from the merged lO-best CRF output.10 The motivation of merging the lO-best output is to increase recall for the pipeline methods.
    Page 7, “Experiments”
  5. We compare our approach with the pipeline baselines and CRF (the first step of the pipeline).
    Page 7, “Results”
  6. by adding the relation extraction step, the pipeline baselines are able to improve precision over the CRF but fail at recall.
    Page 7, “Results”
  7. CRF+Syn and CRF+Adj provide the same performance as CRF , since the relation extraction step only affects the results of opinion arguments.
    Page 7, “Results”
  8. We compare our approach with pipelined baselines in two settings: one employs relation extraction on l-best output of CRF (top half of table) and the other employs the merged lO-best output of CRF (bottom half of table).
    Page 7, “Results”
  9. We can see that in general, using merged lO-best CRF outputs boosts the recall while sacrificing precision.
    Page 7, “Results”
  10. This is expected since merging the lO-best CRF outputs favors candidates that are
    Page 7, “Results”
  11. believed to be more accurate by the CRF predictor.
    Page 8, “Results”

See all papers in Proc. ACL 2013 that mention CRF.

See all papers in Proc. ACL that mention CRF.

Back to top.

ILP

Appears in 14 sentences as: ILP (16)
In Joint Inference for Fine-grained Opinion Extraction
  1. (2006), which proposed an ILP approach to jointly identify opinion holders, opinion expressions and their IS-FROM linking relations, and demonstrated the effectiveness of joint inference.
    Page 2, “Introduction”
  2. Their ILP formulation, however, does not handle implicit linking relations, i.e.
    Page 2, “Introduction”
  3. (2006), which jointly extracts opinion expressions, holders and their IS-FROM relations using an ILP approach.
    Page 2, “Related Work”
  4. In contrast, our approach (1) also considers the IS-AB OUT relation which is arguably more complex due to the larger variety in the syntactic structure exhibited by opinion expressions and their targets, (2) handles implicit opinion relations (opinion expressions without any associated argument), and (3) uses a simpler ILP formulation.
    Page 2, “Related Work”
  5. Note that in our ILP formulation, the label assignment for a candidate span involves one multiple-choice decision among different opinion entity labels and the “NONE” entity label.
    Page 5, “Model”
  6. This makes our ILP formulation advantageous over the ILP formulation proposed in Choi et al.
    Page 5, “Model”
  7. For joint inference, we used GLPK9 to provide the optimal ILP solution.
    Page 6, “Experiments”
  8. To demonstrate the effectiveness of different potentials in our joint inference model, we consider three variants of our ILP formulation that omit some potentials in the joint inference: one is ILP-W/O-ENTITY, which extracts opinion relations without integrating information from opinion entity identification; one is ILP-W-SINGLE-RE, which focuses on extracting a single opinion relation and ignores the information from the other relation; the third one is ILP-W/O-IMPLICIT—RE, which omits the potential for opinion-implicit-arg relation and assumes every opinion expression is linked to an explicit argument.
    Page 8, “Results”
  9. It can be viewed as an extension to the ILP approach in Choi et al.
    Page 8, “Results”
  10. (2006) that includes opinion targets and uses simpler ILP formulation with only one parameter and fewer binary variables and constraints to represent entity label assignments 11.
    Page 8, “Results”
  11. 11We compared the proposed [LP formulation with the ILP
    Page 8, “Results”

See all papers in Proc. ACL 2013 that mention ILP.

See all papers in Proc. ACL that mention ILP.

Back to top.

fine-grained

Appears in 6 sentences as: Fine-grained (1) fine-grained (5)
In Joint Inference for Fine-grained Opinion Extraction
  1. This paper addresses the task of fine-grained opinion extraction — the identification of opinion-related entities: the opinion expressions, the opinion holders, and the targets of the opinions, and the relations between opinion expressions and their targets and holders.
    Page 1, “Abstract”
  2. Fine-grained opinion analysis is concerned with identifying opinions in text at the expression level; this includes identifying the subjective (i.e., opinion) expression itself, the opinion holder and the target of the opinion (Wiebe et al., 2005).
    Page 1, “Introduction”
  3. Not surprisingly, fine-grained opinion extraction is a challenging task due to the complexity and variety of the language used to express opinions and their components (Pang and Lee, 2008).
    Page 1, “Introduction”
  4. We evaluate our approach using a standard corpus for fine-grained opinion analysis (the MPQA corpus (Wiebe et al., 2005)) and demonstrate that our model outperforms by a significant margin traditional baselines that do not employ joint inference for extracting opinion entities and different types of opinion relations.
    Page 2, “Introduction”
  5. Significant research effort has been invested into fine-grained opinion extraction for open-domain text such as news articles (Wiebe et al., 2005; Wilson et al., 2009).
    Page 2, “Related Work”
  6. For evaluation, we used version 2.0 of the MPQA corpus (Wiebe et al., 2005; Wilson, 2008), a widely used data set for fine-grained opinion analysis.6 We considered the subset of 482 documents7 that contain attitude and target annotations.
    Page 6, “Experiments”

See all papers in Proc. ACL 2013 that mention fine-grained.

See all papers in Proc. ACL that mention fine-grained.

Back to top.

gold standard

Appears in 4 sentences as: gold standard (5)
In Joint Inference for Fine-grained Opinion Extraction
  1. Similar to the preprocessing approach in (Choi et al., 2006), we filter pairs of opinion and argument candidates that do not overlap with any gold standard relation in our training data.
    Page 4, “Model”
  2. Our gold standard opinion expressions, opinion targets and opinion holders correspond to the direct subjective annotations, target annotations and agent annotations, respectively.
    Page 6, “Experiments”
  3. Analyzing the errors, we found that the joint model extracts comparable number of opinion entities compared to the gold standard, while the CRF-based baselines extract significantly fewer opinion entities (around 60% of the number of entities in the gold standard ).
    Page 8, “Discussion”
  4. It is possible that the generated candidates do not contain the gold standard answers.
    Page 8, “Discussion”

See all papers in Proc. ACL 2013 that mention gold standard.

See all papers in Proc. ACL that mention gold standard.

Back to top.

objective function

Appears in 4 sentences as: objective function (4)
In Joint Inference for Fine-grained Opinion Extraction
  1. The objective function is defined as a linear combination of the potentials from different predictors with a parameter A to balance the contribution of two components: opinion entity identification and opinion relation extraction.
    Page 5, “Model”
  2. The objective function of ILP-W/O-ENTITY can be represented as
    Page 8, “Results”
  3. For ILP-W-SINGLE-RE, we simply remove the variables associated with one opinion relation in the objective function (1) and constraints.
    Page 8, “Results”
  4. The formulation of ILP-W/O-IMPLICIT—RE removes the variables associated with potential 7“,- in the objective function and corresponding constraints.
    Page 8, “Results”

See all papers in Proc. ACL 2013 that mention objective function.

See all papers in Proc. ACL that mention objective function.

Back to top.

parse tree

Appears in 4 sentences as: parse tree (4) parse trees (1)
In Joint Inference for Fine-grained Opinion Extraction
  1. Phrase type: the syntactic category of the deepest constituent that covers the candidate in the parse tree , e.g.
    Page 4, “Model”
  2. 2We use the Stanford Parser to generate parse trees and dependency graphs.
    Page 4, “Model”
  3. Neighboring constituents: The words and grammatical roles of neighboring constituents of the opinion expression in the parse tree — the left and right sibling of the deepest constituent containing the opinion expression in the parse tree .
    Page 4, “Model”
  4. Recall that the joint model finds the global optimal solution over a set of opinion entity and relation candidates, which are obtained from the n-best CRF predictions and constituents in the parse tree that satisfy certain syntactic patterns.
    Page 8, “Discussion”

See all papers in Proc. ACL 2013 that mention parse tree.

See all papers in Proc. ACL that mention parse tree.

Back to top.

binary classification

Appears in 3 sentences as: binary classification (2) binary classifiers (1)
In Joint Inference for Fine-grained Opinion Extraction
  1. We model entity identification as a sequence tagging problem and relation extraction as binary classification .
    Page 2, “Introduction”
  2. We treat the relation extraction problem as a combination of two binary classification problems: opinion-arg classification, which decides whether a pair consisting of an opinion candidate 0 and an argument candidate a forms a relation; and opinion-implicit-arg classification, which decides whether an opinion candidate 0 is linked to an implicit argument, i.e.
    Page 3, “Model”
  3. By using binary classifiers to predict relations, CRF+RE produces high precision on opinion and target extraction but also results in very low recall.
    Page 7, “Results”

See all papers in Proc. ACL 2013 that mention binary classification.

See all papers in Proc. ACL that mention binary classification.

Back to top.

CRFs

Appears in 3 sentences as: CRFs (3)
In Joint Inference for Fine-grained Opinion Extraction
  1. We formulate the task of opinion entity identification as a sequence labeling problem and employ conditional random fields ( CRFs ) (Lafferty et al., 2001) to learn the probability of a sequence assignment y for a given sentence x.
    Page 3, “Model”
  2. We define potential function fig that gives the probability of assigning a span 2' with entity label 2, and the probability is estimated based on the learned parameters from CRFs .
    Page 3, “Model”
  3. We trained CRFs for opinion entity identification using the following features: indicators for words, POS tags, and lexicon features (the subjectivity strength of the word in the Subjectivity Lexicon).
    Page 6, “Experiments”

See all papers in Proc. ACL 2013 that mention CRFs.

See all papers in Proc. ACL that mention CRFs.

Back to top.

development set

Appears in 3 sentences as: development set (3)
In Joint Inference for Fine-grained Opinion Extraction
  1. We set aside 132 documents as a development set and use 350 documents as the evaluation set.
    Page 6, “Experiments”
  2. We used L2-regu1arization; the regularization parameter was tuned using the development set .
    Page 6, “Experiments”
  3. The parameter A was tuned using the development set .
    Page 6, “Experiments”

See all papers in Proc. ACL 2013 that mention development set.

See all papers in Proc. ACL that mention development set.

Back to top.

POS tags

Appears in 3 sentences as: POS tags (4)
In Joint Inference for Fine-grained Opinion Extraction
  1. Words and POS tags: the words contained in the candidate and their POS tags .
    Page 4, “Model”
  2. For features, we use words, POS tags , phrase types, lexicon and semantic frames (see Section 3.2.1 for details) to capture the properties of the opinion expression, and also features that capture the context of the opinion expression:
    Page 4, “Model”
  3. We trained CRFs for opinion entity identification using the following features: indicators for words, POS tags , and lexicon features (the subjectivity strength of the word in the Subjectivity Lexicon).
    Page 6, “Experiments”

See all papers in Proc. ACL 2013 that mention POS tags.

See all papers in Proc. ACL that mention POS tags.

Back to top.

semantic role

Appears in 3 sentences as: semantic role (2) semantic roles (1)
In Joint Inference for Fine-grained Opinion Extraction
  1. Kim and Hovy (2006) identifies opinion holders and targets by using their semantic roles related to opinion words.
    Page 2, “Related Work”
  2. (2008) argued that semantic role labeling is not sufficient for identifying opinion holders and targets.
    Page 2, “Related Work”
  3. Joint inference has also been applied to semantic role labeling (Punyakanok et al., 2008; Srikumar and Roth, 2011; Das et al., 2012), where the goal is to jointly identify semantic arguments for given lexical predicates.
    Page 3, “Related Work”

See all papers in Proc. ACL 2013 that mention semantic role.

See all papers in Proc. ACL that mention semantic role.

Back to top.

sequence labeling

Appears in 3 sentences as: Sequence labeling (1) sequence labeling (2)
In Joint Inference for Fine-grained Opinion Extraction
  1. Sequence labeling models have been successfully employed to identify opinion expressions (e.g.
    Page 1, “Introduction”
  2. J ohansson and Moschitti (2010a) extract opinion expressions and holders by applying reranking on top of sequence labeling methods.
    Page 2, “Related Work”
  3. We formulate the task of opinion entity identification as a sequence labeling problem and employ conditional random fields (CRFs) (Lafferty et al., 2001) to learn the probability of a sequence assignment y for a given sentence x.
    Page 3, “Model”

See all papers in Proc. ACL 2013 that mention sequence labeling.

See all papers in Proc. ACL that mention sequence labeling.

Back to top.

PDF available from www.aclweb.org.
Send us an email