ConnotationWordNet: Learning Connotation over the Word+Sense Network
Kang, Jun Seok and Feng, Song and Akoglu, Leman and Choi, Yejin

Article Structure

Abstract

We introduce ConnotationWordNet, a connotation lexicon over the network of words in conjunction with senses.

Introduction

We introduce ConnotationWordNet, a connotation lexicon over the network of words in conjunction with senses, as defined in WordNet.

Network of Words and Senses

The connotation graph, called GWORD+SENSE, is a heterogeneous graph with multiple types of nodes and edges.

Pairwise Markov Random Fields and Loopy Belief Propagation

We formulate the task of learning sense- and word-level connotation lexicon as a graph-based classification task (Sen et al., 2008).

Evaluation 1: Agreement with Sentiment Lexicons

ConnotationWordNet is expected to be the superset of a sentiment lexicon, as it is highly likely for any word with positive/negative sentiment to carry connotation of the same polarity.

Evaluation 11: Human Evaluation on ConnotationWordNet

In this section, we present the result of human evaluation we executed using Amazon Mechanical Turk (AMT).

Evaluation 111: Sentiment Analysis using ConnotationWordNet

Finally, to show the utility of the resulting lexicon in the context of a concrete sentiment analysis

Related Work

Several previous approaches explored the use of graph propagation for sentiment lexicon induction (Velikovich et al., 2010) and connotation lexicon

Conclusion

We have introduced a novel formulation of lexicon induction operating over both words and senses, by exploiting the innate structure between the words and senses as encoded in WordNet.

Topics

synsets

Appears in 24 sentences as: synset (12) synsets (15)
In ConnotationWordNet: Learning Connotation over the Word+Sense Network
  1. 1Hence a sense in WordNet is defined by synset (= synonym set), which is the set of words sharing the same sense.
    Page 1, “Introduction”
  2. As shown in Figure 1, it contains two types of nodes; (i) lemmas (i.e., words, 115K) and (ii) synsets (63K), and four types of edges; (t1) predicate-argument (179K), (t2) argument-argument (144K), (t3) argument-synset (126K), and (t4) synset-synset (3.4K) edges.
    Page 2, “Network of Words and Senses”
  3. The argument-synset edges capture the synonymy between argument nodes through the corresponding synsets .
    Page 2, “Network of Words and Senses”
  4. Finally, the synset-synset edges depict the antonym relations between synset pairs.
    Page 2, “Network of Words and Senses”
  5. Most notably, we model both words and synsets explicitly, and exploit the membership relations between words and senses.
    Page 2, “Network of Words and Senses”
  6. Conversely, we expect that these edges will also encourage words that belong to the same sense (i.e., synset definition) to receive the same connotation label.
    Page 3, “Network of Words and Senses”
  7. Another benefit of our approach is that for various WordNet relations (e.g., antonym relations), which are defined over synsets (not over words), we can add edges directly between corresponding synsets , rather than projecting (i.e., approximating) those relations over words.
    Page 3, “Network of Words and Senses”
  8. (2013)), could be a source of noise, as one needs to assume that the semantic relation between a pair of synsets transfers over the pair of words corresponding to that pair of synsets .
    Page 3, “Network of Words and Senses”
  9. More formally, we denote the connotation graph GWORDJ'SEI‘ISE by G = (V, E), in which a total of n word and synset nodes V = {211, .
    Page 3, “Pairwise Markov Random Fields and Loopy Belief Propagation”
  10. and synsets connected with typed edges, - prior knowledge (i.e., probabilities) of (some or all) nodes belonging to each class,
    Page 3, “Pairwise Markov Random Fields and Loopy Belief Propagation”
  11. The construction of the connotation graph, denoted by GWORD+SENSE, which includes words and synsets , has been described in Section 2.
    Page 5, “Evaluation 1: Agreement with Sentiment Lexicons”

See all papers in Proc. ACL 2014 that mention synsets.

See all papers in Proc. ACL that mention synsets.

Back to top.

word-level

Appears in 15 sentences as: Word-Level (1) Word-level (1) word-level (13)
In ConnotationWordNet: Learning Connotation over the Word+Sense Network
  1. For non-polysemous words, which constitute a significant portion of English vocabulary, learning the general connotation at the word-level (rather than at the sense-level) would be a natural operational choice.
    Page 1, “Introduction”
  2. As a result, researchers often would need to aggregate labels across different senses to derive the word-level label.
    Page 2, “Introduction”
  3. Therefore, in this work, we present the first unified approach that learns both sense- and word-level connotations simultaneously.
    Page 2, “Introduction”
  4. This way, end-users will have access to more accurate sense-level connotation labels if needed, while also having access to more general word-level connotation labels.
    Page 2, “Introduction”
  5. In addition, previous studies (for both sentiment and connotation lexicons) aimed to produce only either of the two aspects of the polarity: word-level or sense-level, while we address both.
    Page 2, “Introduction”
  6. We formulate the task of learning sense- and word-level connotation lexicon as a graph-based classification task (Sen et al., 2008).
    Page 3, “Pairwise Markov Random Fields and Loopy Belief Propagation”
  7. We collect two separate sets of labels: a set of labels at the word-level , and another set at the sense-level.
    Page 7, “Evaluation 11: Human Evaluation on ConnotationWordNet”
  8. For word-level labels we apply similar procedure as above.
    Page 7, “Evaluation 11: Human Evaluation on ConnotationWordNet”
  9. Lexicon Word-level Sense-level SentiWordNet 27.22 14.29 OpinionFinder 3 1 .95 -Feng2013 62.72 -GWORD+SENSE(95%) 84.91 83.43 GWORD+SENSE(99%) 84.91 83.71 E-GWORD+SENSE(95%) 86.98 86.29 E-GWORD+SENSE(99%) 86.69 85.71
    Page 7, “Evaluation 11: Human Evaluation on ConnotationWordNet”
  10. 5.1 Word-Level Evaluation
    Page 7, “Evaluation 11: Human Evaluation on ConnotationWordNet”
  11. We first evaluate the word-level assignment of connotation, as shown in Table 3.
    Page 7, “Evaluation 11: Human Evaluation on ConnotationWordNet”

See all papers in Proc. ACL 2014 that mention word-level.

See all papers in Proc. ACL that mention word-level.

Back to top.

WordNet

Appears in 11 sentences as: WordNet (12)
In ConnotationWordNet: Learning Connotation over the Word+Sense Network
  1. The key aspect of our method is that it is the first unified approach that assigns the polarity of both word- and sense-level connotations, exploiting the innate bipartite graph structure encoded in WordNet .
    Page 1, “Abstract”
  2. We introduce ConnotationWordNet, a connotation lexicon over the network of words in conjunction with senses, as defined in WordNet .
    Page 1, “Introduction”
  3. For example, consider “abound”, for which lexicographers of WordNet prescribe two different senses:
    Page 1, “Introduction”
  4. Especially if we look up the WordNet entry for “bristle”, there are noticeably more negatively connotative words involved in its gloss and examples.
    Page 1, “Introduction”
  5. 1Hence a sense in WordNet is defined by synset (= synonym set), which is the set of words sharing the same sense.
    Page 1, “Introduction”
  6. The key aspect of our approach is that we exploit the innate bipartite graph structure between words and senses encoded in WordNet .
    Page 2, “Introduction”
  7. ConnotationWordNet, the final outcome of our study, is a new lexical resource that has connotation labels over both words and senses following the structure of WordNet .
    Page 2, “Introduction”
  8. Another benefit of our approach is that for various WordNet relations (e.g., antonym relations), which are defined over synsets (not over words), we can add edges directly between corresponding synsets, rather than projecting (i.e., approximating) those relations over words.
    Page 3, “Network of Words and Senses”
  9. 7Because senses in WordNet can be tricky to understand, care should be taken in designing the task so that the Turkers will focus only on the corresponding sense of a word.
    Page 7, “Evaluation 11: Human Evaluation on ConnotationWordNet”
  10. Therefore, we provided the part of speech tag, the WordNet gloss of the selected sense, and a few examples as given in WordNet .
    Page 7, “Evaluation 11: Human Evaluation on ConnotationWordNet”
  11. We have introduced a novel formulation of lexicon induction operating over both words and senses, by exploiting the innate structure between the words and senses as encoded in WordNet .
    Page 9, “Conclusion”

See all papers in Proc. ACL 2014 that mention WordNet.

See all papers in Proc. ACL that mention WordNet.

Back to top.

sentiment lexicon

Appears in 10 sentences as: sentiment lexicon (7) Sentiment lexicons (1) sentiment lexicons (3)
In ConnotationWordNet: Learning Connotation over the Word+Sense Network
  1. We present comprehensive evaluation to demonstrate the quality and utility of the resulting lexicon in comparison to existing connotation and sentiment lexicons .
    Page 1, “Abstract”
  2. ConnotationWordNet is expected to be the superset of a sentiment lexicon , as it is highly likely for any word with positive/negative sentiment to carry connotation of the same polarity.
    Page 5, “Evaluation 1: Agreement with Sentiment Lexicons”
  3. Thus, we use two conventional sentiment lexicons , General Inquirer (GENINQ) (Stone et al., 1966) and MPQA (Wilson et al., 2005b), as surrogates to measure the performance of our inference algorithm.
    Page 5, “Evaluation 1: Agreement with Sentiment Lexicons”
  4. The sentiment lexicons we use as gold standard are small, compared to the size (i.e., number of words) our graphs contain.
    Page 6, “Evaluation 1: Agreement with Sentiment Lexicons”
  5. Sentiment lexicons such as SentiWordNet (Baccianella et al.
    Page 7, “Evaluation 11: Human Evaluation on ConnotationWordNet”
  6. We also extended the seed set with the sentiment lexicon words and denote these runs with E- for ‘Extended’.
    Page 7, “Evaluation 11: Human Evaluation on ConnotationWordNet”
  7. For comparison, we also test the connotation lexicon from (Feng et al., 2013) and the combined sentiment lexicon GENINQ+MPQA.
    Page 8, “Evaluation 111: Sentiment Analysis using ConnotationWordNet”
  8. Several previous approaches explored the use of graph propagation for sentiment lexicon induction (Velikovich et al., 2010) and connotation lexicon
    Page 8, “Related Work”
  9. There have been a number of previous studies that aim to construct a word-level sentiment lexicon (Wiebe et al., 2005; Qiu et al., 2009) and a sense-level sentiment lexicon (Esuli and Sebas-tiani, 2006).
    Page 9, “Related Work”
  10. Although we focus on learning connotative polarity of words and senses in this paper, the same approach would be applicable to constructing a sentiment lexicon as well.
    Page 9, “Related Work”

See all papers in Proc. ACL 2014 that mention sentiment lexicon.

See all papers in Proc. ACL that mention sentiment lexicon.

Back to top.

sentiment analysis

Appears in 7 sentences as: sentiment analysis (8)
In ConnotationWordNet: Learning Connotation over the Word+Sense Network
  1. This word sense issue has been a universal challenge for a range of Natural Language Processing applications, including sentiment analysis .
    Page 1, “Introduction”
  2. Recent studies have shown that it is fruitful to tease out subjectivity and objectivity corresponding to different senses of the same word, in order to improve computational approaches to sentiment analysis (e.g.
    Page 1, “Introduction”
  3. Finally, to show the utility of the resulting lexicon in the context of a concrete sentiment analysis
    Page 8, “Evaluation 111: Sentiment Analysis using ConnotationWordNet”
  4. task, we perform lexicon-based sentiment analysis .
    Page 8, “Evaluation 111: Sentiment Analysis using ConnotationWordNet”
  5. There have been recent studies that address word sense disambiguation issues for sentiment analysis .
    Page 9, “Related Work”
  6. (2009) report a successful empirical result where WSD helps improving sentiment analysis, while Wiebe and Mihalcea (2006) study the distinction between objectivity and subjectivity in each different sense of a word, and their empirical effects in the context of sentiment analysis .
    Page 9, “Related Work”
  7. (2013) share this spirit by targeting more subtle, nuanced sentiment even from those words that would be considered as objective in early studies of sentiment analysis .
    Page 9, “Related Work”

See all papers in Proc. ACL 2014 that mention sentiment analysis.

See all papers in Proc. ACL that mention sentiment analysis.

Back to top.

human judges

Appears in 4 sentences as: human judges (3) humans judge (1)
In ConnotationWordNet: Learning Connotation over the Word+Sense Network
  1. The agreement between the new lexicon and human judges varies between 84% and 86.98%.
    Page 7, “Evaluation 11: Human Evaluation on ConnotationWordNet”
  2. (2005a)) show low agreement rate with human, which is somewhat as expected: human judges in this study are labeling for subtle connotation, not for more explicit sentiment.
    Page 7, “Evaluation 11: Human Evaluation on ConnotationWordNet”
  3. Because different human judges have different notion of scales however, subtle differences are more likely to be noisy.
    Page 8, “Evaluation 11: Human Evaluation on ConnotationWordNet”
  4. Note that there is a difference in how humans judge the orientation and the degree of connotation for a given word out of context, and how the use of such words in context can be perceived as good/bad news.
    Page 8, “Evaluation 111: Sentiment Analysis using ConnotationWordNet”

See all papers in Proc. ACL 2014 that mention human judges.

See all papers in Proc. ACL that mention human judges.

Back to top.

classification task

Appears in 3 sentences as: classification task (3)
In ConnotationWordNet: Learning Connotation over the Word+Sense Network
  1. We formulate the task of learning sense- and word-level connotation lexicon as a graph-based classification task (Sen et al., 2008).
    Page 3, “Pairwise Markov Random Fields and Loopy Belief Propagation”
  2. In this classification task , we denote by 3?
    Page 3, “Pairwise Markov Random Fields and Loopy Belief Propagation”
  3. Problem Definition Having introduced our graph-based classification task and objective formulation, we define our problem more formally.
    Page 3, “Pairwise Markov Random Fields and Loopy Belief Propagation”

See all papers in Proc. ACL 2014 that mention classification task.

See all papers in Proc. ACL that mention classification task.

Back to top.

objective function

Appears in 3 sentences as: objective function (3)
In ConnotationWordNet: Learning Connotation over the Word+Sense Network
  1. We next define our objective function .
    Page 3, “Pairwise Markov Random Fields and Loopy Belief Propagation”
  2. and x to observed ones X (variables with known labels, if any), our objective function is associated with the following joint probability distribution
    Page 3, “Pairwise Markov Random Fields and Loopy Belief Propagation”
  3. Finding the best assignments to unobserved variables in our objective function is the inference problem.
    Page 4, “Pairwise Markov Random Fields and Loopy Belief Propagation”

See all papers in Proc. ACL 2014 that mention objective function.

See all papers in Proc. ACL that mention objective function.

Back to top.

probability distribution

Appears in 3 sentences as: probability distribution (2) probability distributions (1)
In ConnotationWordNet: Learning Connotation over the Word+Sense Network
  1. and x to observed ones X (variables with known labels, if any), our objective function is associated with the following joint probability distribution
    Page 3, “Pairwise Markov Random Fields and Loopy Belief Propagation”
  2. A message mizj is sent from node i to node j and captures the belief of 2' about j, which is the probability distribution over the labels of j; i.e.
    Page 4, “Pairwise Markov Random Fields and Loopy Belief Propagation”
  3. what i “thinks” j’s label is, given the current label of i and the type of the edge that connects i and j. Beliefs refer to marginal probability distributions of nodes over labels; for example denotes the belief of node 2' having label 3),.
    Page 4, “Pairwise Markov Random Fields and Loopy Belief Propagation”

See all papers in Proc. ACL 2014 that mention probability distribution.

See all papers in Proc. ACL that mention probability distribution.

Back to top.

Turker

Appears in 3 sentences as: Turker (2) Turkers (2)
In ConnotationWordNet: Learning Connotation over the Word+Sense Network
  1. We first describe the labeling process of sense-level connotation: We selected 350 polysemous words and one of their senses, and each Turker was asked to rate the connotative polarity of a given word (or of a given sense), from -5 to 5, 0 being the neutral.7 For each word, we asked 5 Turkers to rate and we took the average of the 5 ratings as the connotative intensity score of the word.
    Page 7, “Evaluation 11: Human Evaluation on ConnotationWordNet”
  2. 7Because senses in WordNet can be tricky to understand, care should be taken in designing the task so that the Turkers will focus only on the corresponding sense of a word.
    Page 7, “Evaluation 11: Human Evaluation on ConnotationWordNet”
  3. As an incentive, each Turker was rewarded $0.07 per hit which consists of 10 words to label.
    Page 7, “Evaluation 11: Human Evaluation on ConnotationWordNet”

See all papers in Proc. ACL 2014 that mention Turker.

See all papers in Proc. ACL that mention Turker.

Back to top.

word sense

Appears in 3 sentences as: Word Sense (1) word sense (2)
In ConnotationWordNet: Learning Connotation over the Word+Sense Network
  1. This word sense issue has been a universal challenge for a range of Natural Language Processing applications, including sentiment analysis.
    Page 1, “Introduction”
  2. End-users of such a lexicon may not wish to deal with Word Sense Disam-
    Page 1, “Introduction”
  3. There have been recent studies that address word sense disambiguation issues for sentiment analysis.
    Page 9, “Related Work”

See all papers in Proc. ACL 2014 that mention word sense.

See all papers in Proc. ACL that mention word sense.

Back to top.

PDF available from www.aclweb.org.
Send us an email