A Non-negative Matrix Tri-factorization Approach to Sentiment Classification with Lexical Prior Knowledge
Li, Tao and Zhang, Yi and Sindhwani, Vikas

Article Structure

Abstract

Sentiment classification refers to the task of automatically identifying whether a given piece of text expresses positive or negative opinion towards a subject at hand.

Introduction

Web 2.0 platforms such as blogs, discussion forums and other such social media have now given a public voice to every consumer.

Related Work

We point the reader to a recent book (Pang and Lee, 2008) for an in-depth survey of literature on sentiment analysis.

Background

3.1 Basic Matrix Factorization Model

Incorporating Lexical Knowledge

We used a sentiment lexicon generated by the IBM India Research Labs that was developed for other text mining applications (Ramakrishnan et al., 2003).

Semi-Supervised Learning With Lexical Knowledge

So far our models have made no demands on human effort, other than unsupervised collection of the term-document matrix and a onetime effort in compiling a domain-independent sentiment lexicon.

Experiments

6.1 Datasets Description

Conclusion

The primary contribution of this paper is to propose and benchmark new methodologies for sentiment analysis.

Topics

sentiment analysis

Appears in 12 sentences as: Sentiment Analysis (2) sentiment analysis (10)
In A Non-negative Matrix Tri-factorization Approach to Sentiment Classification with Lexical Prior Knowledge
  1. In Section 4, we present a constrained model and computational algorithm for incorporating lexical knowledge in sentiment analysis .
    Page 2, “Introduction”
  2. We point the reader to a recent book (Pang and Lee, 2008) for an in-depth survey of literature on sentiment analysis .
    Page 2, “Related Work”
  3. In this section, we briskly cover related work to position our contributions appropriately in the sentiment analysis and machine learning literature.
    Page 2, “Related Work”
  4. (Goldberg and Zhu, 2006) adapt semi-supervised graph-based methods for sentiment analysis but do not incorporate lexical prior knowledge in the form of labeled features.
    Page 2, “Related Work”
  5. We also note the very recent work of (Sindhwani and Melville, 2008) which proposes a dual-supervision model for semi-supervised sentiment analysis .
    Page 3, “Related Work”
  6. Finally, recent efforts have also looked at transfer learning mechanisms for sentiment analysis , e.g., see (Blitzer et al., 2007).
    Page 3, “Related Work”
  7. Movies Reviews: This is a popular dataset in sentiment analysis literature (Pang et al., 2002).
    Page 5, “Experiments”
  8. 6.2 Sentiment Analysis with Lexical Knowledge
    Page 6, “Experiments”
  9. 6.3 Sentiment Analysis with Dual Supervision
    Page 7, “Experiments”
  10. Both of these methods have been widely used in sentiment analysis .
    Page 7, “Experiments”
  11. The primary contribution of this paper is to propose and benchmark new methodologies for sentiment analysis .
    Page 8, “Conclusion”

See all papers in Proc. ACL 2009 that mention sentiment analysis.

See all papers in Proc. ACL that mention sentiment analysis.

Back to top.

semi-supervised

Appears in 10 sentences as: semi-supervised (11)
In A Non-negative Matrix Tri-factorization Approach to Sentiment Classification with Lexical Prior Knowledge
  1. However, the treatment of such dictionaries as forms of prior knowledge that can be incorporated in machine learning models is a relatively less explored topic; even lesser so in conjunction with semi-supervised models that attempt to utilize un-
    Page 1, “Introduction”
  2. In this regard, our model brings two interrelated but distinct themes from machine learning to bear on this problem: semi-supervised learning and learning from labeled features.
    Page 2, “Related Work”
  3. (Goldberg and Zhu, 2006) adapt semi-supervised graph-based methods for sentiment analysis but do not incorporate lexical prior knowledge in the form of labeled features.
    Page 2, “Related Work”
  4. We also note the very recent work of (Sindhwani and Melville, 2008) which proposes a dual-supervision model for semi-supervised sentiment analysis.
    Page 3, “Related Work”
  5. Therefore, the semi-supervised learning with lexical knowledge can be described as
    Page 5, “Semi-Supervised Learning With Lexical Knowledge”
  6. Thus the algorithm for semi-supervised learning with lexical knowledge based on our matrix factorization framework, referred as SSMFLK, consists of an iterative procedure using the above three rules until convergence.
    Page 5, “Semi-Supervised Learning With Lexical Knowledge”
  7. Genuinely unlabeled posts for Political and Lotus were used for semi-supervised learning experiments in section 6.3; they were not used in section 6.2 on the effect of lexical prior knowledge.
    Page 6, “Experiments”
  8. Robustness to Vocabulary Size High dimensionality and noise can have profound impact on the comparative performance of clustering and semi-supervised learning algorithms.
    Page 7, “Experiments”
  9. The natural question is whether the presence of lexical constraints leads to better semi-supervised models.
    Page 7, “Experiments”
  10. In this section, we compare our method (SSMFLK) with the following three semi-supervised approaches: (1) The algorithm proposed in (Zhou et al., 2003) which conducts semi-supervised learning with local and global consistency (Consistency Method); (2) Zhu et al.’s harmonic Gaussian field method coupled with the Class Mass Normalization (Harmonic-CMN) (Zhu et al., 2003); and (3) Green’s function learning algorithm (Green’s Function) proposed in (Ding et al., 2007).
    Page 7, “Experiments”

See all papers in Proc. ACL 2009 that mention semi-supervised.

See all papers in Proc. ACL that mention semi-supervised.

Back to top.

sentiment lexicon

Appears in 10 sentences as: Sentiment Lexicon (1) sentiment lexicon (10)
In A Non-negative Matrix Tri-factorization Approach to Sentiment Classification with Lexical Prior Knowledge
  1. Treated as a set of labeled features, the sentiment lexicon is incorporated as one set of constraints that enforce domain-independent prior knowledge.
    Page 2, “Introduction”
  2. Our goal now is to bias these models with constraints incorporating (a) labels of features (coming from a domain-independent sentiment lexicon ), and (b) labels of documents for the purposes of domain-specific adaptation.
    Page 3, “Background”
  3. We used a sentiment lexicon generated by the IBM India Research Labs that was developed for other text mining applications (Ramakrishnan et al., 2003).
    Page 3, “Incorporating Lexical Knowledge”
  4. So far our models have made no demands on human effort, other than unsupervised collection of the term-document matrix and a onetime effort in compiling a domain-independent sentiment lexicon .
    Page 5, “Semi-Supervised Learning With Lexical Knowledge”
  5. Our interest in the first set of experiments is to explore the benefits of incorporating a sentiment lexicon over unsupervised approaches.
    Page 6, “Experiments”
  6. These methods do not make use of the sentiment lexicon .
    Page 6, “Experiments”
  7. Size of Sentiment Lexicon We also investigate the effects of the size of the sentiment lexicon on the performance of our model.
    Page 7, “Experiments”
  8. Figure 2: MFLK accuracy as size of sentiment lexicon (i.e., number of words in the lexicon) increases on the four datasets
    Page 7, “Experiments”
  9. We now assume that together with labeled features from the sentiment lexicon , we also have access to a few labeled documents.
    Page 7, “Experiments”
  10. Learning Domain-Specific Connotations In our first set of experiments, we incorporated the sentiment lexicon in our models and learnt the sentiment orientation of words and documents via F ,0 factors respectively.
    Page 7, “Experiments”

See all papers in Proc. ACL 2009 that mention sentiment lexicon.

See all papers in Proc. ACL that mention sentiment lexicon.

Back to top.

machine learning

Appears in 7 sentences as: machine learning (7)
In A Non-negative Matrix Tri-factorization Approach to Sentiment Classification with Lexical Prior Knowledge
  1. These methodologies are likely to be rooted in natural language processing and machine learning techniques.
    Page 1, “Introduction”
  2. Automatically classifying the sentiment expressed in a blog around selected topics of interest is a canonical machine learning task in this discussion.
    Page 1, “Introduction”
  3. However, the treatment of such dictionaries as forms of prior knowledge that can be incorporated in machine learning models is a relatively less explored topic; even lesser so in conjunction with semi-supervised models that attempt to utilize un-
    Page 1, “Introduction”
  4. In this section, we briskly cover related work to position our contributions appropriately in the sentiment analysis and machine learning literature.
    Page 2, “Related Work”
  5. In this regard, our model brings two interrelated but distinct themes from machine learning to bear on this problem: semi-supervised learning and learning from labeled features.
    Page 2, “Related Work”
  6. Most work in machine learning literature on utilizing labeled features has focused on using them to generate weakly labeled examples that are then used for standard supervised learning: (Schapire et al., 2002) propose one such framework for boosting logistic regression; (Wu and Srihari, 2004) build a modified SVM and (Liu et al., 2004) use a combination of clustering and EM based methods to instantiate similar frameworks.
    Page 2, “Related Work”
  7. In particular, the use of SVMs in (Pang et al., 2002) initially sparked interest in using machine learning methods for sentiment classification.
    Page 7, “Experiments”

See all papers in Proc. ACL 2009 that mention machine learning.

See all papers in Proc. ACL that mention machine learning.

Back to top.

SVM

Appears in 6 sentences as: SVM (6)
In A Non-negative Matrix Tri-factorization Approach to Sentiment Classification with Lexical Prior Knowledge
  1. Most work in machine learning literature on utilizing labeled features has focused on using them to generate weakly labeled examples that are then used for standard supervised learning: (Schapire et al., 2002) propose one such framework for boosting logistic regression; (Wu and Srihari, 2004) build a modified SVM and (Liu et al., 2004) use a combination of clustering and EM based methods to instantiate similar frameworks.
    Page 2, “Related Work”
  2. We also compare the results of SSMFLK with those of two supervised classification methods: Support Vector Machine ( SVM ) and Naive Bayes.
    Page 7, “Experiments”
  3. —e— Consistency Method 0.51 —I— Homonic—CMN + Green Function 0.45 7 + SVM
    Page 8, “Experiments”
  4. 6 f —9— SSMFLK 0.5 - —6— Consistency Method -—I— Homonic—CMN + Green Function 0.4 ' + SVM + Naive Bayes
    Page 8, “Experiments”
  5. 0.45 7 —9— Consistency Method * —I— Homonic—CMN 0-4 ’ + Green Function 0 35 _ + SVM
    Page 8, “Experiments”
  6. —6— Consistency Method —l— Homonic—CMN + Green Function 0.45 + SVM
    Page 8, “Experiments”

See all papers in Proc. ACL 2009 that mention SVM.

See all papers in Proc. ACL that mention SVM.

Back to top.

unlabeled data

Appears in 4 sentences as: unlabeled data (4)
In A Non-negative Matrix Tri-factorization Approach to Sentiment Classification with Lexical Prior Knowledge
  1. We propose a novel approach to learn from lexical prior knowledge in the form of domain-independent sentiment-laden terms, in conjunction with domain-dependent unlabeled data and a few labeled documents.
    Page 1, “Abstract”
  2. The goal of the former theme is to learn from few labeled examples by making use of unlabeled data , while the goal of the latter theme is to utilize weak prior knowledge about term-class affinities (e.g., the term “awful” indicates negative sentiment and therefore may be considered as a negatively labeled feature).
    Page 2, “Related Work”
  3. It should be noted, that this list was constructed without a specific domain in mind; which is further motivation for using training examples and unlabeled data to learn domain specific connotations.
    Page 4, “Incorporating Lexical Knowledge”
  4. To more effectively utilize unlabeled data and induce domain-specific adaptation of our models, several extensions are possible: facilitating learning from related domains, incorporating hyperlinks between documents, incorporating synonyms or co-occurences between words etc.
    Page 8, “Conclusion”

See all papers in Proc. ACL 2009 that mention unlabeled data.

See all papers in Proc. ACL that mention unlabeled data.

Back to top.

Sentiment classification

Appears in 3 sentences as: Sentiment classification (1) sentiment classification (1) sentiment classifier (1)
In A Non-negative Matrix Tri-factorization Approach to Sentiment Classification with Lexical Prior Knowledge
  1. Sentiment classification refers to the task of automatically identifying whether a given piece of text expresses positive or negative opinion towards a subject at hand.
    Page 1, “Abstract”
  2. A two-tier scheme (Pang and Lee, 2004) where sentences are first classified as subjective versus objective, and then applying the sentiment classifier on only the subjective sentences further improves performance.
    Page 2, “Related Work”
  3. In particular, the use of SVMs in (Pang et al., 2002) initially sparked interest in using machine learning methods for sentiment classification .
    Page 7, “Experiments”

See all papers in Proc. ACL 2009 that mention Sentiment classification.

See all papers in Proc. ACL that mention Sentiment classification.

Back to top.

PDF available from www.aclweb.org.
Send us an email