A Joint Model of Text and Aspect Ratings for Sentiment Summarization
Titov, Ivan and McDonald, Ryan

Article Structure

Abstract

Online reviews are often accompanied with numerical ratings provided by users for a set of service or product aspects.

Introduction

User generated content represents a unique source of information in which user interface tools have facilitated the creation of an abundance of labeled content, e. g., topics in blogs, numerical product and service ratings in user reviews, and helpfulness rankings in online discussion forums.

The Model

In this section we describe a new statistical model called the Multi-Aspect Sentiment model (MAS), which consists of two parts.

Experiments

In this section we present qualitative and quantitative experiments.

Related Work

There is a growing body of work on summarizing sentiment by extracting and aggregating sentiment over ratable aspects and providing corresponding textual evidence.

Conclusions

In this paper we presented a joint model of text and aspect ratings for extracting text to be displayed in sentiment summaries.

Topics

topic model

Appears in 11 sentences as: topic model (6) topic modeling (1) topic models (3) topics model (1)
In A Joint Model of Text and Aspect Ratings for Sentiment Summarization
  1. The model is at heart a topic model in that it assigns words to a set of induced topics, each of which may represent one particular aspect.
    Page 2, “Introduction”
  2. For example, other topic models can be used as a part of our model and the proposed class of models can be employed in other tasks beyond sentiment summarization, e.g., segmentation of blogs on the basis of topic labels provided by users, or topic discovery on the basis of tags given by users on social bookmarking sites.3
    Page 2, “Introduction”
  3. Distributions of words in each topic were estimated as the proportion of words assigned to each topic, taking into account topic model priors Bgl and Bloc.
    Page 6, “Experiments”
  4. Before applying the topic models we removed punctuation and also removed stop words using the standard list of stop words,8 however, all the words and punctuation were used in the sentiment predictors.
    Page 6, “Experiments”
  5. To combat this problem we first train the sentiment classifiers by assuming that pygm is equal for all the local topics, which effectively ignores the topic model .
    Page 6, “Experiments”
  6. It can be observed that the topic model discovered appropriate topics while the number of topics was below 4.
    Page 7, “Experiments”
  7. A primary advantage of MAS over unsupervised models, such as MG-LDA or clustering, is that topics are linked to a rated aspect, i.e., we know exactly which topics model which aspects.
    Page 7, “Experiments”
  8. We can observe that the topic model , which does not use any explicitly aspect-labeled text, achieves accuracies lower than, but comparable to a supervised model.
    Page 8, “Experiments”
  9. Text excerpts are usually extracted through string matching (Hu and Liu, 2004a; Popescu and Etzioni, 2005), sentence clustering (Gamon et al., 2005), or through topic models (Mei et al., 2007; Titov and McDonald, 2008).
    Page 8, “Related Work”
  10. String extraction methods are limited to fine-grained aspects whereas clustering and topic model approaches must resort to ad-hoc means of labeling clusters or topics.
    Page 8, “Related Work”
  11. Recently, Blei and McAuliffe (2008) proposed an approach for joint sentiment and topic modeling that can be viewed as a supervised LDA (sLDA) model that tries to infer topics appropriate for use in a given classification or regression problem.
    Page 8, “Related Work”

See all papers in Proc. ACL 2008 that mention topic model.

See all papers in Proc. ACL that mention topic model.

Back to top.

LDA

Appears in 8 sentences as: LDA (8)
In A Joint Model of Text and Aspect Ratings for Sentiment Summarization
  1. 2.1 Multi-Grain LDA
    Page 2, “The Model”
  2. The Multi-Grain Latent Dirichlet Allocation model (MG-LDA) is an extension of Latent Dirichlet Allocation ( LDA ) (Blei et al., 2003).
    Page 2, “The Model”
  3. strated in Titov and McDonald (2008), the topics produced by LDA do not correspond to ratable aspects of entities.
    Page 3, “The Model”
  4. As in LDA , the distribution of global topics is fixed for a document (a user review).
    Page 3, “The Model”
  5. Following Titov and McDonald (2008) we use a collapsed Gibbs sampling algorithm that was derived for the MG-LDA model based on the Gibbs sampling method proposed for LDA in (Griffiths and Steyvers, 2004).
    Page 5, “The Model”
  6. A naive application of this technique to LDA would imply that both assignments of topics to words 2 and distributions 6 and go should be sampled.
    Page 5, “The Model”
  7. It is difficult to compare our model to other unsupervised systems such as MG—LDA or LDA .
    Page 8, “Experiments”
  8. Recently, Blei and McAuliffe (2008) proposed an approach for joint sentiment and topic modeling that can be viewed as a supervised LDA (sLDA) model that tries to infer topics appropriate for use in a given classification or regression problem.
    Page 8, “Related Work”

See all papers in Proc. ACL 2008 that mention LDA.

See all papers in Proc. ACL that mention LDA.

Back to top.

Gibbs sampling

Appears in 6 sentences as: Gibbs sampler (2) Gibbs sampling (5)
In A Joint Model of Text and Aspect Ratings for Sentiment Summarization
  1. Following Titov and McDonald (2008) we use a collapsed Gibbs sampling algorithm that was derived for the MG-LDA model based on the Gibbs sampling method proposed for LDA in (Griffiths and Steyvers, 2004).
    Page 5, “The Model”
  2. Gibbs sampling is an example of a Markov Chain Monte Carlo algorithm (Geman and Geman, 1984).
    Page 5, “The Model”
  3. In Gibbs sampling , variables are sequentially sampled from their distributions conditioned on all other variables in the model.
    Page 5, “The Model”
  4. However, (Griffiths and Steyvers, 2004) demonstrated that an efficient collapsed Gibbs sampler can be constructed, where only assignments 2 need to be sampled, whereas the dependency on distributions 6 and go can be integrated out analytically.
    Page 5, “The Model”
  5. For details on computing gradients for log-linear graphical models with Gibbs sampling we refer the reader to (Neal, 1992).
    Page 5, “The Model”
  6. This factor is proportional to the conditional distribution used in the Gibbs sampler of the MG—LDA model (Titov and McDonald, 2008).
    Page 5, “The Model”

See all papers in Proc. ACL 2008 that mention Gibbs sampling.

See all papers in Proc. ACL that mention Gibbs sampling.

Back to top.

sentiment classifiers

Appears in 4 sentences as: Sentiment classification (1) sentiment classification (1) sentiment classifiers (2)
In A Joint Model of Text and Aspect Ratings for Sentiment Summarization
  1. The second problem is sentiment classification .
    Page 1, “Introduction”
  2. Sentiment classification is a well studied problem (Wiebe, 2000; Pang et a1., 2002; Tumey, 2002) and in many domains users explicitly
    Page 1, “Introduction”
  3. Therefore, the use of the aspect sentiment classifiers based only on the words assigned to the corresponding topics is problematic.
    Page 4, “The Model”
  4. To combat this problem we first train the sentiment classifiers by assuming that pygm is equal for all the local topics, which effectively ignores the topic model.
    Page 6, “Experiments”

See all papers in Proc. ACL 2008 that mention sentiment classifiers.

See all papers in Proc. ACL that mention sentiment classifiers.

Back to top.

co-occurrence

Appears in 3 sentences as: co-occurrence (3)
In A Joint Model of Text and Aspect Ratings for Sentiment Summarization
  1. First, ratable aspects normally represent coherent topics which can be potentially discovered from co-occurrence information in the text.
    Page 2, “Introduction”
  2. Importantly, the fact that windows overlap permits the model to exploit a larger co-occurrence domain.
    Page 3, “The Model”
  3. The first factor, 77%,” expresses a preference for topics likely from the co-occurrence information, whereas the second one, pig, favors the choice of topics which are predictive of the observable sentiment ratings.
    Page 6, “Experiments”

See all papers in Proc. ACL 2008 that mention co-occurrence.

See all papers in Proc. ACL that mention co-occurrence.

Back to top.

labeled data

Appears in 3 sentences as: labeled data (3)
In A Joint Model of Text and Aspect Ratings for Sentiment Summarization
  1. Our model achieves high accuracy, without any explicitly labeled data except the user provided opinion ratings.
    Page 1, “Abstract”
  2. When labeled data exists, this problem can be solved effectively using a wide variety of methods available for text classification and information extraction (Manning and Schutze, 1999).
    Page 2, “Introduction”
  3. However, labeled data is often hard to come by, especially when one considers all possible domains of products and services.
    Page 2, “Introduction”

See all papers in Proc. ACL 2008 that mention labeled data.

See all papers in Proc. ACL that mention labeled data.

Back to top.

n-gram

Appears in 3 sentences as: n-gram (5)
In A Joint Model of Text and Aspect Ratings for Sentiment Summarization
  1. In this model the distribution of the overall sentiment rating you is based on all the n-gram features of a review text.
    Page 4, “The Model”
  2. Then the distribution of ya, for every rated aspect a, can be computed from the distribution of you and from any n-gram feature where at least one word in the n-gram is assigned to the associated aspect topic (7“ 2 Zoe, 2 = a).
    Page 4, “The Model”
  3. b; is the bias term which regulates the prior distribution P(ya = y), f iterates through all the n-grams, J1me and Jif are common weights and aspect-specific weights for n-gram feature f. pinz is equal to a fraction of words in n-gram feature f assigned to the aspect topic (7“ = [00, z = a).
    Page 5, “The Model”

See all papers in Proc. ACL 2008 that mention n-gram.

See all papers in Proc. ACL that mention n-gram.

Back to top.

n-grams

Appears in 3 sentences as: n-grams (3)
In A Joint Model of Text and Aspect Ratings for Sentiment Summarization
  1. The first score is computed on the basis of all the n-grams , but using a common set of weights independent of the aspect a.
    Page 4, “The Model”
  2. Another score is computed only using n-grams associated with the related topic, but an aspect-specific set of weights is used in this computation.
    Page 4, “The Model”
  3. b; is the bias term which regulates the prior distribution P(ya = y), f iterates through all the n-grams , J1me and Jif are common weights and aspect-specific weights for n-gram feature f. pinz is equal to a fraction of words in n-gram feature f assigned to the aspect topic (7“ = [00, z = a).
    Page 5, “The Model”

See all papers in Proc. ACL 2008 that mention n-grams.

See all papers in Proc. ACL that mention n-grams.

Back to top.

PDF available from www.aclweb.org.
Send us an email