Aspect Extraction with Automated Prior Knowledge Learning
Chen, Zhiyuan and Mukherjee, Arjun and Liu, Bing

Article Structure

Abstract

Aspect extraction is an important task in sentiment analysis.

Introduction

Aspect extraction aims to extract target entities and their aspects (or attributes) that people have expressed opinions upon (Hu and Liu, 2004, Liu, 2012).

Related Work

Aspect extraction has been studied by many researchers in sentiment analysis (Liu, 2012, Pang and Lee, 2008), e.g., using supervised sequence labeling or classification (Choi and Cardie, 2010, Jakob and Gurevych, 2010, Kobayashi et al., 2007, Li et al., 2010, Yang and Cardie, 2013) and using word frequency and syntactic patterns (Hu and Liu, 2004, Ku et al., 2006, Liu et al., 2013, Popescu and Etzioni, 2005, Qiu et al., 2011, So-masundaran and Wiebe, 2009, Wu et al., 2009, Xu et al., 2013, Yu et al., 2011, Zhao et al., 2012, Zhou et al., 2013, Zhuang et al., 2006).

Overall Algorithm

This section introduces the proposed overall algorithm.

Learning Quality Knowledge

This section details Step 1 in the overall algorithm, which has three sub-steps: running LDA (or AKL) on each domain corpus, clustering the resulting topics, and mining frequent patterns from the topics in each cluster.

AKL: Using the Learned Knowledge

We now present the proposed topic model AKL, which is able to use the automatically learned knowledge to improve aspect extraction.

Experiments

This section evaluates and compares the proposed AKL model with three baseline models LDA, MC-LDA, and GK—LDA.

Conclusions

This paper proposed an advanced aspect extraction framework which can learn knowledge automatically from a large number of review corpora and exploit the learned knowledge in extracting more coherent aspects.

Topics

LDA

Appears in 26 sentences as: LDA (26)
In Aspect Extraction with Automated Prior Knowledge Learning
  1. Traditional topic models such as LDA (Blei et al., 2003) and pLSA (Hofmann, 1999) are unsupervised methods for extracting latent topics in text documents.
    Page 1, “Introduction”
  2. We thus propose to first use LDA to learn topics/aspects from each individual domain and then discover the shared aspects (or topics) and aspect terms among a subset of domains.
    Page 2, “Introduction”
  3. We propose a method to solve this problem, which also results in a new topic model, called AKL (Automated Knowledge LDA ), whose inference can exploit the automatically learned prior knowledge and handle the issues of incorrect knowledge to produce superior aspects.
    Page 2, “Introduction”
  4. Lines 3 and 5 run LDA on each review domain corpus Di 6 D L to generate a set of aspects/topics A, (lines 2, 4, and 6-9 will be discussed below).
    Page 3, “Overall Algorithm”
  5. Scalability: the proposed algorithm is naturally scalable as both LDA and AKL run on each domain independently.
    Page 3, “Overall Algorithm”
  6. This section details Step 1 in the overall algorithm, which has three sub-steps: running LDA (or AKL) on each domain corpus, clustering the resulting topics, and mining frequent patterns from the topics in each cluster.
    Page 3, “Learning Quality Knowledge”
  7. Since running LDA is simple, we will not discuss it further.
    Page 3, “Learning Quality Knowledge”
  8. After running LDA (or AKL) on each domain corpus, a set of topics is obtained.
    Page 3, “Learning Quality Knowledge”
  9. The purpose of clustering is to group raw topics from a topic model ( LDA or AKL) into clusters.
    Page 4, “Learning Quality Knowledge”
  10. To compute this distribution, instead of considering how well 21- matches with w,- only (as in LDA ), we also consider two other factors:
    Page 5, “AKL: Using the Learned Knowledge”
  11. This section evaluates and compares the proposed AKL model with three baseline models LDA , MC-LDA, and GK—LDA.
    Page 6, “Experiments”

See all papers in Proc. ACL 2014 that mention LDA.

See all papers in Proc. ACL that mention LDA.

Back to top.

topic models

Appears in 26 sentences as: topic model (8) Topic modeling (1) topic modeling (2) Topic models (1) topic models (15)
In Aspect Extraction with Automated Prior Knowledge Learning
  1. Topic modeling is a popular method for the task.
    Page 1, “Abstract”
  2. However, unsupervised topic models often generate incoherent aspects.
    Page 1, “Abstract”
  3. Such knowledge can then be used by a topic model to discover more coherent aspects.
    Page 1, “Abstract”
  4. Recently, topic models have been extensively applied to aspect extraction because they can perform both subtasks at the same time while other
    Page 1, “Introduction”
  5. Traditional topic models such as LDA (Blei et al., 2003) and pLSA (Hofmann, 1999) are unsupervised methods for extracting latent topics in text documents.
    Page 1, “Introduction”
  6. However, researchers have shown that fully unsupervised models often produce incoherent topics because the objective functions of topic models do not always correlate well with human judgments (Chang et al., 2009).
    Page 1, “Introduction”
  7. To tackle the problem, several semi-supervised topic models, also called knowledge-based topic models , have been proposed.
    Page 1, “Introduction”
  8. We propose a method to solve this problem, which also results in a new topic model , called AKL (Automated Knowledge LDA), whose inference can exploit the automatically learned prior knowledge and handle the issues of incorrect knowledge to produce superior aspects.
    Page 2, “Introduction”
  9. It proposes to exploit the big data to learn prior knowledge and leverage the knowledge in topic models to extract more coherent aspects.
    Page 2, “Introduction”
  10. It proposes a new inference mechanism for topic modeling , which can handle incorrect knowledge in aspect extraction.
    Page 2, “Introduction”
  11. To extract and group aspects simultaneously, topic models have been applied by researchers (Branavan et al., 2008, Brody and Elhadad, 2010, Chen et al., 2013b, Fang and Huang, 2012, He et al., 2011, Jo and Oh, 2011, Kim et al., 2013, Lazaridou et al., 2013, Li et al., 2011, Lin and He, 2009, Lu et al., 2009, Lu et al., 2012, Lu and Zhai, 2008, Mei et al., 2007, Moghaddam and Ester, 2013, Mukherjee and Liu, 2012, Sauper and Barzilay, 2013, Titov and McDonald, 2008, Wang et al., 2010, Zhao et al., 2010).
    Page 2, “Related Work”

See all papers in Proc. ACL 2014 that mention topic models.

See all papers in Proc. ACL that mention topic models.

Back to top.

Gibbs sampler

Appears in 8 sentences as: Gibbs Sampler (2) Gibbs sampler (7) Gibbs sampling (1)
In Aspect Extraction with Automated Prior Knowledge Learning
  1. Most importantly, due to the use of the new form of knowledge, AKL’s inference mechanism ( Gibbs sampler ) is entirely different from that of MC-LDA (Section 5.2), which results in superior performances (Section 6).
    Page 5, “AKL: Using the Learned Knowledge”
  2. In short, our modeling contributions are (1) the capability of handling more expressive knowledge in the form of clusters, (2) a novel Gibbs sampler to deal with inappropriate knowledge.
    Page 5, “AKL: Using the Learned Knowledge”
  3. 5.2 The Gibbs Sampler
    Page 5, “AKL: Using the Learned Knowledge”
  4. Instead of assigning weights to each piece of knowledge as a fixed prior in (Chen et al., 2013a), we propose a new Gibbs sampler, which can dynamically balance the use of prior knowledge and the information in the corpus during the Gibbs sampling iterations.
    Page 5, “AKL: Using the Learned Knowledge”
  5. We adopt a Blocked Gibbs sampler (Rosen-Zvi et al., 2010) as it improves convergence and reduces autocorrelation when the variables (topic 2 and cluster c in AKL) are highly related.
    Page 5, “AKL: Using the Learned Knowledge”
  6. term 212,- in each document, we jointly sample a topic 21- and cluster Ci (containing 21),) based on the conditional distribution in Gibbs sampler (will be detailed in Equation 4).
    Page 5, “AKL: Using the Learned Knowledge”
  7. Putting together Equations 1, 2 and 3 into a blocked Gibbs Sampler, we can define the following sampling distribution in Gibbs sampler so that it provides helpful guidance in determining the usefulness of the prior knowledge and in selecting the semantically coherent topic.
    Page 6, “AKL: Using the Learned Knowledge”
  8. Note that although the above Gibbs sampler is able to distinguish useful knowledge from wrong knowledge, it is possible that there is no cluster corroborates for a particular term.
    Page 6, “AKL: Using the Learned Knowledge”

See all papers in Proc. ACL 2014 that mention Gibbs sampler.

See all papers in Proc. ACL that mention Gibbs sampler.

Back to top.

human judgments

Appears in 3 sentences as: human judges (1) human judgments (2)
In Aspect Extraction with Automated Prior Knowledge Learning
  1. However, researchers have shown that fully unsupervised models often produce incoherent topics because the objective functions of topic models do not always correlate well with human judgments (Chang et al., 2009).
    Page 1, “Introduction”
  2. However, perpleXity on the held-out test set does not reflect the semantic coherence of topics and may be contrary to human judgments (Chang et al., 2009).
    Page 7, “Experiments”
  3. As our objective is to discover more coherent aspects, we recruited two human judges .
    Page 7, “Experiments”

See all papers in Proc. ACL 2014 that mention human judgments.

See all papers in Proc. ACL that mention human judgments.

Back to top.

knowledge base

Appears in 3 sentences as: knowledge base (3)
In Aspect Extraction with Automated Prior Knowledge Learning
  1. A good knowledge base should have the capacity of handling this ambiguity.
    Page 4, “Learning Quality Knowledge”
  2. Such patterns compose our knowledge base as shown below.
    Page 4, “Learning Quality Knowledge”
  3. As the knowledge is extracted from each cluster individually, we represent our knowledge base as a set of clusters, where each cluster consists of a set of frequent 2-patterns mined using FPM, e. g.,
    Page 4, “Learning Quality Knowledge”

See all papers in Proc. ACL 2014 that mention knowledge base.

See all papers in Proc. ACL that mention knowledge base.

Back to top.

semantically related

Appears in 3 sentences as: semantic relationship (1) semantically related (2)
In Aspect Extraction with Automated Prior Knowledge Learning
  1. Each cluster contains semantically related topics likely to indicate the same real-world aspect.
    Page 4, “Learning Quality Knowledge”
  2. Using two terms in a set is sufficient to cover the semantic relationship of the terms belonging to the same aspect.
    Page 4, “Learning Quality Knowledge”
  3. We further employ the Generalized Plya urn (GPU) model (Mahmoud, 2008) which was shown to be effective in leveraging semantically related words (Chen et al., 2013a, Chen et al., 2013b, Mimno et al., 2011).
    Page 6, “AKL: Using the Learned Knowledge”

See all papers in Proc. ACL 2014 that mention semantically related.

See all papers in Proc. ACL that mention semantically related.

Back to top.