ReNew: A Semi-Supervised Framework for Generating Domain-Specific Lexicons and Sentiment Analysis
Zhang, Zhe and Singh, Munindar P.

Article Structure

Abstract

The sentiment captured in opinionated text provides interesting and valuable information for social media services.

Introduction

Automatically extracting sentiments from user-generated opinionated text is important in building social media services.

Background

Let us introduce some of the key terminology used in ReNew.

Framework

Bootstrapping Process

Experiments

To assess ReNew’s effectiveness, we prepare two hotel review datasets crawled from TripadVisor.

Related Work

Two bodies of work are relevant.

Conclusions and Future Work

The leading lexical approaches to sentiment analysis from text are based on fixed lexicons that are painstakingly built by hand.

Topics

sentiment lexicon

Appears in 23 sentences as: sentiment lexicon (19) sentiment lexicons (8)
In ReNew: A Semi-Supervised Framework for Generating Domain-Specific Lexicons and Sentiment Analysis
  1. We propose a semi-supervised framework for generating a domain-specific sentiment lexicon and inferring sentiments at the segment level.
    Page 1, “Abstract”
  2. Our framework can greatly reduce the human effort for building a domain-specific sentiment lexicon with high quality.
    Page 1, “Abstract”
  3. Specifically, in our evaluation, working with just 20 manually labeled reviews, it generates a domain-specific sentiment lexicon that yields weighted average F-Measure gains of 3%.
    Page 1, “Abstract”
  4. High-quality sentiment lexicons can improve the performance of sentiment analysis models over general-purpose lexicons (Choi and Cardie, 2009).
    Page 1, “Introduction”
  5. (2) ReNew leverages the relationships between consecutive segments to infer their sentiments and automatically generates a domain-specific sentiment lexicon in a semi-su-pervised fashion.
    Page 1, “Introduction”
  6. (3) To capture the contextual sentiment of words, ReNew uses dependency relation pairs as the basic elements in the generated sentiment lexicon .
    Page 1, “Introduction”
  7. ReNew can greatly reduce the human effort for building a domain-specific sentiment lexicon with high quality.
    Page 2, “Introduction”
  8. Specifically, in our evaluation on two real datasets, working with just 20 manually labeled reviews, ReNew generates a domain-specific sentiment lexicon that yields weighted average F-Measure gains of 3%.
    Page 2, “Introduction”
  9. A domain-specific sentiment lexicon con-
    Page 2, “Background”
  10. Given a set of reviews, the tasks of sentiment analysis in ReNew are (l) splitting each review into segments, (2) associating each segment with a sentiment label (positive, neutral, negative), and (3) automatically generating a domain-specific sentiment lexicon .
    Page 2, “Background”
  11. a general sentiment lexicon and a small labeled training dataset.
    Page 3, “Framework”

See all papers in Proc. ACL 2014 that mention sentiment lexicon.

See all papers in Proc. ACL that mention sentiment lexicon.

Back to top.

sentiment classification

Appears in 18 sentences as: Sentiment Classification (1) sentiment classification (17)
In ReNew: A Semi-Supervised Framework for Generating Domain-Specific Lexicons and Sentiment Analysis
  1. Our sentiment classification model achieves approximately 1% greater accuracy than a state-of—the-art approach based on elementary discourse units.
    Page 1, “Abstract”
  2. In respect to sentiment classification , Pang et al.
    Page 1, “Introduction”
  3. (1) Instead of using sentences, ReNew uses segments as the basic units for sentiment classification .
    Page 1, “Introduction”
  4. Additionally, our sentiment classification model achieves approximately 1% greater accuracy than a state-of-the-art approach based on elementary discourse units (Lazaridou et al., 2013).
    Page 2, “Introduction”
  5. Knowledge from this initial training set is not sufficient to build an accurate sentiment classification model or to generate a domain-specific sentiment lexicon.
    Page 3, “Framework”
  6. for training a CRF-based sentiment classification model.
    Page 6, “Framework”
  7. We train 15 sentiment classification models using all basic features and their combinations.
    Page 6, “Experiments”
  8. To this end, we train and compare sentiment classification models using three configurations.
    Page 6, “Experiments”
  9. This is important because to compare only the lexicons’ impact on sentiment classification , we need to avoid the effect of other factors, such as syntax, transition cues, and so on.
    Page 7, “Experiments”
  10. 4.4 Lexicon Generation and Sentiment Classification
    Page 7, “Experiments”
  11. Our fourth experiment evaluates the robustness of ReNew’s lexicon generation process as well as the performance of the sentiment classification models using these lexicons.
    Page 7, “Experiments”

See all papers in Proc. ACL 2014 that mention sentiment classification.

See all papers in Proc. ACL that mention sentiment classification.

Back to top.

dependency relation

Appears in 10 sentences as: Dependency relation (2) dependency relation (5) dependency relations (3)
In ReNew: A Semi-Supervised Framework for Generating Domain-Specific Lexicons and Sentiment Analysis
  1. (3) To capture the contextual sentiment of words, ReNew uses dependency relation pairs as the basic elements in the generated sentiment lexicon.
    Page 1, “Introduction”
  2. After classifying the sentiment of Segment 5 as NEG, we associate the dependency relation pairs {“sign”, “wear”} and {“sign”, “tear”} with that sentiment.
    Page 2, “Introduction”
  3. A dependency relation defines a binary relation that describes whether a pairwise syntactic relation among two words holds in a sentence.
    Page 2, “Background”
  4. In ReNew, we exploit the Stanford typed dependency representations (de Marneffe et al., 2006) that use triples to formalize dependency relations .
    Page 2, “Background”
  5. tains three lists of dependency relations , associated respectvely with positive, neutral, or negative sentiment.
    Page 2, “Background”
  6. Third, the lexicon generator determines which newly learned dependency relation triples to promote to the lexicon.
    Page 3, “Framework”
  7. For each sentiment, the Triple Extractor (TE) extracts candidate dependency relation triples using a novel rule-based approach.
    Page 4, “Framework”
  8. Table l: Dependency relation types used in extracting (E) and domain-specific lexicon (L).
    Page 4, “Framework”
  9. Dependency relation : The lexicon generated by ReNew uses the Stanford typed dependency representation as its structure.
    Page 6, “Framework”
  10. To do this, we first divide all features into four basic feature sets: T (transition cues), P (punctuations, special name-entities, and segment positions), G (grammar), and 0D (opinion words and dependency relations ).
    Page 6, “Experiments”

See all papers in Proc. ACL 2014 that mention dependency relation.

See all papers in Proc. ACL that mention dependency relation.

Back to top.

EDUs

Appears in 10 sentences as: EDUs (10)
In ReNew: A Semi-Supervised Framework for Generating Domain-Specific Lexicons and Sentiment Analysis
  1. First, the basic units of their model are elementary discourse units ( EDUs ) from Rhetorical Structure Theory (RST) (Mann and Thompson, 1988).
    Page 8, “Experiments”
  2. Second, their model considers the forward relationship between EDUs , whereas ReNew captures both forward and backward relationship between segments.
    Page 8, “Experiments”
  3. Third, they use a generative model to capture the transition distributions over EDUs whereas ReNew uses a discriminative model to capture the transition sequences of segments.
    Page 8, “Experiments”
  4. EDUs are defined as minimal units of text and consider many more relations than the two types
    Page 8, “Experiments”
  5. We posit that EDUs are too fine-grained for sentiment analysis.
    Page 9, “Experiments”
  6. Consider the following sentence from Lazaridou et al.’s dataset with its EDUs identified.
    Page 9, “Experiments”
  7. Although Lazaridou et al.’s model can capture the forward relationship between any two consecutive EDUs , it cannot handle such cases because their model assumes that each EDU is associated with a topic and a sentiment.
    Page 9, “Experiments”
  8. Just to compare with Lazaridou et al., we apply our sentiment labeling component at the level of EDUs .
    Page 9, “Experiments”
  9. Their labeled dataset contains 65 reviews, corresponding to 1,541 EDUs .
    Page 9, “Experiments”
  10. As shown in Table 5, ReNew outperforms their approach on their dataset: Although ReNew is not optimized for EDUs , it achieves better accuracy.
    Page 9, “Experiments”

See all papers in Proc. ACL 2014 that mention EDUs.

See all papers in Proc. ACL that mention EDUs.

Back to top.

F-score

Appears in 8 sentences as: F-score (11)
In ReNew: A Semi-Supervised Framework for Generating Domain-Specific Lexicons and Sentiment Analysis
  1. Accuracy Macro F-score Micro F-score
    Page 7, “Experiments”
  2. Figure 8 reports the accuracy, macro F-score, and micro F-score .
    Page 7, “Experiments”
  3. It shows that the BR learner produces better accuracy and a micro F-score than the FR learner but a slightly worse macro F-score .
    Page 7, “Experiments”
  4. Macro F-score
    Page 8, “Experiments”
  5. Figure 10: Macro F-score with different lexicons.
    Page 8, “Experiments”
  6. Figure 10 and 11 show the pairwise comparisons of macro and micro F-score together with the results of the paired t—tests.
    Page 8, “Experiments”
  7. Micro F-score O U!
    Page 8, “Experiments”
  8. Figure 11: Micro F-score with different lexicons.
    Page 8, “Experiments”

See all papers in Proc. ACL 2014 that mention F-score.

See all papers in Proc. ACL that mention F-score.

Back to top.

sentiment analysis

Appears in 6 sentences as: sentiment analysis (6)
In ReNew: A Semi-Supervised Framework for Generating Domain-Specific Lexicons and Sentiment Analysis
  1. High-quality sentiment lexicons can improve the performance of sentiment analysis models over general-purpose lexicons (Choi and Cardie, 2009).
    Page 1, “Introduction”
  2. Given a set of reviews, the tasks of sentiment analysis in ReNew are (l) splitting each review into segments, (2) associating each segment with a sentiment label (positive, neutral, negative), and (3) automatically generating a domain-specific sentiment lexicon.
    Page 2, “Background”
  3. They show that this pattern is a useful indicator for sentiment analysis .
    Page 4, “Framework”
  4. We posit that EDUs are too fine-grained for sentiment analysis .
    Page 9, “Experiments”
  5. The leading lexical approaches to sentiment analysis from text are based on fixed lexicons that are painstakingly built by hand.
    Page 9, “Conclusions and Future Work”
  6. In future work, we plan to apply ReNew to additional sentiment analysis problems such as review quality analysis and sentiment summarization.
    Page 9, “Conclusions and Future Work”

See all papers in Proc. ACL 2014 that mention sentiment analysis.

See all papers in Proc. ACL that mention sentiment analysis.

Back to top.

models trained

Appears in 5 sentences as: model trained (2) models trained (3)
In ReNew: A Semi-Supervised Framework for Generating Domain-Specific Lexicons and Sentiment Analysis
  1. Figure 7 shows the results of a 10-fold cross validation on the 200-review dataset (light grey bars show the accuracy of the model trained without using transition cue features).
    Page 6, “Experiments”
  2. In particular, the accuracy of 0D is markedly improved by adding T. The model trained using all the feature sets yields the best accuracy.
    Page 6, “Experiments”
  3. We compare models trained using (1) our domain-specific lexicon, (2) Affective Norms for English Words (ANEW) (Bradley and Lang, 1999), and (3) Linguistic Inquiry and Word Count (LIWC) (Tausczik and Pennebaker, 2010).
    Page 7, “Experiments”
  4. To evaluate the benefit of using domain-specific sentiment lexicons, we train ten sentiment classification models using the ten lexicons and then compare them, pairwise, against models trained with the general sentiment lexicon LIWC.
    Page 7, “Experiments”
  5. Each group of bars represents the accuracy of two sentiment classification models trained using LIWC (CRFs-General) and the generated domain-specific lexicon (CRFs-Domain), respectively.
    Page 7, “Experiments”

See all papers in Proc. ACL 2014 that mention models trained.

See all papers in Proc. ACL that mention models trained.

Back to top.

semi-supervised

Appears in 5 sentences as: semi-supervised (5)
In ReNew: A Semi-Supervised Framework for Generating Domain-Specific Lexicons and Sentiment Analysis
  1. We propose a semi-supervised framework for generating a domain-specific sentiment lexicon and inferring sentiments at the segment level.
    Page 1, “Abstract”
  2. To address the above shortcomings of lexicon and granularity, we propose a semi-supervised framework named ReNew.
    Page 1, “Introduction”
  3. Rao and Ravichandran (2009) formalize the problem of sentiment detection as a semi-supervised label propagation problem in a graph.
    Page 9, “Related Work”
  4. Esuli and Sebas-tiani (2006) use a set of classifiers in a semi-supervised fashion to iteratively expand a manu-
    Page 9, “Related Work”
  5. (2011) introduce a semi-supervised approach that uses recursive autoencoders to learn the hierarchical structure and sentiment distribution of a sentence.
    Page 9, “Related Work”

See all papers in Proc. ACL 2014 that mention semi-supervised.

See all papers in Proc. ACL that mention semi-supervised.

Back to top.

cross validation

Appears in 4 sentences as: cross validation (4)
In ReNew: A Semi-Supervised Framework for Generating Domain-Specific Lexicons and Sentiment Analysis
  1. Figure 7 shows the results of a 10-fold cross validation on the 200-review dataset (light grey bars show the accuracy of the model trained without using transition cue features).
    Page 6, “Experiments”
  2. Table 4 shows the results obtained by 10-fold cross validation .
    Page 7, “Experiments”
  3. Each pairwise comparison is evaluated on a testing dataset with 10-fold cross validation .
    Page 7, “Experiments”
  4. Follow the same training and testing regimen (10-fold cross validation ), we compare ReNew with their model.
    Page 9, “Experiments”

See all papers in Proc. ACL 2014 that mention cross validation.

See all papers in Proc. ACL that mention cross validation.

Back to top.

fine-grained

Appears in 3 sentences as: fine-grained (3)
In ReNew: A Semi-Supervised Framework for Generating Domain-Specific Lexicons and Sentiment Analysis
  1. However, these changes can be successfully exploited for inferring fine-grained sentiments.
    Page 1, “Introduction”
  2. Segments can be shorter than sentences and therefore help capture fine-grained sentiments.
    Page 1, “Introduction”
  3. We posit that EDUs are too fine-grained for sentiment analysis.
    Page 9, “Experiments”

See all papers in Proc. ACL 2014 that mention fine-grained.

See all papers in Proc. ACL that mention fine-grained.

Back to top.

iteratively

Appears in 3 sentences as: iteratively (3)
In ReNew: A Semi-Supervised Framework for Generating Domain-Specific Lexicons and Sentiment Analysis
  1. ReNew starts with LIWC and a labeled dataset and generates ten lexicons and sentiment classification models by iteratively learning 4,017 unlabeled reviews without any human guidance.
    Page 8, “Experiments”
  2. Hu and Liu (2004), manually collect a small set of sentiment words and expand it iteratively by searching synonyms and antonyms in WordNet (Miller, 1995).
    Page 9, “Related Work”
  3. Esuli and Sebas-tiani (2006) use a set of classifiers in a semi-supervised fashion to iteratively expand a manu-
    Page 9, “Related Work”

See all papers in Proc. ACL 2014 that mention iteratively.

See all papers in Proc. ACL that mention iteratively.

Back to top.

Rule-Based

Appears in 3 sentences as: Rule-Based (1) Rule-based (1) rule-based (1)
In ReNew: A Semi-Supervised Framework for Generating Domain-Specific Lexicons and Sentiment Analysis
  1. 3.1 Rule-Based Segmentation Algorithm
    Page 3, “Framework”
  2. Algorithm 1 Rule-based segmentation.
    Page 3, “Framework”
  3. For each sentiment, the Triple Extractor (TE) extracts candidate dependency relation triples using a novel rule-based approach.
    Page 4, “Framework”

See all papers in Proc. ACL 2014 that mention Rule-Based.

See all papers in Proc. ACL that mention Rule-Based.

Back to top.

PDF available from www.aclweb.org.
Send us an email