Multilingual Affect Polarity and Valence Prediction in Metaphor-Rich Texts
Kozareva, Zornitsa

Article Structure

Abstract

Metaphor is an important way of conveying the affect of people, hence understanding how people use metaphors to convey affect is important for the communication between individuals and increases cohesion if the perceived affect of the concrete example is the same for the two individuals.

Introduction

Metaphor is a figure of speech in which a word or phrase that ordinarily designates one thing is used to designate another, thus making an implicit comparison (Lakoff and Johnson, 1980; Martin, 1988; Wilks, 2007).

Related Work

A substantial body of work has been done on determining the affect (sentiment analysis) of texts (Kim and Hovy, 2004; Strapparava and Mihalcea, 2007; Wiebe and Cardie, 2005; Yessenalina and Cardie, 2011; Breck et al., 2007).

Metaphors

Although there are different views on metaphor in linguistics and philosophy (Black, 1962; Lakoff and Johnson, 1980; Gentner, 1983; Wilks, 2007), the common among all approaches is the idea of an interconceptual mapping that underlies the production of metaphorical expressions.

Task A: Polarity Classification

4.1 Problem Formulation

Task B: Valence Prediction

5.1 Problem Formulation

Conclusion

People use metaphor-rich language to express affect and often affect is expressed through the usage of metaphors.

Topics

regression model

Appears in 10 sentences as: Regression Model (1) regression model (6) regression models (4)
In Multilingual Affect Polarity and Valence Prediction in Metaphor-Rich Texts
  1. 5.2 Regression Model
    Page 6, “Task B: Valence Prediction”
  2. Full details of the regression model and its implementation are beyond the scope of this paper; for more details see (Scho'lkopf and Smola, 2001; Smola et al., 2003).
    Page 6, “Task B: Valence Prediction”
  3. Evaluation Measures: To evaluate the quality of the valence prediction model, we compare the actual valence score of the metaphor given by human annotators denoted with 3/ against those valence scores predicted by the regression model denoted with ac.
    Page 7, “Task B: Valence Prediction”
  4. We estimate the goodness of the regression model calculating boZth the grregltion coef-.
    Page 7, “Task B: Valence Prediction”
  5. Similarly the smaller the mean squared error rate, the better the regression model fits the valence predictions to the actual score.
    Page 7, “Task B: Valence Prediction”
  6. For each language and information source we built separate valence prediction regression models .
    Page 7, “Task B: Valence Prediction”
  7. The Farsi and Russian regression models are based only on n-gram features, while the English and Spanish regression models have both n-gram and LIWC features.
    Page 7, “Task B: Valence Prediction”
  8. This means that the LIWC based valence regression model approximates the predicted values better to those of the human annotators.
    Page 7, “Task B: Valence Prediction”
  9. To summarize, in this section we have defined the task of valence prediction of metaphor-rich texts and we have described a regression model for its solution.
    Page 7, “Task B: Valence Prediction”
  10. Through experiments carried out on the developed datasets, we showed that the proposed polarity classification and valence regression models significantly improve baselines (from 11.90% to 39.69% depending on the language) and work well for all four languages.
    Page 8, “Conclusion”

See all papers in Proc. ACL 2013 that mention regression model.

See all papers in Proc. ACL that mention regression model.

Back to top.

n-gram

Appears in 9 sentences as: N-gram (3) n-gram (7)
In Multilingual Affect Polarity and Valence Prediction in Metaphor-Rich Texts
  1. 4.4 N-gram Evaluation and Results
    Page 4, “Task A: Polarity Classification”
  2. N-gram features are widely used in a variety of classification tasks, therefore we also use them in our polarity classification task.
    Page 4, “Task A: Polarity Classification”
  3. Figure 2 shows a study of the influence of the different information sources and their combination with n-gram features for English.
    Page 4, “Task A: Polarity Classification”
  4. For each information source (metaphor, context, source, target and their combinations), we built a separate n-gram feature set and model, which was evaluated on 10-fold cross validation.
    Page 4, “Task A: Polarity Classification”
  5. Table 2 shows the influence of the information sources for Spanish, Russian and Farsi with the n-gram features.
    Page 4, “Task A: Polarity Classification”
  6. Table 2: N-gram features, F-scores on 10-fold validation for Spanish, Russian and Farsi
    Page 4, “Task A: Polarity Classification”
  7. LIWC Repository: In addition to the n-gram features, we also used the Linguistic Inquiry and Word Count (LIWC) repository (Tausczik and Pennebaker, 2010), which has 64 word categories corresponding to different classes like emotional states, psychological processes, personal concerns among other.
    Page 4, “Task A: Polarity Classification”
  8. The learned lessons from this study are: (1) for n-gram usage, the larger the context of the metaphor, the better the classification accuracy becomes; (2) if present source and target information can further boost the performance of the classifiers; (3) LIWC is a useful resource for polarity identification in metaphor-rich texts; (4) analyzing the usages of tense like past vs. present and pronouns are important triggers for positive and negative polarity of metaphors; (5) some categories like family, social presence indicate positive polarity, while others like inhibition, anger and swear words are indicative of negative affect; (6) the built models significantly outperform majority baselines.
    Page 6, “Task A: Polarity Classification”
  9. The Farsi and Russian regression models are based only on n-gram features, while the English and Spanish regression models have both n-gram and LIWC features.
    Page 7, “Task B: Valence Prediction”

See all papers in Proc. ACL 2013 that mention n-gram.

See all papers in Proc. ACL that mention n-gram.

Back to top.

human annotators

Appears in 8 sentences as: human annotation (1) human annotators (7)
In Multilingual Affect Polarity and Valence Prediction in Metaphor-Rich Texts
  1. To conduct our study, we use human annotators to collect metaphor-rich texts (Shutova and Teufel, 2010) and tag each metaphor with its corresponding polarity (Posi-tive/Negative) and valence [—3, +3] scores.
    Page 2, “Metaphors”
  2. In our study, the source and target domains are provided by the human annotators who agree on these definitions, however the source and target can be also automatically generated by an interpretation system or a concept mapper.
    Page 3, “Task A: Polarity Classification”
  3. Evaluation Measures: To evaluate the quality of the valence prediction model, we compare the actual valence score of the metaphor given by human annotators denoted with 3/ against those valence scores predicted by the regression model denoted with ac.
    Page 7, “Task B: Valence Prediction”
  4. To conduct our valence prediction study, we used the same human annotators from the polarity classification task for each one of the English, Spanish, Russian and Farsi languages.
    Page 7, “Task B: Valence Prediction”
  5. This means that the LIWC based valence regression model approximates the predicted values better to those of the human annotators .
    Page 7, “Task B: Valence Prediction”
  6. The MSE for English and Spanish is the lowest, meaning that the prediction is the closest to those of the human annotators .
    Page 7, “Task B: Valence Prediction”
  7. The learned lessons from this study are: (l) valence prediction is a much harder task than polarity classification both for human annotation and for the machine learning algorithms; (2) the obtained results showed that despite its difficulty this is still a plausible problem; (3) similarly to the polarity classification task, valence prediction with LIWC is improved when shorter contexts (the metaphor/source/target information source) are considered.
    Page 7, “Task B: Valence Prediction”
  8. From the two tasks, the valence prediction problem was more challenging both for the human annotators and the automated system.
    Page 8, “Conclusion”

See all papers in Proc. ACL 2013 that mention human annotators.

See all papers in Proc. ACL that mention human annotators.

Back to top.

machine learning

Appears in 6 sentences as: machine learning (6)
In Multilingual Affect Polarity and Valence Prediction in Metaphor-Rich Texts
  1. Multiple techniques have been employed, from various machine learning classifiers, to clustering and topic models.
    Page 2, “Related Work”
  2. We tested five different machine learning algorithms such as Nave Bayes, SVM with polynomial kernel, SVM with RBF kernel, AdaBoost and Stacking, out of which AdaBoost performed the best.
    Page 3, “Task A: Polarity Classification”
  3. For our metaphor polarity task, we use LIWC’s statistics of all 64 categories and feed this information as features for the machine learning classifiers.
    Page 4, “Task A: Polarity Classification”
  4. To summarize, in this section we have defined the task of polarity classification and we have presented a machine learning solution.
    Page 6, “Task A: Polarity Classification”
  5. The learned lessons from this study are: (l) valence prediction is a much harder task than polarity classification both for human annotation and for the machine learning algorithms; (2) the obtained results showed that despite its difficulty this is still a plausible problem; (3) similarly to the polarity classification task, valence prediction with LIWC is improved when shorter contexts (the metaphor/source/target information source) are considered.
    Page 7, “Task B: Valence Prediction”
  6. We have conducted exhaustive evaluation with multiple machine learning classifiers and different features sets spanning from lexical information to psychological categories developed by (Tausczik and Pennebaker, 2010).
    Page 8, “Conclusion”

See all papers in Proc. ACL 2013 that mention machine learning.

See all papers in Proc. ACL that mention machine learning.

Back to top.

feature set

Appears in 5 sentences as: feature set (2) feature sets (2) features sets (1)
In Multilingual Affect Polarity and Valence Prediction in Metaphor-Rich Texts
  1. We studied the influence of unigrams, bigrams and a combination of the two, and saw that the best performing feature set consists of the combination of unigrams and bigrams.
    Page 4, “Task A: Polarity Classification”
  2. For each information source (metaphor, context, source, target and their combinations), we built a separate n-gram feature set and model, which was evaluated on 10-fold cross validation.
    Page 4, “Task A: Polarity Classification”
  3. We have used different feature sets and information sources to solve the task.
    Page 6, “Task A: Polarity Classification”
  4. We have studied different feature sets and information sources to solve the task.
    Page 7, “Task B: Valence Prediction”
  5. We have conducted exhaustive evaluation with multiple machine learning classifiers and different features sets spanning from lexical information to psychological categories developed by (Tausczik and Pennebaker, 2010).
    Page 8, “Conclusion”

See all papers in Proc. ACL 2013 that mention feature set.

See all papers in Proc. ACL that mention feature set.

Back to top.

classification task

Appears in 4 sentences as: classification task (4) classification tasks (1)
In Multilingual Affect Polarity and Valence Prediction in Metaphor-Rich Texts
  1. N-gram features are widely used in a variety of classification tasks, therefore we also use them in our polarity classification task .
    Page 4, “Task A: Polarity Classification”
  2. To conduct our valence prediction study, we used the same human annotators from the polarity classification task for each one of the English, Spanish, Russian and Farsi languages.
    Page 7, “Task B: Valence Prediction”
  3. We used the same features for the regression task as we have used in the classification task .
    Page 7, “Task B: Valence Prediction”
  4. The learned lessons from this study are: (l) valence prediction is a much harder task than polarity classification both for human annotation and for the machine learning algorithms; (2) the obtained results showed that despite its difficulty this is still a plausible problem; (3) similarly to the polarity classification task , valence prediction with LIWC is improved when shorter contexts (the metaphor/source/target information source) are considered.
    Page 7, “Task B: Valence Prediction”

See all papers in Proc. ACL 2013 that mention classification task.

See all papers in Proc. ACL that mention classification task.

Back to top.

bigrams

Appears in 3 sentences as: bigrams (4)
In Multilingual Affect Polarity and Valence Prediction in Metaphor-Rich Texts
  1. We studied the influence of unigrams, bigrams and a combination of the two, and saw that the best performing feature set consists of the combination of unigrams and bigrams .
    Page 4, “Task A: Polarity Classification”
  2. In this paper, we will refer from now on to n-grams as the combination of unigrams and bigrams .
    Page 4, “Task A: Polarity Classification”
  3. Those include n-grams (unigrams, bigrams and combination of the two), LIWC scores.
    Page 7, “Task B: Valence Prediction”

See all papers in Proc. ACL 2013 that mention bigrams.

See all papers in Proc. ACL that mention bigrams.

Back to top.

significantly outperform

Appears in 3 sentences as: significantly outperform (3)
In Multilingual Affect Polarity and Valence Prediction in Metaphor-Rich Texts
  1. 0 We have conducted in depth experimental evaluation and showed that the developed methods significantly outperform baseline methods.
    Page 2, “Introduction”
  2. As we can see from Figure 5 that all classifiers significantly outperform the majority base-
    Page 5, “Task A: Polarity Classification”
  3. The learned lessons from this study are: (1) for n-gram usage, the larger the context of the metaphor, the better the classification accuracy becomes; (2) if present source and target information can further boost the performance of the classifiers; (3) LIWC is a useful resource for polarity identification in metaphor-rich texts; (4) analyzing the usages of tense like past vs. present and pronouns are important triggers for positive and negative polarity of metaphors; (5) some categories like family, social presence indicate positive polarity, while others like inhibition, anger and swear words are indicative of negative affect; (6) the built models significantly outperform majority baselines.
    Page 6, “Task A: Polarity Classification”

See all papers in Proc. ACL 2013 that mention significantly outperform.

See all papers in Proc. ACL that mention significantly outperform.

Back to top.

unigrams

Appears in 3 sentences as: unigrams (4)
In Multilingual Affect Polarity and Valence Prediction in Metaphor-Rich Texts
  1. We studied the influence of unigrams, bigrams and a combination of the two, and saw that the best performing feature set consists of the combination of unigrams and bigrams.
    Page 4, “Task A: Polarity Classification”
  2. In this paper, we will refer from now on to n-grams as the combination of unigrams and bigrams.
    Page 4, “Task A: Polarity Classification”
  3. Those include n-grams ( unigrams , bigrams and combination of the two), LIWC scores.
    Page 7, “Task B: Valence Prediction”

See all papers in Proc. ACL 2013 that mention unigrams.

See all papers in Proc. ACL that mention unigrams.

Back to top.

PDF available from www.aclweb.org.
Send us an email