Trainable Generation of Big-Five Personality Styles through Data-Driven Parameter Estimation
Mairesse, François and Walker, Marilyn

Article Structure

Abstract

Previous work on statistical language generation has primarily focused on grammaticality and naturalness, scoring generation possibilities according to a language model or user feedback.

Introduction

Over the last 20 years, statistical language models (SLMs) have been used successfully in many tasks in natural language processing, and the data available for modeling has steadily grown (Lapata and Keller, 2005).

Parameter Estimation Models

The data-driven parameter estimation method consists of a development phase and a generation phase (Section 3).

Evaluation Experiment

The generation phase of our parameter estimation SNLG method consists of the following steps:

Conclusion

We present a new method for generating linguistic variation projecting multiple personality traits continuously, by combining and extending previous research in statistical natural language generation (Paiva and Evans, 2005; Rambow et al., 2001; Isard et al., 2006; Mairesse and Walker, 2007).

Topics

rule-based

Appears in 17 sentences as: Rule-Based (1) Rule-based (3) rule-based (15)
In Trainable Generation of Big-Five Personality Styles through Data-Driven Parameter Estimation
  1. Another line of work has produced handcrafted rule-based systems to control specific stylistic dimensions, such as politeness and personality.
    Page 1, “Abstract”
  2. We compare our performance to a rule-based generator in the same domain.
    Page 1, “Abstract”
  3. Langkilde and Knight (1998) first applied SLMs to statistical natural language generation (SNLG), showing that high quality paraphrases can be generated from an underspecified representation of meaning, by first applying a very undercon-strained, rule-based overgeneration phase, whose outputs are then ranked by an SLM scoring phase.
    Page 1, “Introduction”
  4. In previous work, we presented PERSONAGE, a psychologically-informed rule-based generator based on the Big Five personality model, and we showed that PERSONAGE can project extreme personality on the extraversion scale, i.e.
    Page 1, “Introduction”
  5. Section 3.2 shows that humans accurately perceive the intended variation, and Section 3.3 compares PERSONAGE-PE (trained) with PERSONAGE ( rule-based ; Mairesse and Walker, 2007).
    Page 2, “Introduction”
  6. We test a Naive Bayes classifier (NB), a j48 decision tree (J48), a nearest-neighbor classifier using one neighbor (NN), a Java implementation of the RIPPER rule-based learner (J RIP), the AdaBoost boosting algorithm (ADA), and a support vector machines classifier with a linear kernel (SVM).
    Page 4, “Parameter Estimation Models”
  7. Q3: How does PERSONAGE-PE compare to PERSONAGE, a psychologically-informed rule-based generator for projecting extreme personality?
    Page 6, “Evaluation Experiment”
  8. comparison with rule-based results in Section 3.3 suggests that this is not because conscientiousness cannot be exhibited in our domain or manifested in a single utterance, so perhaps this arises from differing perceptions of conscientiousness between the expert and naive judges.
    Page 7, “Evaluation Experiment”
  9. 3.3 Comparison with Rule-Based Generation PERSONAGE is a rule-based personality generator based on handcrafted parameter settings derived from psychological studies.
    Page 7, “Evaluation Experiment”
  10. Method Rule-based Learned parameters Trait Low High Low High Extraversion 2.96 5.98 3.69 o 5.05 o Emotional stability 3.29 5.96 3.75 4.75 o Agreeableness 3.41 5.66 3.42 4.33 o Conscientiousness 3.71 5.53 4.16 4.15 o Openness to experience 2.89 4.21 3.71 o 4.06
    Page 7, “Evaluation Experiment”
  11. 0,0 significant increase or decrease of the variation range over the average rule-based ratings (p < .05, two-tailed)
    Page 7, “Evaluation Experiment”

See all papers in Proc. ACL 2008 that mention rule-based.

See all papers in Proc. ACL that mention rule-based.

Back to top.

SVM

Appears in 6 sentences as: SVM (6)
In Trainable Generation of Big-Five Personality Styles through Data-Driven Parameter Estimation
  1. Continuous parameters are modeled with a linear regression model (LR), an M5’ model tree (M5), and a model based on support vector machines with a linear kernel ( SVM ).
    Page 4, “Parameter Estimation Models”
  2. We test a Naive Bayes classifier (NB), a j48 decision tree (J48), a nearest-neighbor classifier using one neighbor (NN), a Java implementation of the RIPPER rule-based learner (J RIP), the AdaBoost boosting algorithm (ADA), and a support vector machines classifier with a linear kernel ( SVM ).
    Page 4, “Parameter Estimation Models”
  3. Figure 3: SVM model with a linear kernel predicting the CONTENT POLARITY parameter.
    Page 4, “Parameter Estimation Models”
  4. Continuous parameters | LR M5 SVM |
    Page 5, “Parameter Estimation Models”
  5. Binary parameters | NB J48 NN ADA SVM |
    Page 5, “Parameter Estimation Models”
  6. the most accurately, with the SVM model in Figure 3 producing a correlation of .47 with the true parameter values.
    Page 5, “Parameter Estimation Models”

See all papers in Proc. ACL 2008 that mention SVM.

See all papers in Proc. ACL that mention SVM.

Back to top.

human judges

Appears in 5 sentences as: human judges (3) human judgments (2)
In Trainable Generation of Big-Five Personality Styles through Data-Driven Parameter Estimation
  1. Another thread investigates SNLG scoring models trained using higher-level linguistic features to replicate human judgments of utterance quality (Rambow et al., 2001; Nakatsu and White, 2006; Stent and Guo, 2005).
    Page 1, “Introduction”
  2. Collects human judgments rating the personality of each utterance;
    Page 2, “Parameter Estimation Models”
  3. We then evaluate the output utterances using naive human judges to rate their perceived personality and naturalness.
    Page 6, “Evaluation Experiment”
  4. Table 5 shows several sample outputs and the mean personality ratings from the human judges .
    Page 6, “Evaluation Experiment”
  5. Additionally, our data-driven approach can be applied to any dimension that is meaningful to human judges , and it provides an elegant way to project multiple dimensions simultaneously, by including the relevant dimensions as features of the parameter models’ training data.
    Page 8, “Conclusion”

See all papers in Proc. ACL 2008 that mention human judges.

See all papers in Proc. ACL that mention human judges.

Back to top.

models trained

Appears in 5 sentences as: Model Training (1) models trained (3) models’ training (1)
In Trainable Generation of Big-Five Personality Styles through Data-Driven Parameter Estimation
  1. Another thread investigates SNLG scoring models trained using higher-level linguistic features to replicate human judgments of utterance quality (Rambow et al., 2001; Nakatsu and White, 2006; Stent and Guo, 2005).
    Page 1, “Introduction”
  2. 2.3 Statistical Model Training
    Page 4, “Parameter Estimation Models”
  3. Q1: Is the personality projected by models trained on
    Page 6, “Evaluation Experiment”
  4. Additionally, our data-driven approach can be applied to any dimension that is meaningful to human judges, and it provides an elegant way to project multiple dimensions simultaneously, by including the relevant dimensions as features of the parameter models’ training data.
    Page 8, “Conclusion”
  5. In terms of our research questions in Section 3.1, we show that models trained on expert judges to project multiple traits in a single utterance generate utterances whose personality is recognized by naive judges.
    Page 8, “Conclusion”

See all papers in Proc. ACL 2008 that mention models trained.

See all papers in Proc. ACL that mention models trained.

Back to top.

regression model

Appears in 4 sentences as: regression model (2) regression models (2)
In Trainable Generation of Big-Five Personality Styles through Data-Driven Parameter Estimation
  1. Continuous parameters are modeled with a linear regression model (LR), an M5’ model tree (M5), and a model based on support vector machines with a linear kernel (SVM).
    Page 4, “Parameter Estimation Models”
  2. As regression models can extrapolate beyond the [0, 1] interval, the output parameter values are truncated if needed—at generation time—before being sent to the base generator.
    Page 4, “Parameter Estimation Models”
  3. Table 3: Pearson’s correlation between parameter model predictions and continuous parameter values, for different regression models .
    Page 5, “Parameter Estimation Models”
  4. Models of the PERIOD aggregation operation also perform well, with a linear regression model yielding a correlation of .36 when realizing a justification, and .27 when contrasting two propositions.
    Page 5, “Parameter Estimation Models”

See all papers in Proc. ACL 2008 that mention regression model.

See all papers in Proc. ACL that mention regression model.

Back to top.

natural language

Appears in 3 sentences as: natural language (3)
In Trainable Generation of Big-Five Personality Styles through Data-Driven Parameter Estimation
  1. Over the last 20 years, statistical language models (SLMs) have been used successfully in many tasks in natural language processing, and the data available for modeling has steadily grown (Lapata and Keller, 2005).
    Page 1, “Introduction”
  2. Langkilde and Knight (1998) first applied SLMs to statistical natural language generation (SNLG), showing that high quality paraphrases can be generated from an underspecified representation of meaning, by first applying a very undercon-strained, rule-based overgeneration phase, whose outputs are then ranked by an SLM scoring phase.
    Page 1, “Introduction”
  3. We present a new method for generating linguistic variation projecting multiple personality traits continuously, by combining and extending previous research in statistical natural language generation (Paiva and Evans, 2005; Rambow et al., 2001; Isard et al., 2006; Mairesse and Walker, 2007).
    Page 8, “Conclusion”

See all papers in Proc. ACL 2008 that mention natural language.

See all papers in Proc. ACL that mention natural language.

Back to top.

PDF available from www.aclweb.org.
Send us an email