SciSurf: Index of 'Comparing Multi-label Classification with Reinforcement Learning for Summarisation of Time-series Data'

Comparing Multi-label Classification with Reinforcement Learning for Summarisation of Time-series Data

Gkatzia, Dimitra and Hastie, Helen and Lemon, Oliver

Published in Proc. ACL, 2014

Article Structure

Abstract

We present a novel approach for automatic report generation from time-series data, in the context of student feedback generation.

Introduction

Summarisation of time-series data refers to the task of automatically generating text from variables whose values change over time.

Related Work

Natural Language Generation from time-series data has been investigated for various tasks such as weather forecast generation (Belz and Kow, 2010; Angeli et al., 2010; Sripada et al., 2004), report generation from clinical data (Hunter et al.,

Data

The dataset consists of 37 instances referring to the activities of 26 students.

Methodology

In this section, the content selection task and the suggested multi-label classification approach are presented.

Evaluation

Firstly, we performed a preliminary evaluation on classification methods, comparing our proposed ML classification with multiple iterated classification approaches.

Results

Topics

classification task

Appears in 4 sentences as: classification task (4) classification tasks (1)

In Comparing Multi-label Classification with Reinforcement Learning for Summarisation of Time-series Data

We frame content selection as a simple classification task : given a set of time-series data, decide for each template whether it should be included in a summary or not.
Page 1, “Introduction”
Collective content selection (Barzilay and Lapata, 2004) is similar to our proposed method in that it is a classification task that predicts the templates from the same instance simultaneously.
Page 2, “Related Work”
Problem transformation approaches (Tsoumakas and Katakis, 2007) transform the ML classification task into one or more simple classification tasks .
Page 3, “Related Work”
The LP method transforms the ML task, into one single-label multi-class classification task , where the possible set of predicted variables for the transformed class is the powerset of labels present in the original dataset.
Page 4, “Methodology”

See all papers in Proc. ACL 2014 that mention classification task.

See all papers in Proc. ACL that mention classification task.

F-score

Appears in 4 sentences as: F-score (4)

In Comparing Multi-label Classification with Reinforcement Learning for Summarisation of Time-series Data

We show that this method generates output closer to the feedback that lecturers actually generated, achieving 3.5% higher accuracy and 15% higher F-score than multiple simple classifiers that keep a history of selected templates.
Page 1, “Abstract”
The accuracy, the weighted precision, the weighted recall, and the weighted F-score of the classifiers are shown in Table 3.
Page 6, “Evaluation”
It was found that in 10-fold cross validation RAkEL performs significantly better in all these automatic measures (accuracy = 76.95%, F-score = 85.50%).
Page 6, “Evaluation”
Remarkably, ML achieves more than 10% higher F-score than the other methods (Table 3).
Page 6, “Evaluation”

See all papers in Proc. ACL 2014 that mention F-score.

See all papers in Proc. ACL that mention F-score.

Rule-based

Appears in 4 sentences as: Rule-based (2) rule-based (2)

In Comparing Multi-label Classification with Reinforcement Learning for Summarisation of Time-series Data

Development of time-series generation systems (Section 4.2, Section 5.3): ML system, RL system, Rule-based and Random system 5.
Page 4, “Methodology”
In order to reduce the confounding variables, we kept the ordering of content in all systems the same, by adopting the ordering of the rule-based system.
Page 6, “Evaluation”
Rule-based System: generates summaries based on Content Selection rules derived by working with a L&T expert and a student (Gkatzia et al., 2013).
Page 6, “Evaluation”
from left to right: ML system, RL, rule-based and randor.
Page 8, “Results”

See all papers in Proc. ACL 2014 that mention Rule-based.

See all papers in Proc. ACL that mention Rule-based.