Article Structure
Abstract
This paper presents a pilot study of opinion summarization on conversations.
Introduction
Both sentiment analysis (opinion recognition) and summarization have been well studied in recent years in the natural language processing (NLP) community.
Related Work
Research in document summarization has been well established over the past decades.
Corpus Creation
Though there are many annotated data sets for the research of speech summarization and sentiment analysis, there is no corpus available for opinion summarization on spontaneous speech.
Opinion Summarization Methods
Automatic summarization can be divided into extractive summarization and abstractive summarization.
Experiments
5.1 Experimental Setup
Conclusion and Future Work
This paper investigates two unsupervised methods in opinion summarization on spontaneous conversations by incorporating topic score and sentiment score in existing summarization techniques.
Acknowledgments
The authors thank Julia Hirschberg and Ani Nenkova for useful discussions.
Topics
graph-based
- The second one is a graph-based method, which incorporates topic and sentiment information, as well as additional information about sentence-to-sentence relations extracted based on dialogue structure.Page 1, “Abstract”
- In particular, we find that incorporating dialogue structure in the graph-based method contributes to the improved system performance.Page 1, “Abstract”
- widely used in extractive summarization: sentence-ranking and graph-based methods.Page 2, “Introduction”
- Furthermore, in the graph-based method, we propose to better incorporate the dialogue structure information in the graph in order to select salient summary utterances.Page 2, “Introduction”
- The second one is a graph-based method, which incorporates the dialogue structure in ranking.Page 4, “Opinion Summarization Methods”
- 4.2 Graph-based SummarizationPage 5, “Opinion Summarization Methods”
- Graph-based methods have been widely used in document summarization.Page 5, “Opinion Summarization Methods”
- Here in our proposed graph-based method, we introduce connections between the two speakers, so that the adjacency pairs between them can be utilized to extract salient utterances.Page 6, “Opinion Summarization Methods”
- In the graph-based method, the best parameters are Asim = 0,)\adj = 0.3,Arel = 0.4,Asent = 0.3.Page 6, “Experiments”
- This is different from graph-based summarization systems for text domains.Page 6, “Experiments”
- When compared to abstractive reference summaries, the graph-based method is slightly better.Page 7, “Experiments”
See all papers in Proc. ACL 2011 that mention graph-based.
See all papers in Proc. ACL that mention graph-based.
Back to top.
cosine similarity
- 0 sim(s, D) is the cosine similarity between DA 3 and all the utterances in the dialogue from the same speaker, D. It measures the relevancy of s to the entire dialogue from the target speaker.Page 4, “Opinion Summarization Methods”
- For cosine similarity measure, we use TF*IDF (term frequency, inverse document frequency) term weighting.Page 4, “Opinion Summarization Methods”
- is modeled as an adjacency matrix, where each node represents a sentence, and the weight of the edge between each pair of sentences is their similarity ( cosine similarity is typically used).Page 5, “Opinion Summarization Methods”
- where C is the set of all DAs in the dialogue; REL(8, topic) and sentiment(s) are the same as those in the above sentence ranking method; sim(s, v) is the cosine similarity between two DAs s and 2).Page 5, “Opinion Summarization Methods”
- Our experiments show that both methods are able to improve the baseline approach, and we find that the cosine similarity between utterances or between an utterance and the whole document is not as useful as in other document summarization tasks.Page 8, “Conclusion and Future Work”
See all papers in Proc. ACL 2011 that mention cosine similarity.
See all papers in Proc. ACL that mention cosine similarity.
Back to top.
human annotated
- Often human annotators have different interpretations about the same sentence, and a speaker’s opiniorflattitude is sometimes ambiguous.Page 4, “Corpus Creation”
- We use human annotated dialogue acts (DA) as the extraction units.Page 6, “Experiments”
- The system-generated summaries are compared to human annotated extractive and abstractive summaries.Page 6, “Experiments”
- We also examined the system output and human annotation and found some reasons for the system errors:Page 8, “Experiments”
See all papers in Proc. ACL 2011 that mention human annotated.
See all papers in Proc. ACL that mention human annotated.
Back to top.
sentiment analysis
- Both sentiment analysis (opinion recognition) and summarization have been well studied in recent years in the natural language processing (NLP) community.Page 1, “Introduction”
- Most of the previous work on sentiment analysis has been conducted on reviews.Page 1, “Introduction”
- However, this problem is challenging in that: (a) Summarization in spontaneous speech is more difficult than well structured text (Mckeown et al., 2005), because speech is always less organized and has recognition errors when using speech recognition output; (b) Sentiment analysis in dialogues is also much harder because of the genre difference compared to other domains like product reviews or news resources, as reported in (Raaijmakers et al., 2008); (c) In conversational speech, information density is low and there are often off topic discussions, therefore presenting a need to identify utterances that are relevant to the topic.Page 1, “Introduction”
- Though there are many annotated data sets for the research of speech summarization and sentiment analysis , there is no corpus available for opinion summarization on spontaneous speech.Page 2, “Corpus Creation”
See all papers in Proc. ACL 2011 that mention sentiment analysis.
See all papers in Proc. ACL that mention sentiment analysis.
Back to top.
development set
- The 18 conversations annotated by all 3 annotators are used as test set, and the rest of 70 conversations are used as development set to tune the parameters (determining the best combination weights).Page 6, “Experiments”
- From the development set , we used the grid search method to obtain the best combination weights for the two summarization methods.Page 6, “Experiments”
- In the sentence-ranking method, the best parameters found on the development set are Asim = 0, Are; 2 0.3, Agent 2 0.3, Alen = 0.4.Page 6, “Experiments”
See all papers in Proc. ACL 2011 that mention development set.
See all papers in Proc. ACL that mention development set.
Back to top.
news articles
- Summarization has been applied to different genres, such as news articles , scientific articles, and speech domains including broadcast news, meetings, conversations and lectures.Page 1, “Introduction”
- Previous studies have used various domains, including news articles , scientific articles, web documents, reviews.Page 2, “Related Work”
- To obtain this, we trained a maximum entropy classifier with a bag-of-words model using a combination of data sets from several domains, including movie data (Pang and Lee, 2004), news articles from MPQA corpus (Wilson and Wiebe, 2003), and meeting transcripts from AMI corpus (Wilson, 2008a).Page 5, “Opinion Summarization Methods”
See all papers in Proc. ACL 2011 that mention news articles.
See all papers in Proc. ACL that mention news articles.
Back to top.
