Sentence Dependency Tagging in Online Question Answering Forums
Qu, Zhonghua and Liu, Yang

Article Structure

Abstract

Online forums are becoming a popular resource in the state of the art question answering (QA) systems.

Introduction

Automatic Question Answering (QA) systems rely heavily on good sources of data that contain questions and answers.

Related Work

There is a lot of useful knowledge in the user generated content such as forums.

Thread Structure Tagging

As described earlier, we decompose the structure analysis of QA threads into two tasks, first determine the sentence type, and then identify related sentences.

Data

We used data from ubuntu community forum general help section for the experiments and evaluation.

Experiment

In the experiment, we randomly split annotated threads into three disjoint sets, and run a threefold cross validation.

Conclusion

In this paper, we investigated sentence dependency tagging of question and answer (QA) threads in online forums.

Acknowledgment

This work is supported by DARPA under Contract No.

Topics

CRFs

Appears in 24 sentences as: CRFs (27)
In Sentence Dependency Tagging in Online Question Answering Forums
  1. Because multiple runs of separate linear-chain CRFs ignore the dependency between source sentences, the second approach we propose is to use a 2D CRF that models all pair relationships jointly.
    Page 2, “Introduction”
  2. Our experimental results show that our proposed sentence type tagging method works very well, even for the minority categories, and that using 2D CRF further improves performance over linear-chain CRFs for identifying dependency relation between sentences.
    Page 2, “Introduction”
  3. In Section 3, we introduce the use of CRFs for sentence type and dependency tagging.
    Page 2, “Introduction”
  4. Our study is different in several aspects: we are using forum domains, unlike most work of DA tagging on conversational speech; we use CRFs for sentence type tagging; and more importantly, we also propose to use different CRFs for sentence relation detection.
    Page 3, “Related Work”
  5. To automatically label sentences in a thread with their types, we adopt a sequence labeling approach, specifically linear-chain conditional random fields ( CRFs ), which have shown good performance in many other tasks (Lafferty, 2001).
    Page 3, “Thread Structure Tagging”
  6. Linear-chain CRFs is a type of undirected graphical models.
    Page 4, “Thread Structure Tagging”
  7. CRFs is a special case of undirected graphical model in which w are log-linear functions:
    Page 4, “Thread Structure Tagging”
  8. Figure 3: Graphical Structure of Linear-chain CRFs .
    Page 4, “Thread Structure Tagging”
  9. Linear-chain CRF is a special case of the general CRFs .
    Page 4, “Thread Structure Tagging”
  10. In order to take advantage of such adjacent sentence dependency, we use the linear-chain CRFs for the sequence labeling.
    Page 5, “Thread Structure Tagging”
  11. The linear-chain CRFs can represent the dependency between adjacent target sentences quite well.
    Page 5, “Thread Structure Tagging”

See all papers in Proc. ACL 2012 that mention CRFs.

See all papers in Proc. ACL that mention CRFs.

Back to top.

CRF

Appears in 19 sentences as: CRF (20)
In Sentence Dependency Tagging in Online Question Answering Forums
  1. We use linear-chain conditional random fields (CRF) for sentence type tagging, and a 2D CRF to label the dependency relation between sentences.
    Page 1, “Abstract”
  2. We use linear-chain conditional random fields ( CRF ) to take advantage of many long-distance and nonlocal features.
    Page 2, “Introduction”
  3. First each sentence is considered as a source, and we run a linear-chain CRF to label whether each of the other sentences is its target.
    Page 2, “Introduction”
  4. Because multiple runs of separate linear-chain CRFs ignore the dependency between source sentences, the second approach we propose is to use a 2D CRF that models all pair relationships jointly.
    Page 2, “Introduction”
  5. Our experimental results show that our proposed sentence type tagging method works very well, even for the minority categories, and that using 2D CRF further improves performance over linear-chain CRFs for identifying dependency relation between sentences.
    Page 2, “Introduction”
  6. In (Ding et al., 2008), a two-pass approach was used to find relevant solutions for a given question, and a skip-chain CRF was adopted to model long range de-
    Page 2, “Related Work”
  7. Linear-chain CRF is a special case of the general CRFs.
    Page 4, “Thread Structure Tagging”
  8. In linear-chain CRF , cliques only involve two adjacent variables in the sequence.
    Page 4, “Thread Structure Tagging”
  9. Figure 3 shows the graphical structure of a linear-chain CRF .
    Page 4, “Thread Structure Tagging”
  10. For a thread of N sentences, we need to perform N runs of CRF labeling, one for each sentence (as the source sentence) in order to label the target sentence corresponding to this source sentence.
    Page 5, “Thread Structure Tagging”
  11. The graphical structure of a 2D CRF is shown in Figure 4.
    Page 5, “Thread Structure Tagging”

See all papers in Proc. ACL 2012 that mention CRF.

See all papers in Proc. ACL that mention CRF.

Back to top.

dependency relations

Appears in 18 sentences as: dependency relation (7) dependency relations (8) Dependency relationship (1) dependency relationship (1) dependency relationships (1)
In Sentence Dependency Tagging in Online Question Answering Forums
  1. We use linear-chain conditional random fields (CRF) for sentence type tagging, and a 2D CRF to label the dependency relation between sentences.
    Page 1, “Abstract”
  2. Towards this goal, in this paper, we define two tasks: labeling the types for sentences, and finding the dependency relations between sentences.
    Page 2, “Introduction”
  3. In this study, we use two approaches for labeling of dependency relation between sentences.
    Page 2, “Introduction”
  4. Our experimental results show that our proposed sentence type tagging method works very well, even for the minority categories, and that using 2D CRF further improves performance over linear-chain CRFs for identifying dependency relation between sentences.
    Page 2, “Introduction”
  5. In this paper, in order to provide a better foundation for question answer detection in online forums, we investigate tagging sentences with a much richer set of categories, as well as identifying their dependency relationships .
    Page 3, “Related Work”
  6. Knowing only the sentence types without their dependency relations is not enough for question answering tasks.
    Page 5, “Thread Structure Tagging”
  7. Note that sentence dependency relations might not be a one-to-one relation.
    Page 5, “Thread Structure Tagging”
  8. Dependency relationship could happen between many different types of sentences, for example, answer(s) to question(s), problem clarification to question inquiry, feedback to solutions, etc.
    Page 5, “Thread Structure Tagging”
  9. Instead of developing models for each dependency type, we treat them uniformly as dependency relations between sentences.
    Page 5, “Thread Structure Tagging”
  10. Hence, for every two sentences, it becomes a binary classification problem, i.e., whether or not there exists a dependency relation between them.
    Page 5, “Thread Structure Tagging”
  11. The sentence dependency task is formally defined as, given a set of sentences St of a thread, find the dependency relation {(3, t)|s E St, 75 E St}, where s is the source sentence and t is the target sentence that 8 depends on.
    Page 5, “Thread Structure Tagging”

See all papers in Proc. ACL 2012 that mention dependency relations.

See all papers in Proc. ACL that mention dependency relations.

Back to top.

question answering

Appears in 16 sentences as: question answer (3) Question Answering (2) Question answering (1) question answering (10)
In Sentence Dependency Tagging in Online Question Answering Forums
  1. Online forums are becoming a popular resource in the state of the art question answering (QA) systems.
    Page 1, “Abstract”
  2. Most prior work focused on extracting only question answering sentences from user conversations.
    Page 1, “Abstract”
  3. Automatic Question Answering (QA) systems rely heavily on good sources of data that contain questions and answers.
    Page 1, “Introduction”
  4. Question answering forums, such as technical support forums, are places where users find answers through conversations.
    Page 1, “Introduction”
  5. Because of their nature as online communities, question answering forums provide more updated answers to new problems.
    Page 1, “Introduction”
  6. Only identifying question and answering sentences from the thread is not enough for automatic question answering .
    Page 1, “Introduction”
  7. Figure 1: Example of a Question Answering Thread in Ubuntu Support Forum
    Page 2, “Introduction”
  8. This knowledge source could substantially help automatic question answering systems.
    Page 2, “Related Work”
  9. An approach using email structure to detect and summarize question answer pairs was introduced in (Shrestha and Mck-eown, 2004).
    Page 3, “Related Work”
  10. In this paper, in order to provide a better foundation for question answer detection in online forums, we investigate tagging sentences with a much richer set of categories, as well as identifying their dependency relationships.
    Page 3, “Related Work”
  11. The sentence types we use are similar to dialog acts (DA), but defined specifically for question answering forums.
    Page 3, “Related Work”

See all papers in Proc. ACL 2012 that mention question answering.

See all papers in Proc. ACL that mention question answering.

Back to top.

Cosine similarity

Appears in 4 sentences as: Cosine similarity (3) cosine similarity (1)
In Sentence Dependency Tagging in Online Question Answering Forums
  1. Cosine similarity with previous sentence.
    Page 4, “Thread Structure Tagging”
  2. Here we use the cosine similarity between sentences, where each sentence is represented as a vector of words, with term weight calculated using TD-IDF (term frequency times inverse document frequency).
    Page 4, “Thread Structure Tagging”
  3. * Cosine similarity with previous sentence.
    Page 5, “Thread Structure Tagging”
  4. Cosine similarity between target and source sentence.
    Page 5, “Thread Structure Tagging”

See all papers in Proc. ACL 2012 that mention Cosine similarity.

See all papers in Proc. ACL that mention Cosine similarity.

Back to top.

graphical models

Appears in 4 sentences as: graphical model (1) graphical models (3)
In Sentence Dependency Tagging in Online Question Answering Forums
  1. For example, in (Stolcke et al., 2000), Hidden Markov Models (HMMs) were used for DA tagging; in (J i and Bilmes, 2005), different types of graphical models were explored.
    Page 3, “Related Work”
  2. Linear-chain CRFs is a type of undirected graphical models .
    Page 4, “Thread Structure Tagging”
  3. Distribution of a set of variables in undirected graphical models can be written as
    Page 4, “Thread Structure Tagging”
  4. CRFs is a special case of undirected graphical model in which w are log-linear functions:
    Page 4, “Thread Structure Tagging”

See all papers in Proc. ACL 2012 that mention graphical models.

See all papers in Proc. ACL that mention graphical models.

Back to top.

sequence labeling

Appears in 4 sentences as: sequence labeling (4)
In Sentence Dependency Tagging in Online Question Answering Forums
  1. To automatically label sentences in a thread with their types, we adopt a sequence labeling approach, specifically linear-chain conditional random fields (CRFs), which have shown good performance in many other tasks (Lafferty, 2001).
    Page 3, “Thread Structure Tagging”
  2. In the sequence labeling task, feature functions across the sequence are often tied together.
    Page 4, “Thread Structure Tagging”
  3. In order to take advantage of such adjacent sentence dependency, we use the linear-chain CRFs for the sequence labeling .
    Page 5, “Thread Structure Tagging”
  4. Unlike one dimensional sequence labeling , a node in 2D environment is dependent on both X-aXis neighbors and y-aXis neighbors.
    Page 5, “Thread Structure Tagging”

See all papers in Proc. ACL 2012 that mention sequence labeling.

See all papers in Proc. ACL that mention sequence labeling.

Back to top.