Knowledge-Based Question Answering as Machine Translation
Bao, Junwei and Duan, Nan and Zhou, Ming and Zhao, Tiejun

Article Structure

Abstract

A typical knowledge-based question answering (KB-QA) system faces two challenges: one is to transform natural language questions into their meaning representations (MRs); the other is to retrieve answers from knowledge bases (KBs) using generated MRs.

Introduction

Knowledge-based question answering (KB-QA) computes answers to natural language (NL) questions based on existing knowledge bases (KBs).

Topics

semantic parsing

Appears in 11 sentences as: semantic parser (4) semantic parsing (8)
In Knowledge-Based Question Answering as Machine Translation
  1. Compared to a KB-QA system using a state-of-the-art semantic parser , our method achieves better results.
    Page 1, “Abstract”
  2. Most previous systems tackle this task in a cascaded manner: First, the input question is transformed into its meaning representation (MR) by an independent semantic parser (Zettlemoyer and Collins, 2005; Mooney, 2007; Artzi and Zettlemoyer, 2011; Liang et al., 2011; Cai and Yates,
    Page 1, “Introduction”
  3. Unlike existing KB-QA systems which treat semantic parsing and answer retrieval as two cascaded tasks, this paper presents a unified framework that can integrate semantic parsing into the question answering procedure directly.
    Page 1, “Introduction”
  4. The contributions of this work are twofold: (1) We propose a translation-based KB-QA method that integrates semantic parsing and QA in one unified framework.
    Page 1, “Introduction”
  5. Such assumption simplifies the efforts of semantic parsing to the minimum question units, while leaving the capability of handling multiple-relation questions (Figure 1 gives one such example) to the outer CYK—parsing based translation procedure.
    Page 3, “Introduction”
  6. Our work intersects with two research directions: semantic parsing and question answering.
    Page 6, “Introduction”
  7. Some previous works on semantic parsing (Zelle and Mooney, 1996; Zettlemoyer and Collins, 2005; Wong and Mooney, 2006; Zettlemoyer and Collins, 2007; Wong and Mooney,
    Page 6, “Introduction”
  8. In comparison, our method has further advantages: (1) Question answering and semantic parsing are performed in an joint way under a unified framework; (2) A robust method is proposed to map NL questions to their formal triple queries, which trades off the mapping quality by using question patterns and relation expressions in a cascaded way; and (3) We use domain independent feature set which allowing us to use a relatively small number of question-answer pairs to tune model parameters.
    Page 7, “Introduction”
  9. Actually, the size of our system’s search space is much smaller than the one of the semantic parser used in the baseline.This is due to the fact that, if triple queries generated by the question translation component cannot derive any answer from KB, we will discard such triple queries directly during the QA procedure.
    Page 9, “Introduction”
  10. This paper presents a translation-based KB-QA method that integrates semantic parsing and QA in one unified framework.
    Page 9, “Introduction”
  11. Comparing to the baseline system using an independent semantic parser with state-of-the-art performance, we achieve better results on a general domain evaluation set.
    Page 9, “Introduction”

See all papers in Proc. ACL 2014 that mention semantic parsing.

See all papers in Proc. ACL that mention semantic parsing.

Back to top.

feature weights

Appears in 8 sentences as: Feature Weight (1) feature weight (2) feature weights (5)
In Knowledge-Based Question Answering as Machine Translation
  1. A linear model is defined over derivations, and minimum error rate training is used to tune feature weights based on a set of question-answer pairs.
    Page 1, “Abstract”
  2. Derivations generated during such a translation procedure are modeled by a linear model, and minimum error rate training (MERT) (Och, 2003) is used to tune feature weights based on a set of question-answer pairs.
    Page 1, “Introduction”
  3. 0 Ai denotes the feature weight of
    Page 2, “Introduction”
  4. According to the above description, our KB-QA method can be decomposed into four tasks as: (1) search space generation for H(Q); (2) question translation for transforming question spans into their corresponding formal triples; (3) feature design for and (4) feature weight tuning for {A1}.
    Page 2, “Introduction”
  5. 2.5 Feature Weight Tuning
    Page 6, “Introduction”
  6. Given a set of question-answer pairs {Qh A1?” } as the development (dev) set, we use the minimum error rate training (MERT) (Och, 2003) algorithm to tune the feature weights A11” in our proposed model.
    Page 6, “Introduction”
  7. The training criterion is to seek the feature weights that can minimize the accumulated errors of the top-l answer of questions in the dev set:
    Page 6, “Introduction”
  8. N is the number of questions in the dev set, Agef is the correct answers as references of the ith question in the dev set, is the top-l answer candidate of the ith question in the dev set based on feature weights AM , Err(-) is the error function which is defined as:
    Page 6, “Introduction”

See all papers in Proc. ACL 2014 that mention feature weights.

See all papers in Proc. ACL that mention feature weights.

Back to top.

entity mention

Appears in 7 sentences as: entity mention (4) entity mentions (3)
In Knowledge-Based Question Answering as Machine Translation
  1. 3 For simplicity, a cleaned entity dictionary dumped from the entire ICE is used to detect entity mentions in Q.
    Page 3, “Introduction”
  2. Algorithm 2: Q’P-based Question Translation T = (Z); foreach entity mention eg 6 Q do Qpattem = replace eg in Q with [Slot]; foreach question pattern Q73 do if Qpattern == Qppattern then 5 = Disambiguate(eg, QPpredicate); foreach e E 5 do create a new triple query q; q : {ea Qppredicatea {At} = AnswerRetrieve(q, ICE); foreach A 6 {At} do create a new formal triple t; t = {q-esbj, q-p,«4}; t.score = 1.0; insert t to T;
    Page 3, “Introduction”
  3. For each entity mention eg 6 Q, we replace it with [Slot] and obtain a pattern string Qpattem (Line 3).
    Page 3, “Introduction”
  4. The name of each entity returned equals the input entity mention eg and occurs in some assertions where Q’Ppredicate are the predicates.
    Page 3, “Introduction”
  5. Question patterns are collected as follows: First, 5 W queries, which begin with What, Where, Who, When, or Which, are selected from a large scale query log of a commercial search engine; Then, a cleaned entity dictionary is used to annotate each query by replacing all entity mentions it contains with the symbol [Slot].
    Page 3, “Introduction”
  6. For each possible entity mention eg 6 Q and a KB predicate ]?
    Page 4, “Introduction”
  7. Since named entity recognizers trained on Penn TreeBank usually perform poorly on web queries, We instead use a simple string-match method to detect entity mentions in the question using a cleaned entity dictionary dumped from our KB.
    Page 9, “Introduction”

See all papers in Proc. ACL 2014 that mention entity mention.

See all papers in Proc. ACL that mention entity mention.

Back to top.

question answering

Appears in 6 sentences as: Question answering (1) question answering (5)
In Knowledge-Based Question Answering as Machine Translation
  1. A typical knowledge-based question answering (KB-QA) system faces two challenges: one is to transform natural language questions into their meaning representations (MRs); the other is to retrieve answers from knowledge bases (KBs) using generated MRs.
    Page 1, “Abstract”
  2. Knowledge-based question answering (KB-QA) computes answers to natural language (NL) questions based on existing knowledge bases (KBs).
    Page 1, “Introduction”
  3. Unlike existing KB-QA systems which treat semantic parsing and answer retrieval as two cascaded tasks, this paper presents a unified framework that can integrate semantic parsing into the question answering procedure directly.
    Page 1, “Introduction”
  4. Our work intersects with two research directions: semantic parsing and question answering .
    Page 6, “Introduction”
  5. In comparison, our method has further advantages: (1) Question answering and semantic parsing are performed in an joint way under a unified framework; (2) A robust method is proposed to map NL questions to their formal triple queries, which trades off the mapping quality by using question patterns and relation expressions in a cascaded way; and (3) We use domain independent feature set which allowing us to use a relatively small number of question-answer pairs to tune model parameters.
    Page 7, “Introduction”
  6. This means how to extract high-quality question patterns is worth to be studied for the question answering task.
    Page 8, “Introduction”

See all papers in Proc. ACL 2014 that mention question answering.

See all papers in Proc. ACL that mention question answering.

Back to top.

knowledge bases

Appears in 5 sentences as: knowledge base (2) knowledge bases (3)
In Knowledge-Based Question Answering as Machine Translation
  1. A typical knowledge-based question answering (KB-QA) system faces two challenges: one is to transform natural language questions into their meaning representations (MRs); the other is to retrieve answers from knowledge bases (KBs) using generated MRs.
    Page 1, “Abstract”
  2. Knowledge-based question answering (KB-QA) computes answers to natural language (NL) questions based on existing knowledge bases (KBs).
    Page 1, “Introduction”
  3. Compared to their work, our method gains an improvement in two aspects: (1) Instead of using facts extracted using the open IE method, we leverage a large scale, high-quality knowledge base ; (2) We can handle multiple-relation questions, instead of single-relation queries only, based on our translation based KB-QA framework.
    Page 7, “Introduction”
  4. (2013) is one of the latest work which has reported QA results based on a large scale, general domain knowledge base (Freebase), we consider their evaluation result on WEBQUESTIONS as our baseline.
    Page 7, “Introduction”
  5. As we discussed in the experiment part, how to mine high-quality question patterns is worth further study for the QA task; (ii) We plan to integrate an ad-hoc NER into our KB-QA system to alleviate the entity detection issue; (iii) In fact, our proposed QA framework can be generalized to other intelligence besides knowledge bases as well.
    Page 9, “Introduction”

See all papers in Proc. ACL 2014 that mention knowledge bases.

See all papers in Proc. ACL that mention knowledge bases.

Back to top.

error rate

Appears in 4 sentences as: error rate (4)
In Knowledge-Based Question Answering as Machine Translation
  1. A linear model is defined over derivations, and minimum error rate training is used to tune feature weights based on a set of question-answer pairs.
    Page 1, “Abstract”
  2. Derivations generated during such a translation procedure are modeled by a linear model, and minimum error rate training (MERT) (Och, 2003) is used to tune feature weights based on a set of question-answer pairs.
    Page 1, “Introduction”
  3. Given a set of question-answer pairs {Qh A1?” } as the development (dev) set, we use the minimum error rate training (MERT) (Och, 2003) algorithm to tune the feature weights A11” in our proposed model.
    Page 6, “Introduction”
  4. A lower disambiguation error rate results in better relation expressions.
    Page 8, “Introduction”

See all papers in Proc. ACL 2014 that mention error rate.

See all papers in Proc. ACL that mention error rate.

Back to top.

beam size

Appears in 3 sentences as: beam size (3)
In Knowledge-Based Question Answering as Machine Translation
  1. As the performance of our KB-QA system relies heavily on the k-best beam approximation, we evaluate the impact of the beam size and list the comparison results in Figure 6.
    Page 8, “Introduction”
  2. Figure 6: Impacts of beam size on accuracy.
    Page 9, “Introduction”
  3. We can see that using a small k can achieve better results than baseline, where the beam size is set to be 200.
    Page 9, “Introduction”

See all papers in Proc. ACL 2014 that mention beam size.

See all papers in Proc. ACL that mention beam size.

Back to top.

subtrees

Appears in 3 sentences as: subtrees (3)
In Knowledge-Based Question Answering as Machine Translation
  1. Note, question decomposition only operates on the original question and question spans covered by complete dependency subtrees .
    Page 5, “Introduction”
  2. 0 hsyntaasubtreefl), which counts the number of spans in Q that are (I) converted to formal triples, whose predicates are not Null, and (2) covered by complete dependency subtrees at the same time.
    Page 6, “Introduction”
  3. The underlying intuition is that, dependency subtrees of Q should be treated as units for question translation.
    Page 6, “Introduction”

See all papers in Proc. ACL 2014 that mention subtrees.

See all papers in Proc. ACL that mention subtrees.

Back to top.

PDF available from www.aclweb.org.
Send us an email