A Taxonomy, Dataset, and Classifier for Automatic Noun Compound Interpretation
Tratz, Stephen and Hovy, Eduard

Article Structure

Abstract

The automatic interpretation of noun-noun compounds is an important subproblem within many natural language processing applications and is an area of increasing interest.

Introduction

Noun compounds (e.g., ‘maple leaf’) occur very frequently in text, and their interpretation—determining the relationships between adjacent nouns as well as the hierarchical dependency structure of the NP in which they occur—is an important problem within a wide variety of natural language processing (NLP) applications, including machine translation (B aldwin and Tanaka, 2004) and question answering (Ahn et al., 2005).

Related Work

2.1 Taxonomies

Taxonomy

3.1 Creation

Dataset

Our noun compound dataset was created from two principal sources: an in-house collection of terms extracted from a large corpus using part-of-speech tagging and mutual information and the Wall Street Journal section of the Penn Treebank.

Automated Classification

We use a Maximum Entropy (Berger et al., 1996) classifier with a large number of boolean features, some of which are novel (e. g., the inclusion of words from WordNet definitions).

Evaluation

6.1 Evaluation Data

Conclusion

In this paper, we present a novel, fine-grained taxonomy of 43 noun-noun semantic relations, the largest annotated noun compound dataset yet created, and a supervised classification method for automatic noun compound interpretation.

Future Work

In the future, we plan to focus on the interpretation of noun compounds with 3 or more nouns, a problem that includes bracketing noun compounds into their dependency structures in addition to noun-noun semantic relation interpretation.

Topics

Turkers

Appears in 24 sentences as: Turker (9) Turkers (17)
In A Taxonomy, Dataset, and Classifier for Automatic Noun Compound Interpretation
  1. We then embarked on a series of changes, testing each generation by annotation using Amazon’s Mechanical Turk service, a relatively quick and inexpensive online platform where requesters may publish tasks for anonymous online workers ( Turkers ) to perform.
    Page 2, “Taxonomy”
  2. Turkers were asked to select one or, if they deemed it appropriate, two categories for each noun pair.
    Page 4, “Taxonomy”
  3. In addition to influencing the category definitions, some taxonomy groupings were altered with the hope that this would improve inter-annotator agreement for cases where Turker disagreement was systematic.
    Page 4, “Taxonomy”
  4. For example, LOCATION and WHOLE + PART/MEMBER OF were commonly disagreed upon by Turkers so they were placed within their own taxonomic subgroup.
    Page 4, “Taxonomy”
  5. Turkers also tended to disagree between the categories related to composition and containment.
    Page 4, “Taxonomy”
  6. The ATTRIBUTE categories are positioned near the TOPIC group because some Turkers chose a TOPIC category when an ATTRIBU TE category was deemed more appropriate.
    Page 4, “Taxonomy”
  7. Similarly, while WHOLE+PART/MEMBER was selected by most Turkers for bike tire, one individual chose PURPOSE.
    Page 5, “Taxonomy”
  8. Unlike the previous two distinguishing characteristics, which were motivated primarily by Turker annotations, this separation was largely motivated by author dissatisfaction with a single TOPIC category.
    Page 5, “Taxonomy”
  9. Due to the relatively high speed and low cost of Amazon’s Mechanical Turk serVice, we chose to use Mechanical Turkers as our annotators.
    Page 7, “Evaluation”
  10. The first and most significant drawback is that it is impossible to force each Turker to label every data point without putting all the terms onto a single web page, which is highly impractical for a large taxonomy.
    Page 7, “Evaluation”
  11. Some Turkers may label every compound, but most do not.
    Page 7, “Evaluation”

See all papers in Proc. ACL 2010 that mention Turkers.

See all papers in Proc. ACL that mention Turkers.

Back to top.

semantic relations

Appears in 7 sentences as: semantic relation (1) semantic relations (6)
In A Taxonomy, Dataset, and Classifier for Automatic Noun Compound Interpretation
  1. In contrast to studies that claim the existence of a relatively small number of semantic relations , Downing (1977) presents a strong case for the existence of an unbounded number of relations.
    Page 2, “Related Work”
  2. Table 1: The semantic relations , their frequency ir
    Page 3, “Taxonomy”
  3. Kim and Baldwin (2005) report an agreement of 52.31% (not H) for their dataset using Barker and Sz-pakowicz’s (1998) 20 semantic relations .
    Page 8, “Evaluation”
  4. (2005) report .58 K: using a set of 35 semantic relations , only 21 of which were used, and a .80 H score using Lauer’s 8 prepositional paraphrases.
    Page 8, “Evaluation”
  5. Girju (2007) reports .61 H agreement using a similar set of 22 semantic relations for noun compound annotation in which the annotators are shown translations of the compound in foreign languages.
    Page 8, “Evaluation”
  6. In this paper, we present a novel, fine-grained taxonomy of 43 noun-noun semantic relations , the largest annotated noun compound dataset yet created, and a supervised classification method for automatic noun compound interpretation.
    Page 8, “Conclusion”
  7. In the future, we plan to focus on the interpretation of noun compounds with 3 or more nouns, a problem that includes bracketing noun compounds into their dependency structures in addition to noun-noun semantic relation interpretation.
    Page 9, “Future Work”

See all papers in Proc. ACL 2010 that mention semantic relations.

See all papers in Proc. ACL that mention semantic relations.

Back to top.

Mechanical Turk

Appears in 6 sentences as: Mechanical Turk (6) Mechanical Turkers (1)
In A Taxonomy, Dataset, and Classifier for Automatic Noun Compound Interpretation
  1. We then embarked on a series of changes, testing each generation by annotation using Amazon’s Mechanical Turk service, a relatively quick and inexpensive online platform where requesters may publish tasks for anonymous online workers (Turkers) to perform.
    Page 2, “Taxonomy”
  2. Mechanical Turk has been previously used in a variety of NLP research, including recent work on noun compounds by Nakov (2008) to collect short phrases for linking the nouns within noun compounds.
    Page 2, “Taxonomy”
  3. For the Mechanical Turk annotation tests, we created five sets of 100 noun compounds from noun compounds automatically extracted from a random subset of New York Times articles written between 1987 and 2007 (Sandhaus, 2008).
    Page 2, “Taxonomy”
  4. Our results from Mechanical Turk showed significant overlap between PURPOSE and OBJECT categories (present in an earlier version of the taxonomy).
    Page 4, “Taxonomy”
  5. Due to the relatively high speed and low cost of Amazon’s Mechanical Turk serVice, we chose to use Mechanical Turkers as our annotators.
    Page 7, “Evaluation”
  6. Using Mechanical Turk to obtain inter-annotator agreement figures has several drawbacks.
    Page 7, “Evaluation”

See all papers in Proc. ACL 2010 that mention Mechanical Turk.

See all papers in Proc. ACL that mention Mechanical Turk.

Back to top.

cross validation

Appears in 4 sentences as: Cross Validation (1) cross validation (4)
In A Taxonomy, Dataset, and Classifier for Automatic Noun Compound Interpretation
  1. 5.2 Cross Validation Experiments
    Page 6, “Automated Classification”
  2. We performed 10-fold cross validation on our dataset, and, for the purpose of comparison, we also performed 5-fold cross validation on C) Séaghdha’s (2007) dataset using his folds.
    Page 6, “Automated Classification”
  3. To assess the impact of the various features, we ran the cross validation experiments for each feature type, alternating between including only one
    Page 6, “Automated Classification”
  4. Table 4: Impact of features; cross validation accuracy for only one feature type and all but one feature type experiments, denoted by l and M-1 respectively.
    Page 7, “Automated Classification”

See all papers in Proc. ACL 2010 that mention cross validation.

See all papers in Proc. ACL that mention cross validation.

Back to top.

WordNet

Appears in 4 sentences as: WordNet (4)
In A Taxonomy, Dataset, and Classifier for Automatic Noun Compound Interpretation
  1. Kim and Baldwin (2005) and Turney (2006) use nearest neighbor approaches based upon WordNet (Fellbaum, 1998) and Tumey’s Latent Relational Analysis, respectively.
    Page 2, “Related Work”
  2. We use a Maximum Entropy (Berger et al., 1996) classifier with a large number of boolean features, some of which are novel (e. g., the inclusion of words from WordNet definitions).
    Page 5, “Automated Classification”
  3. The WordNet gloss terms had a surprisingly strong influence.
    Page 7, “Automated Classification”
  4. As far as we know, this is the first time that WordNet definition words have been used as features for noun compound interpretation.
    Page 7, “Automated Classification”

See all papers in Proc. ACL 2010 that mention WordNet.

See all papers in Proc. ACL that mention WordNet.

Back to top.

fine-grained

Appears in 3 sentences as: fine-grained (3)
In A Taxonomy, Dataset, and Classifier for Automatic Noun Compound Interpretation
  1. In this paper, we present a large, fine-grained taxonomy of 43 noun compound relations, a dataset annotated according to this taxonomy, and a supervised, automatic classification method for determining the relation between the head and modifier words in a noun compound.
    Page 1, “Introduction”
  2. The .57-.67 H figures achieved by the Voted annotations compare well with previously reported inter-annotator agreement figures for noun compounds using fine-grained taxonomies.
    Page 8, “Evaluation”
  3. In this paper, we present a novel, fine-grained taxonomy of 43 noun-noun semantic relations, the largest annotated noun compound dataset yet created, and a supervised classification method for automatic noun compound interpretation.
    Page 8, “Conclusion”

See all papers in Proc. ACL 2010 that mention fine-grained.

See all papers in Proc. ACL that mention fine-grained.

Back to top.

Hypernyms

Appears in 3 sentences as: hypernym (1) Hypernyms (1) hypernyms (1)
In A Taxonomy, Dataset, and Classifier for Automatic Noun Compound Interpretation
  1. 0 {Synonyms, Hypernyms } for all NN and VB entries for each word
    Page 5, “Automated Classification”
  2. Intersection of the words’ hypernyms
    Page 5, “Automated Classification”
  3. In fact, by themselves they proved roughly as useful as the hypernym features, and their removal had the single strongest negative impact on accuracy for our dataset.
    Page 7, “Automated Classification”

See all papers in Proc. ACL 2010 that mention Hypernyms.

See all papers in Proc. ACL that mention Hypernyms.

Back to top.

Maximum Entropy

Appears in 3 sentences as: Maximum Entropy (3)
In A Taxonomy, Dataset, and Classifier for Automatic Noun Compound Interpretation
  1. We use a Maximum Entropy (Berger et al., 1996) classifier with a large number of boolean features, some of which are novel (e. g., the inclusion of words from WordNet definitions).
    Page 5, “Automated Classification”
  2. Maximum Entropy classifiers have been effective on a variety of NLP problems including preposition sense disambiguation (Ye and Baldwin, 2007), which is somewhat similar to noun compound interpretation.
    Page 5, “Automated Classification”
  3. The results for these runs using the Maximum Entropy classifier are presented in Table 4.
    Page 7, “Automated Classification”

See all papers in Proc. ACL 2010 that mention Maximum Entropy.

See all papers in Proc. ACL that mention Maximum Entropy.

Back to top.

n-gram

Appears in 3 sentences as: N-gram (1) n-gram (2)
In A Taxonomy, Dataset, and Classifier for Automatic Noun Compound Interpretation
  1. Web 1T N-gram Features
    Page 6, “Automated Classification”
  2. Table 3 describes the extracted n-gram features.
    Page 6, “Automated Classification”
  3. The influence of the Web lT n-gram features was somewhat mixed.
    Page 7, “Automated Classification”

See all papers in Proc. ACL 2010 that mention n-gram.

See all papers in Proc. ACL that mention n-gram.

Back to top.

n-grams

Appears in 3 sentences as: n-grams (3)
In A Taxonomy, Dataset, and Classifier for Automatic Noun Compound Interpretation
  1. To provide information related to term usage to the classifier, we extracted trigram and 4-gram features from the Web lT Corpus (Brants and Franz, 2006), a large collection of n-grams and their counts created from approximately one trillion words of Web text.
    Page 6, “Automated Classification”
  2. Only n-grams containing lowercase words were used.
    Page 6, “Automated Classification”
  3. Only n-grams containing both terms (including plural forms) were extracted.
    Page 6, “Automated Classification”

See all papers in Proc. ACL 2010 that mention n-grams.

See all papers in Proc. ACL that mention n-grams.

Back to top.

PDF available from www.aclweb.org.
Send us an email