Classification of Semantic Relationships between Nominals Using Pattern Clusters
Davidov, Dmitry and Rappoport, Ari

Article Structure

Abstract

There are many possible different semantic relationships between nominals.

Introduction

Automatic extraction and classification of semantic relationships is a major field of activity, of both practical and theoretical interest.

Related Work

Numerous methods have been devised for classification of semantic relationships, among which those holding between nominals constitute a prominent category.

Pattern Clustering Algorithm

Our pattern clustering algorithm is designed for the unsupervised definition and discovery of generic semantic relationships.

Relationship Classification

Up to this stage we did not access the training set in any way and we did not use the fact that the target relations are those holding between nominals.

Experimental Setup

The main problem in a fair evaluation of NR classification is that there is no widely accepted list of possible relationships between nominals.

Results

The pattern clustering phase results in 90 to 3000 distinct pattern clusters, depending on the parameter setup.

Conclusion

Relationship classification is known to improve many practical tasks, e.g., textual entailment (Tatu and Moldovan, 2005).

Topics

WordNet

Appears in 13 sentences as: WordNet (14)
In Classification of Semantic Relationships between Nominals Using Pattern Clusters
  1. Our NR classification evaluation strictly follows the ACL SemEval-07 Task 4 datasets and protocol, obtaining an f-score of 70.6, as opposed to 64.8 of the best previous work that did not use the manually provided WordNet sense disambiguation tags.
    Page 1, “Abstract”
  2. To improve results, some systems utilize additional manually constructed semantic resources such as WordNet (WN) (Beamer et al., 2007).
    Page 1, “Introduction”
  3. Furthermore, usage of such resources frequently requires disambiguation and connection of the data to the resource (word sense disambiguation in the case of WordNet ).
    Page 1, “Introduction”
  4. We evaluated our algorithm on SemEval-07 Task 4 data, showing superior results over participating algorithms that did not utilize WordNet disambiguation tags.
    Page 2, “Introduction”
  5. Many relation classification algorithms utilize WordNet .
    Page 2, “Related Work”
  6. Among the 15 systems presented by the 14 SemEval teams, some utilized the manually provided WordNet tags for the dataset pairs (e.g., (Beamer et al., 2007)).
    Page 2, “Related Work”
  7. Nouns in this pair were manually labeled with their corresponding WordNet 3 labels and the web queries used to
    Page 6, “Experimental Setup”
  8. The 15 submitted systems were assigned into 4 categories according to whether they use the WordNet and Query tags (some systems were assigned to more than a single category, since they reported experiments in several settings).
    Page 7, “Experimental Setup”
  9. In our evaluation we do not utilize WordNet or Query tags, hence we compare ourselves with the corresponding group (A), containing 6 systems.
    Page 7, “Experimental Setup”
  10. Method P R F Acc Unsupervised clustering (4.3.3) 64.5 61.3 62.0 64.5 Cluster Labeling (4.3.1) 65.1 69.0 67.2 68.5 HITS Features (4.3.2) 69.1 70.6 70.6 70.1 Best Task 4 (no WordNet) 66.1 66.7 64.8 66.0 Best Task 4 (with WordNet ) 79.7 69.8 72.4 76.3
    Page 8, “Results”
  11. Table 1 shows our results, along with the best Task 4 result not using WordNet labels (Costello, 2007).
    Page 8, “Results”

See all papers in Proc. ACL 2008 that mention WordNet.

See all papers in Proc. ACL that mention WordNet.

Back to top.

semantic relationships

Appears in 8 sentences as: semantic relationship (1) semantic relationships (8)
In Classification of Semantic Relationships between Nominals Using Pattern Clusters
  1. There are many possible different semantic relationships between nominals.
    Page 1, “Abstract”
  2. Each of the extracted clusters corresponds to some unspecified semantic relationship .
    Page 1, “Abstract”
  3. Automatic extraction and classification of semantic relationships is a major field of activity, of both practical and theoretical interest.
    Page 1, “Introduction”
  4. A prominent type of semantic relationships is that holding between nonnnabl.Forexanqfle,ninouncxnnpoundsrnany different semantic relationships are encoded by the same simple form (Girju et al., 2005): ‘dog food’ denotes food consumed by dogs, while ‘summer mom-
    Page 1, “Introduction”
  5. The semantic relationships between the components of noun compounds and between nominals in general are not easy to categorize rigorously.
    Page 2, “Introduction”
  6. Numerous methods have been devised for classification of semantic relationships , among which those holding between nominals constitute a prominent category.
    Page 2, “Related Work”
  7. Since (Hearst, 1992), numerous works have used patterns for discovery and identification of instances of semantic relationships (e.g., (Girju et al., 2006; Snow et al., 2006; Banko et al, 2007)).
    Page 3, “Related Work”
  8. Our pattern clustering algorithm is designed for the unsupervised definition and discovery of generic semantic relationships .
    Page 3, “Pattern Clustering Algorithm”

See all papers in Proc. ACL 2008 that mention semantic relationships.

See all papers in Proc. ACL that mention semantic relationships.

Back to top.

learning algorithms

Appears in 6 sentences as: Learning Algorithm (1) learning algorithm (2) learning algorithms (3)
In Classification of Semantic Relationships between Nominals Using Pattern Clusters
  1. The standard classification process is to find in an auxiliary corpus a set of patterns in which a given training word pair co-appears, and use pattern-word pair co-appearance statistics as features for machine learning algorithms .
    Page 1, “Introduction”
  2. Various learning algorithms have been used for relation classification.
    Page 3, “Related Work”
  3. Freely available tools like Weka (Witten and Frank, 1999) allow easy experimentation with common learning algorithms (Hendrickx et al., 2007).
    Page 3, “Related Work”
  4. 5.3 Parameters and Learning Algorithm
    Page 7, “Experimental Setup”
  5. Selection of learning algorithm and its algorithm-specific parameters were done as follows.
    Page 7, “Experimental Setup”
  6. Since each dataset has only 140 examples, the computation time of each learning algorithm is negligible.
    Page 7, “Experimental Setup”

See all papers in Proc. ACL 2008 that mention learning algorithms.

See all papers in Proc. ACL that mention learning algorithms.

Back to top.

f-score

Appears in 4 sentences as: F-score (1) f-score (3)
In Classification of Semantic Relationships between Nominals Using Pattern Clusters
  1. Our NR classification evaluation strictly follows the ACL SemEval-07 Task 4 datasets and protocol, obtaining an f-score of 70.6, as opposed to 64.8 of the best previous work that did not use the manually provided WordNet sense disambiguation tags.
    Page 1, “Abstract”
  2. In fact, our results ( f-score 62.0, accuracy 64.5) are better than the averaged results (58.0, 61.1) of the group that did not utilize WN tags.
    Page 8, “Results”
  3. Table 2 shows the HITS-based classification results ( F-score and Accuracy) and the number of positively labeled clusters (C) for each relation.
    Page 8, “Results”
  4. We have used the exact evaluation procedure described in (Turney, 2006), achieving a class f-score average of 60.1, as opposed to 54.6 in (Turney, 2005) and 51.2 in (Nastase et al., 2006).
    Page 8, “Results”

See all papers in Proc. ACL 2008 that mention f-score.

See all papers in Proc. ACL that mention f-score.

Back to top.

feature vectors

Appears in 4 sentences as: feature vectors (4)
In Classification of Semantic Relationships between Nominals Using Pattern Clusters
  1. To do this, we construct feature vectors from each training pair, where each feature is the HITS measure corresponding to a single pattern cluster.
    Page 6, “Relationship Classification”
  2. Once we have feature vectors , we can use a variety of classifiers (we used those in Weka) to construct a model and to evaluate it on the test set.
    Page 6, “Relationship Classification”
  3. If we are not given any training set, it is still possible to separate between different relationship types by grouping the feature vectors of Section 4.3.2 into clusters.
    Page 6, “Relationship Classification”
  4. This can be done by applying k-means or another clustering algorithm to the feature vectors described above.
    Page 6, “Relationship Classification”

See all papers in Proc. ACL 2008 that mention feature vectors.

See all papers in Proc. ACL that mention feature vectors.

Back to top.

word pair

Appears in 4 sentences as: word pair (2) word pairs (2)
In Classification of Semantic Relationships between Nominals Using Pattern Clusters
  1. The standard classification process is to find in an auxiliary corpus a set of patterns in which a given training word pair co-appears, and use pattern-word pair co-appearance statistics as features for machine learning algorithms.
    Page 1, “Introduction”
  2. Co-appearance of nominal pairs can be very rare (in fact, some word pairs in the Task 4 set co-appear only once in Yahoo web search).
    Page 5, “Relationship Classification”
  3. To enrich the set of given word pairs and patterns as described in Section 4.1 and to perform clarifying queries, we utilize the Yahoo API for web queries.
    Page 7, “Experimental Setup”
  4. If only several links were found for a given word pair we perform local crawling to depth 3 in an attempt to discover more instances.
    Page 7, “Experimental Setup”

See all papers in Proc. ACL 2008 that mention word pair.

See all papers in Proc. ACL that mention word pair.

Back to top.

machine learning

Appears in 3 sentences as: machine learning (3)
In Classification of Semantic Relationships between Nominals Using Pattern Clusters
  1. The standard classification process is to find in an auxiliary corpus a set of patterns in which a given training word pair co-appears, and use pattern-word pair co-appearance statistics as features for machine learning algorithms.
    Page 1, “Introduction”
  2. In this paper, we use these pattern clusters as the (only) source of machine learning features for a nominal relationship classification problem.
    Page 3, “Related Work”
  3. In this method we treat the HITS measure for a cluster as a feature for a machine learning classification
    Page 6, “Relationship Classification”

See all papers in Proc. ACL 2008 that mention machine learning.

See all papers in Proc. ACL that mention machine learning.

Back to top.

manually annotated

Appears in 3 sentences as: manually annotated (3)
In Classification of Semantic Relationships between Nominals Using Pattern Clusters
  1. Some of the teams have used the manually annotated WN labels provided with the dataset, and some have not.
    Page 2, “Introduction”
  2. In this paper we do not use any manually annotated resources apart from the classification training set.
    Page 2, “Related Work”
  3. This manually annotated dataset includes a representative rather than exhaustive list of 7 important nominal relationships.
    Page 3, “Related Work”

See all papers in Proc. ACL 2008 that mention manually annotated.

See all papers in Proc. ACL that mention manually annotated.

Back to top.

sense disambiguation

Appears in 3 sentences as: sense disambiguation (3)
In Classification of Semantic Relationships between Nominals Using Pattern Clusters
  1. Our NR classification evaluation strictly follows the ACL SemEval-07 Task 4 datasets and protocol, obtaining an f-score of 70.6, as opposed to 64.8 of the best previous work that did not use the manually provided WordNet sense disambiguation tags.
    Page 1, “Abstract”
  2. Furthermore, usage of such resources frequently requires disambiguation and connection of the data to the resource (word sense disambiguation in the case of WordNet).
    Page 1, “Introduction”
  3. In practical situations, it would not be feasible to provide a large amount of such sense disambiguation tags manually.
    Page 8, “Conclusion”

See all papers in Proc. ACL 2008 that mention sense disambiguation.

See all papers in Proc. ACL that mention sense disambiguation.

Back to top.

PDF available from www.aclweb.org.
Send us an email