Article Structure
Abstract
This paper proposes a framework for representing the meaning of phrases and sentences in vector space.
Introduction
Vector-based models of word meaning (Lund and Burgess, 1996; Landauer and Dumais, 1997) have become increasingly popular in natural language processing (NLP) and cognitive science.
Related Work
The problem of vector composition has received some attention in the connectionist literature, particularly in response to criticisms of the ability of con-nectionist representations to handle complex structures (Fodor and Pylyshyn, 1988).
Composition Models
We formulate semantic composition as a function of two vectors, u and v. We assume that individual words are represented by vectors acquired from a corpus following any of the parametrisa-tions that have been suggested in the literature.1 We briefly note here that a word’s vector typically represents its co-occurrence with neighboring words.
Evaluation Setup
We evaluated the models presented in Section 3 on a sentence similarity task initially proposed by Kintsch (2001).
Results
Our experiments assessed the performance of seven composition models.
Discussion
In this paper we presented a general framework for vector-based semantic composition.
Topics
semantic space
- Moreover, the vector similarities within such semantic spaces have been shown to substantially correlate with human similarity judgments (McDonald, 2000) and word association norms (Denhire and Lemaire, 2004).Page 1, “Introduction”
- Figure l: A hypothetical semantic space for horse and runPage 3, “Related Work”
- The construction of the semantic space depends on the definition of linguistic context (e.g., neighbour-ing words can be documents or collocations), the number of components used (e.g., the k most frequent words in a corpus), and their values (e.g., as raw co-occurrence frequencies or ratios of probabilities).Page 3, “Composition Models”
- A hypothetical semantic space is illustrated in Figure 1.Page 3, “Composition Models”
- 1A detailed treatment of existing semantic space models is outside the scope of the present paper.Page 3, “Composition Models”
- This change in the verb’s sense is equated to a shift in its position in semantic space .Page 5, “Evaluation Setup”
- Model Parameters Irrespectiver of their form, all composition models discussed here are based on a semantic space for representing the meanings of individual words.Page 6, “Evaluation Setup”
- The semantic space we used in our experiments was built on a lemmatised version of the BNC.Page 6, “Evaluation Setup”
- rameters beyond the semantic space just described, with three exceptions.Page 7, “Evaluation Setup”
See all papers in Proc. ACL 2008 that mention semantic space.
See all papers in Proc. ACL that mention semantic space.
Back to top.
human judgments
- Experimental results demonstrate that the multiplicative models are superior to the additive alternatives when compared against human judgments .Page 1, “Abstract”
- The task involves examining the degree of linear relationship between the human judgments for two individual words and vector-based similarity values.Page 6, “Evaluation Setup”
- We assume that the inter-subject agreement can serve as an upper bound for comparing the fit of our models against the human judgments .Page 7, “Evaluation Setup”
- Table 2: Model means for High and Low similarity items and correlation coefficients with human judgments (*z p < 0.05, **: p < 0.01)Page 7, “Results”
See all papers in Proc. ACL 2008 that mention human judgments.
See all papers in Proc. ACL that mention human judgments.
Back to top.
semantic similarity
- The appeal of these models lies in their ability to represent meaning simply by using distributional information under the assumption that words occurring within similar contexts are semantically similar (Harris, 1968).Page 1, “Introduction”
- 3We assessed a wide range of semantic similarity measures using the WordNet similarity package (Pedersen et a1., 2004).Page 5, “Evaluation Setup”
- Following previous work (Bullinaria and Levy, 2007), we optimized its parameters on a word-based semantic similarity task.Page 6, “Evaluation Setup”
- The resulting vector is sparser but expresses more succinctly the meaning of the predicate-argument structure, and thus allows semantic similarity to be modelled more accurately.Page 8, “Discussion”
See all papers in Proc. ACL 2008 that mention semantic similarity.
See all papers in Proc. ACL that mention semantic similarity.
Back to top.
co-occurrence
- We formulate semantic composition as a function of two vectors, u and v. We assume that individual words are represented by vectors acquired from a corpus following any of the parametrisa-tions that have been suggested in the literature.1 We briefly note here that a word’s vector typically represents its co-occurrence with neighboring words.Page 3, “Composition Models”
- The construction of the semantic space depends on the definition of linguistic context (e.g., neighbour-ing words can be documents or collocations), the number of components used (e.g., the k most frequent words in a corpus), and their values (e.g., as raw co-occurrence frequencies or ratios of probabilities).Page 3, “Composition Models”
- Here, the space has only five dimensions, and the matrix cells denote the co-occurrence of the target words (horse and run) with the context words animal, stable, and so on.Page 3, “Composition Models”
See all papers in Proc. ACL 2008 that mention co-occurrence.
See all papers in Proc. ACL that mention co-occurrence.
Back to top.
statistically significant
- A Wilcoxon rank sum test confirmed that the difference is statistically significant (p < 0.01).Page 6, “Evaluation Setup”
- The difference between High and Low similarity values estimated by these models are statistically significant (p < 0.01 using the Wilcoxon rank sum test).Page 7, “Results”
- However, the difference between the two models is not statistically significant .Page 8, “Results”
See all papers in Proc. ACL 2008 that mention statistically significant.
See all papers in Proc. ACL that mention statistically significant.
Back to top.
