This paper proposes a new approach to domain adaptation in statistical machine translation (SMT) based on a vector space model (VSM).

In our experiments, we based the vector space on subcorpora defined by the nature of the training data.

This was done purely out of convenience: there are many, many ways to define a vector space in this situation.

An obvious and appealing one, which we intend to try in future, is a vector space based on a bag-of-words topic model.

In this paper, we propose a new instance weighting approach to domain adaptation based on a vector space model (VSM).

The vector space used by VSM adaptation can be defined in various ways.

More fundamentally, there is nothing about the VSM idea that obliges us to define the vector space in terms of subcorpora.

Vector space models (VSMs) have been widely applied in many information retrieval and natural language processing applications.

Therefore, even within the variant of VSM adaptation we focus on in this paper, where the definition of the vector space is based on the existence of subcorpora, one could utilize other definitions of the vectors of the similarity function than those we utilized in our experiments.

The sense similarity scores are computed by using the vector space model.

In this paper, we have proposed an approach that uses the vector space model to compute the sense

Given two terms to be compared, one first extracts various features for each term from their contexts in a corpus and forms a vector space model (VSM); then, one computes their similarity by using similarity functions.

Use of the vector space model to compute sense similarity has also been adapted to the multilingual condition, based on the assumption that two terms with similar meanings often occur in comparable contexts across languages.

4.2 Vector Space Mapping

A common way to calculate semantic similarity is by vector space cosine distance; we will also

Fung (1998) and Rapp (1999) map the vector one-dimension-to-one-dimension (a context word is a dimension in each vector space ) from one language to another language via an initial bilingual dictionary.

’LU2 is the kth most-similar word to ml in the vector space .

Document-based models divide a corpus into discrete documents and construct the vector space from word frequencies in the documents.

Co-occurrence models build the vector space using the distribution of co-occurring words in a context, which is typically defined as a region around a word or paths rooted in a parse tree.

WSI models also use co-occurrence but also attempt to discover distinct word senses while building the vector space .

Figure 1 illustrates the shared algorithmic structure of all the approaches, which is divided into four components: corpus processing, context selection, feature extraction and global vector space operations.

Feature extraction determines the dimensions of the vector space by selecting which tokens in the context will count as features.

Global vector space operations are applied to the entire space once the initial word features have been computed.

We present two simple paraphrase models, an association model and a vector space model, and train them jointly from question-answer pairs.

We use two complementary paraphrase models: an association model based on aligned phrase pairs extracted from a monolingual parallel corpus, and a vector space model, which represents each utterance as a vector and learns a similarity score between them.

Our framework accommodates any paraphrasing method, and in this paper we propose an association model that learns to associate natural language phrases that co-occur frequently in a monolingual parallel corpus, combined with a vector space model, which learns to score the similarity between vector representations of natural language utterances (Section 5).

The NLP paraphrase literature is vast and ranges from simple methods employing surface features (Wan et al., 2006), through vector space models (Socher et al., 2011), to latent variable models (Das and Smith, 2009; Wang and Manning, 2010; Stern and Dagan, 2011).

Our paraphrase model decomposes into an association model and a vector space model:

5.2 Vector space model

(2013) in that it uses additional features ( vector space word representations) and a different classification method (we use random forests while Tsvetkov et al.

0 Vector space word representations.

Vector space word representations learned using unsupervised algorithms are often effective features in supervised learning methods (Turian et al., 2010).

(2013) reveal an interesting cross-lingual property of distributed word representations: there is a strong similarity between the vector spaces across languages that can be easily captured by linear mapping.

(2013), we use a logistic regression classifier to propagate abstractness and imageability scores from MRC ratings to all words for which we have vector space representations.

More specifically, we calculate the degree of abstractness and imageability of all English items that have a vector space representation, using vector elements as features.

Vector space word representations.

In this paper we draw upon recent advances in the learning of vector space representations of sentential semantics and the transparent interface between syntax and semantics provided by Combinatory Categorial Grammar to introduce Combinatory Categorial Autoencoders.

2.2 Vector Space Models of Semantics

Vector space models of compositional semantics aim to fill this gap by providing a methodology for deriving the representation of an expression from those of its parts.

There are a number of ideas on how to define composition in such vector spaces .

CCG-Vector Interface Exactly how the information contained in a CCG derivation is best applied to a vector space model of compositionality is another issue for future research.

In this paper we have brought a more formal notion of semantic compositionality to vector space models based on recursive autoencoders.

While the connections between formal linguistics and vector space approaches to NLP may not be immediately obvious, we believe that there is a case for the continued investigation of ways to best combine these two schools of thought.

tions: Can recursive vector space models be reconciled with a more formal notion of compositionality; and is there a role for syntax in guiding semantics in these types of models?

Similarly, in the recent work on common vector spaces for the representation of images and text, the current emphasis is on retrieving existing captions (Socher et al., 2014) and not actual generation of image descriptions.

Some research has already established a connection between neural and distributional semantic vector spaces (Mitchell et al., 2008; Murphy et al., 2012).

For the rest of this section we describe the construction of the vector spaces and the (de)composition function learning procedure.

Construction of vector spaces We test two types of vector representations.

Translation is another potential application of the generation framework: Given a semantic space shared between two or more languages, one can compose a word sequence in one language and generate translations in another, with the shared semantic vector space functioning as interlingua.

Creation of cross-lingual vector spaces A common semantic space is required in order to map words and phrases across languages.

This method is applicable to count-type vector spaces , for which the dimen-

To compute the vector space , we consider only a subset of the complete set of dependency triples extracted from the parsed Gigaword corpus.

We go one step further, however, in that we employ syntactically enriched vector models as the basic meaning representations, assuming a vector space spanned by combinations of dependency relations and words (Lin, 1998).

For the problem at hand, the use of second-order vectors alleviates the sparseness problem, and enables the definition of vector space transformations that make the distributional information attached to words in different syntactic positions compatible.

Assuming a set W of words and a set R of dependency relation labels, we consider a Euclidean vector space V1 spanned by the set of orthonormal basis vectors {Enw/ | r E R,w’ E W}, i.e., a vector space whose dimensions correspond to pairs of a relation and a word.

In this vector space we define the first-order vector [w] of a word w as follows:

We further consider a similarly defined vector space V2, spanned by an orthonormal basis {Em/7w, | r, r’ E R,w’ E W}.

Most traditional models are linear models, in the sense that both the features of the data and model parameters are represented as vectors in a vector space .

On the other hand, tensor models have many more degrees of “design freedom” than vector space models.

Most of the learning algorithms for NLP problems are based on vector space models, which represent data as vectors qb E R”, and try to learn feature weight vectors w E R” such that a linear model 3/ = w - qb is able to discriminate between, say, good and bad hypotheses.

A vector space linear model requires estimating 1,000,000 free parameters.

Then the total number of parameters to be learned for this tensor model is H ZdDzl nd, which is usually much smaller than V = HdDzl nd for a traditional vector space model.

In contrast, vector space models of distributional semantics are trained on large corpora, but are typically applied to domain-general lexical disambiguation tasks.

t=1 a=1 3.1 Vector Space Models of Semantics

Simple Vector Space Model In the basic version of the model (SIMPLE), we train a term-context matrix, where rows correspond to target words, and columns correspond to context words.

The most popular approach today is a vector space representation, in which each dimension corresponds to some context word, and the value at that dimension corresponds to the strength of the association between the context word and the target word being modelled.

Vector space models form the basis of modern information retrieval (Salton et al., 1975), but only recently have distributional models been proposed that are compositional (Mitchell and Lapata, 2008; Clark et al., 2008; Grefenstette and Sadrzadeh, 2011, inter alia), or that contextualize the meaning of a word using other words in the same phrase (co-compositionality) (Erk and Padé, 2008; Dinu and Lapata, 2010; Thater et al., 2011).

Experimental results indicate that our approach of identifying question topic and question focus for search significantly outperforms the baseline methods such as Vector Space Model (VSM) and Language Model for Information Retrieval (LMIR).

To obtain the ground-truth of question search, we employed the Vector Space Model (VSM) (Salton et al., 1975) to retrieve the top 20 results and obtained manual judgments.

vector space model, Okapi, language model, and translation-based model, within the setting of question search (Jeon et al., 2005b).

Conventional vector space models are used to calculate the statistical similarity and WordNet (Fellbaum, 1998) is used to estimate the semantic similarity.

vector space model, Okapi, language model (LM), and translation-based model, for automatically fixing the lexical chasm between

This suggests that clustering the attitude vector space will achieve the goal and split the discussants into subgroups according to their opinion.

We collect all the text posted by each participant and create a tf-idf representations of the text in a high dimensional vector space .

We then cluster the vector space to identify subgroups.

We use clustering techniques to cluster the attitude vector space .

For comparison, we implemented several alternative vector space models that are conceptually similar to our own, as discussed in section 2:

The logistic regression weights 2p and be define a linear hyperplane in the word vector space where a word vector’s positive sentiment probability depends on where it lies with respect to this hyperplane.

ing sentiment-imbued topics rather than embedding words in a vector space .

Vector space models (VSMs) seek to model words directly (Turney and Pantel, 2010).

Let (1)3, CDT 6 RIVIXK be the source and target embedding matrices respectively, where K is the dimension of the word vector space , identical in the source and target embeddings, and V is the set of embedded words, given by V5 0 VT.

There are almost no restrictions on (133, except that it must match the desired target vector space dimension K. The objective is convex in w and (PT, thus, yielding a unique target re-embedding.

We use the document’s binary bag-of-words vector vj, and compute the document’s vector space representation through the matrix-vector product (Dij.

While a smaller number of dimensions has been shown to work better in other tasks (Turian et a1., 2010), re-embedding words may benefit from a larger initial dimension of the word vector space .

A great variety of linguistic models are subsumed by this general idea ranging from purely symbolic approaches (like type systems and cate-gorial grammars) to rather statistical models (like vector space and word space models).

In computational linguistics and information retrieval, Vector Space Models (Salton et al., 1975) and its variations — such as Word Space Models (Schutze, 1993), Hyperspace Analogue to Language (Lund and Burgess, 1996), or Latent Semantic Analysis (Deerwester et al., 1990) — have become a mainstream paradigm for text representation.

Vector Space Models (VSMs) have been empirically justified by results from cognitive science (Gardenfors, 2000).

Widdows (2008) proposes a number of more advanced vector operations well-known from quantum mechanics, such as tensor product and convolution, to model composition in vector spaces .

If an element of this vector space , and, represents a sentence with liberal bias, its vector should be distinct from the vector :37. of a conservative-leaning sentence.

Random The most straightforward choice is to initialize the word embedding matrix We and composition matrices WL and WR randomly such that without any training, representations for words and phrases are arbitrarily projected into the vector space .

vector space by listing the most probable n-grams for each political affiliation in Table 2.

After converting both queries and documents to concept-based representations using MetaMap, previous work applied existing retrieval functions such as vector space models (Singhal et al., 1996) to rank the documents.

For example, Qi and Laquerre used MetaMap to generate the concept-based representation and then apply a vector space retrieval model for ranking, and their results are one of the top ranked runs in the TREC 2012 Medical Records track (Qi and Laquerre, 2012).

However, existing studies on concept-based representation still used weighting strategies developed for term-based representation such as vector space models (Qi and Laquerre, 2012) and divergence from randomness (DFR) (Limsopatham et al., 2013a) and did not take the inaccurate concept mapping results into consideration.

We search for the stochastic gradient learning rate in {0.0001, 0.001, 0.01}, the margin 7 E {0.001, m, 0.1, l} and the dimensionality of the final vector space m E {E, 512}, to maximize the frame identification accuracy of ambiguous lexical units; by ambiguous, we imply lexical units that appear in the training data or the lexicon with more than one semantic frame.

First, we extract the words in the syntactic context of runs; next, we concatenate their word embeddings as described in §2.2 to create an initial vector space representation.

This set of dependency paths were deemed as possible positions in the initial vector space representation.

By allowing all words in different regions (or more generally, with different metadata factors) to exist in the same vector space , we are able compare different points in that space — for example, to ask what terms used in Chicago are most similar to hot dog in New York, or what word groups shift together in the same region in comparison to the background (indicating the shift of an entire semantic field).

In all experiments, the contextual variable is the observed US state (including DC), so that |C| = 51; the vector space representation of word w in state 3 is wTWmam + wTWS.

be in the same vector space and can therefore be compared to each other; with individual models (each with different initializations), word vectors across different states may not be directly compared.

Among various word similarity models (Agirre et al., 2009; Reisinger and Mooney, 2010; Gabrilovich and Markovitch, 2007; Radinsky et al., 2011), the vector space models (VSMs) based on the idea of distributional similarity (Turney and Pantel, 2010) are often used as the core component.

Inspired by (Yih and Qazvinian, 2012), which argues the importance of incorporating heterogeneous vector space models for measuring word similarity, we leverage three different VSMs in this work: Wiki term-vectors, recurrent neural

network language model (RNNLM) and a concept vector space model learned from click-through data.

As regards the nature of this task, a vector space model (VSM) is a straightforward and suitable representation for statistical learning.

Our experiments demonstrated that it is indeed a good idea to include longer phrases in the vector space model representation of sentences.

Since hedge cues cause systems to predict false positive labels, our idea here was to train Maximum Entropy Models for the false positive classifications of our ICD-9-CM coding system using the vector space representation of radiology reports.