We present two simple paraphrase models, an association model and a vector space model, and train them jointly from question-answer pairs.

We use two complementary paraphrase models: an association model based on aligned phrase pairs extracted from a monolingual parallel corpus, and a vector space model, which represents each utterance as a vector and learns a similarity score between them.

Our framework accommodates any paraphrasing method, and in this paper we propose an association model that learns to associate natural language phrases that co-occur frequently in a monolingual parallel corpus, combined with a vector space model, which learns to score the similarity between vector representations of natural language utterances (Section 5).

The NLP paraphrase literature is vast and ranges from simple methods employing surface features (Wan et al., 2006), through vector space models (Socher et al., 2011), to latent variable models (Das and Smith, 2009; Wang and Manning, 2010; Stern and Dagan, 2011).

Our paraphrase model decomposes into an association model and a vector space model:

5.2 Vector space model

(2013) in that it uses additional features ( vector space word representations) and a different classification method (we use random forests while Tsvetkov et al.

0 Vector space word representations.

Vector space word representations learned using unsupervised algorithms are often effective features in supervised learning methods (Turian et al., 2010).

(2013) reveal an interesting cross-lingual property of distributed word representations: there is a strong similarity between the vector spaces across languages that can be easily captured by linear mapping.

(2013), we use a logistic regression classifier to propagate abstractness and imageability scores from MRC ratings to all words for which we have vector space representations.

More specifically, we calculate the degree of abstractness and imageability of all English items that have a vector space representation, using vector elements as features.

Vector space word representations.

Similarly, in the recent work on common vector spaces for the representation of images and text, the current emphasis is on retrieving existing captions (Socher et al., 2014) and not actual generation of image descriptions.

Some research has already established a connection between neural and distributional semantic vector spaces (Mitchell et al., 2008; Murphy et al., 2012).

For the rest of this section we describe the construction of the vector spaces and the (de)composition function learning procedure.

Construction of vector spaces We test two types of vector representations.

Translation is another potential application of the generation framework: Given a semantic space shared between two or more languages, one can compose a word sequence in one language and generate translations in another, with the shared semantic vector space functioning as interlingua.

Creation of cross-lingual vector spaces A common semantic space is required in order to map words and phrases across languages.

This method is applicable to count-type vector spaces , for which the dimen-

Most traditional models are linear models, in the sense that both the features of the data and model parameters are represented as vectors in a vector space .

On the other hand, tensor models have many more degrees of “design freedom” than vector space models.

Most of the learning algorithms for NLP problems are based on vector space models, which represent data as vectors qb E R”, and try to learn feature weight vectors w E R” such that a linear model 3/ = w - qb is able to discriminate between, say, good and bad hypotheses.

A vector space linear model requires estimating 1,000,000 free parameters.

Then the total number of parameters to be learned for this tensor model is H ZdDzl nd, which is usually much smaller than V = HdDzl nd for a traditional vector space model.

By allowing all words in different regions (or more generally, with different metadata factors) to exist in the same vector space , we are able compare different points in that space — for example, to ask what terms used in Chicago are most similar to hot dog in New York, or what word groups shift together in the same region in comparison to the background (indicating the shift of an entire semantic field).

In all experiments, the contextual variable is the observed US state (including DC), so that |C| = 51; the vector space representation of word w in state 3 is wTWmam + wTWS.

be in the same vector space and can therefore be compared to each other; with individual models (each with different initializations), word vectors across different states may not be directly compared.

We search for the stochastic gradient learning rate in {0.0001, 0.001, 0.01}, the margin 7 E {0.001, m, 0.1, l} and the dimensionality of the final vector space m E {E, 512}, to maximize the frame identification accuracy of ambiguous lexical units; by ambiguous, we imply lexical units that appear in the training data or the lexicon with more than one semantic frame.

First, we extract the words in the syntactic context of runs; next, we concatenate their word embeddings as described in §2.2 to create an initial vector space representation.

This set of dependency paths were deemed as possible positions in the initial vector space representation.

If an element of this vector space , and, represents a sentence with liberal bias, its vector should be distinct from the vector :37. of a conservative-leaning sentence.

Random The most straightforward choice is to initialize the word embedding matrix We and composition matrices WL and WR randomly such that without any training, representations for words and phrases are arbitrarily projected into the vector space .

vector space by listing the most probable n-grams for each political affiliation in Table 2.

After converting both queries and documents to concept-based representations using MetaMap, previous work applied existing retrieval functions such as vector space models (Singhal et al., 1996) to rank the documents.

For example, Qi and Laquerre used MetaMap to generate the concept-based representation and then apply a vector space retrieval model for ranking, and their results are one of the top ranked runs in the TREC 2012 Medical Records track (Qi and Laquerre, 2012).

However, existing studies on concept-based representation still used weighting strategies developed for term-based representation such as vector space models (Qi and Laquerre, 2012) and divergence from randomness (DFR) (Limsopatham et al., 2013a) and did not take the inaccurate concept mapping results into consideration.