This paper proposes a new approach to domain adaptation in statistical machine translation (SMT) based on a vector space model (VSM).

In our experiments, we based the vector space on subcorpora defined by the nature of the training data.

This was done purely out of convenience: there are many, many ways to define a vector space in this situation.

An obvious and appealing one, which we intend to try in future, is a vector space based on a bag-of-words topic model.

In this paper, we propose a new instance weighting approach to domain adaptation based on a vector space model (VSM).

The vector space used by VSM adaptation can be defined in various ways.

More fundamentally, there is nothing about the VSM idea that obliges us to define the vector space in terms of subcorpora.

Vector space models (VSMs) have been widely applied in many information retrieval and natural language processing applications.

Therefore, even within the variant of VSM adaptation we focus on in this paper, where the definition of the vector space is based on the existence of subcorpora, one could utilize other definitions of the vectors of the similarity function than those we utilized in our experiments.

In this paper we draw upon recent advances in the learning of vector space representations of sentential semantics and the transparent interface between syntax and semantics provided by Combinatory Categorial Grammar to introduce Combinatory Categorial Autoencoders.

2.2 Vector Space Models of Semantics

Vector space models of compositional semantics aim to fill this gap by providing a methodology for deriving the representation of an expression from those of its parts.

There are a number of ideas on how to define composition in such vector spaces .

CCG-Vector Interface Exactly how the information contained in a CCG derivation is best applied to a vector space model of compositionality is another issue for future research.

In this paper we have brought a more formal notion of semantic compositionality to vector space models based on recursive autoencoders.

While the connections between formal linguistics and vector space approaches to NLP may not be immediately obvious, we believe that there is a case for the continued investigation of ways to best combine these two schools of thought.

tions: Can recursive vector space models be reconciled with a more formal notion of compositionality; and is there a role for syntax in guiding semantics in these types of models?

In contrast, vector space models of distributional semantics are trained on large corpora, but are typically applied to domain-general lexical disambiguation tasks.

t=1 a=1 3.1 Vector Space Models of Semantics

Simple Vector Space Model In the basic version of the model (SIMPLE), we train a term-context matrix, where rows correspond to target words, and columns correspond to context words.

The most popular approach today is a vector space representation, in which each dimension corresponds to some context word, and the value at that dimension corresponds to the strength of the association between the context word and the target word being modelled.

Vector space models form the basis of modern information retrieval (Salton et al., 1975), but only recently have distributional models been proposed that are compositional (Mitchell and Lapata, 2008; Clark et al., 2008; Grefenstette and Sadrzadeh, 2011, inter alia), or that contextualize the meaning of a word using other words in the same phrase (co-compositionality) (Erk and Padé, 2008; Dinu and Lapata, 2010; Thater et al., 2011).

Let (1)3, CDT 6 RIVIXK be the source and target embedding matrices respectively, where K is the dimension of the word vector space , identical in the source and target embeddings, and V is the set of embedded words, given by V5 0 VT.

There are almost no restrictions on (133, except that it must match the desired target vector space dimension K. The objective is convex in w and (PT, thus, yielding a unique target re-embedding.

We use the document’s binary bag-of-words vector vj, and compute the document’s vector space representation through the matrix-vector product (Dij.

While a smaller number of dimensions has been shown to work better in other tasks (Turian et a1., 2010), re-embedding words may benefit from a larger initial dimension of the word vector space .

Among various word similarity models (Agirre et al., 2009; Reisinger and Mooney, 2010; Gabrilovich and Markovitch, 2007; Radinsky et al., 2011), the vector space models (VSMs) based on the idea of distributional similarity (Turney and Pantel, 2010) are often used as the core component.

Inspired by (Yih and Qazvinian, 2012), which argues the importance of incorporating heterogeneous vector space models for measuring word similarity, we leverage three different VSMs in this work: Wiki term-vectors, recurrent neural

network language model (RNNLM) and a concept vector space model learned from click-through data.