This has fostered the development of domain adaptation techniques for relation extraction.

Moreover, in all the cases, regularization methods are still effective for domain adaptation of RE.

6.3 Domain Adaptation with Word Embeddings

This is where we need to resort to domain adaptation techniques (DA) to adapt a model trained on one domain (the

Unfortunately, there is very little work on domain adaptation for RE.

as word clusters in domain adaptation of RE (Plank and Moschitti, 2013) is motivated by its successes in semi-supervised methods (Chan and Roth, 2010; Sun et al., 2011) where word representations help to reduce data-sparseness of lexical information in the training data.

Exploiting the shared interest in generalization performance with traditional machine learning, in domain adaptation for RE, we would prefer the relation extractor that fits the source domain data, but also circumvents the overfitting problem

However, none of these works evaluate word embeddings for domain adaptation

Regarding domain adaptation , in representation

Above all, we move one step further by evaluating the effectiveness of word embeddings on domain adaptation for RE which is very different from the principal topic of sequence labeling in the previous research.

This is the problem of domain adaptation .

The empirical evaluation on ACE 2005 domains shows that a suitable combination of syntax and lexical generalization is very promising for domain adaptation .

This is the problem of domain adaptation (DA) or transfer learning (TL).

Technically, domain adaptation addresses the problem of leam-ing when the assumption of independent and identically distributed (i.i.d.)

Domain adaptation has been studied extensively during the last couple of years for various NLP tasks, e.g.

In this paper, we evaluate performance on a domain adaptation setting where we translate sentences from the medical domain.

Our experimental results show that ensemble decoding outperforms various strong baselines including mixture models, the current state-of-the-art for domain adaptation in machine translation.

In this paper, we presented a new approach for domain adaptation using ensemble decoding.

Each of these mixture operations has a specific property that makes it work in specific domain adaptation or system combination scenarios.

For instance, LOPs may not be optimal for domain adaptation in the setting where there are two or more models trained on heterogeneous corpora.

Domain adaptation techniques aim at finding ways to adjust an out-of-domain (OUT) model to represent a target domain (in-domain or IN).

We expect domain adaptation for machine translation can be improved further by combining orthogonal techniques for translation model adaptation combined with language model adaptation.

The main applications of ensemble models are domain adaptation , domain mixing and system combination.

Early approaches to domain adaptation involved information retrieval techniques where sentence pairs related to the target domain were retrieved from the training corpus using IR methods (Eck et al., 2004; Hildebrand et al., 2005).

Other domain adaptation methods involve techniques that distinguish between general and domain-specific examples (Daumé and Marcu, 2006).

We compare our framework with several other methods, including state-of-the-art machine learning, relation extraction and common domain adaptation methods.

Adaptive domain bootstrapping (DAB) This is an instance-based domain adaptation method for relation extraction (Xu et al., 2010).

Structural correspondence learning (SCL) We use the feature-based domain adaptation approach by Blitzer et a1.

To tackle these challenges, we propose a two-phase Robust Domain Adaptation (RDA) framework.

We compare the proposed two-phase framework with state-of-the-art domain adaptation baselines for the relation extraction task, and we find that our method outperforms the baselines.

This section defines the domain adaptation problem and describes our feature extraction scheme.

3.1 Relation Extraction Domain Adaptation

We define domain adaptation as the problem of learning a classifier p for relation extraction in the target domain using the data sets D1, Du and DS, 5 = 1,. .

We address this by augmenting a small labeled data set with other information in the domain adaptation setting.

Domain adaptation methods can be classified broadly into weakly-supervised adaptation (Daume and Marcu, 2007; Blitzer et al., 2006; Jiang and Zhai, 2007a; Jiang, 2009), and unsupervised adaptation (Pan et al., 2010; Blitzer et al., 2006; Plank and Moschitti, 2013).

, fc using the one-versus-rest decoding for multi-class classification.2 Inspired by the Domain Adaptive Machine (Duan et al., 2009), we combine the reference predictions and the labeled data of the target domain to learn these functions:

The fact that the J ST model does not required any labeled documents for training makes it desirable for domain adaptation in sentiment classification.

We proceed with a review of related work on sentiment domain adaptation .

We subsequently show that words from different domains can indeed be grouped under the same polarity-bearing topic through an illustration of example topic words extracted by J ST before proposing a domain adaptation approach based on J ST. We verify our proposed approach by conducting experiments on both the movie review data

5 Domain Adaptation using J ST

Given input data cc and a class label 3/, labeled patterns of one domain can be drawn from the joint distribution P(:c, y) = Domain adaptation usually assume that data distribution are different in source and target domains, i.e., Ps(x) 75 Pt(:c).

The task of domain adaptation is to predict the label corresponding to in the target domain.

There has been significant amount of work on algorithms for domain adaptation in NLP.

for domain adaptation where a mixture model is defined to learn differences between domains.

proposed structural correspondence learning (SCL) for domain adaptation in sentiment classification.

We also discuss other approaches to improve unsupervised domain adaptation for SMT.

To our knowledge, however, this is the first work to use multilingual topic models for domain adaptation in machine translation.

Domain adaptation for language models (Bellegarda, 2004; Wood and Teh, 2009) is an important avenue for improving machine translation.

Domain Adaptation using Topic Models We examine the effectiveness of using topic models for domain adaptation on standard SMT evaluation metrics—BLEU (Papineni et al., 2002) and TER (Snover et al., 2006).

We refer to the SMT model without domain adaptation as baseline.5 LDA marginally improves machine translation (less than half a BLEU point).

Systems that are robust to systematic variation in the training set are said to exhibit domain adaptation .

We show that ptLDA offers better domain adaptation than other topic models for machine translation.

Before considering past approaches using topic models to improve SMT, we briefly review lexical weighting and domain adaptation for SMT.

Domain Adaptation for SMT Training a SMT system using diverse data requires domain adaptation .

This obviates the explicit smoothing used in other domain adaptation systems (Chiang et al., 2011).

While domain adaptation techniques for SMT have proven to be effective at improving translation quality, their practicality for a multi-domain environment is often limited because of the computational and human costs of developing and maintaining multiple systems adapted to different domains.

The effectiveness of domain adaptation approaches such as mixture-modeling (Foster and Kuhn, 2007) has been established, and has led to research on a wide array of adaptation techniques in SMT, for instance (Matsoukas et al., 2009; Shah et al., 2012).

Therefore, when working with multiple and/or unlabelled domains, domain adaptation is often impractical for a number of reasons.

Secondly, domain adaptation bears a risk of performance loss.

Our immediate purpose for this paper is domain adaptation in a multi-domain environment, but the delay of the feature computation has other potential applications, e.g.

The goal is to perform domain adaptation without requiring domain labels or user input, neither for development nor decoding.

Our theoretical expectation is that domain adaptation will fail to perform well if the test data is from

Many other domain adaptations techniques exist and may produce language models with better performance.

This approach has the benefit of simplicity, however, better performance for combining related corpora has been seen by domain adaptation techniques which combine the data in more structured ways (Bacchiani and Roark, 2003).

Our goal for this paper is not to explore domain adaptation techniques, but to determine if normal data is useful for the simple language modeling task.

if domain adaptation techniques may be useful, we also investigated a linearly interpolated language model.

If we view the normal data as out-of-domain data, then the problem of combining simple and normal data is similar to the language model domain adaption problem (Suzuki and Gao, 2005), in particular cross-domain adaptation (Bellegarda, 2004) where a domain-specific model is improved by incorporating additional general data.

guage model domain adaptation problem for text simplification.

Pan and Yang (2010) provide a survey on the related problem of domain adaptation for machine learning (also referred to as “transfer learning”), which utilizes similar techniques.

We extend an existing MSA segmenter with a simple domain adaptation technique and new features in order to segment informal and dialectal Arabic text.

2.2 Domain adaptation

The approach to domain adaptation we use is that of feature space augmentation (Daumé, 2007).

We sampled 100 errors randomly from all errors made by our final model (trained on all three datasets with domain adaptation and additional features) on the ARZ development set; see Table 4.

In this paper we demonstrate substantial gains on Arabic clitic segmentation for both formal and dialectal text using a single model with dialect-independent features and a simple domain adaptation strategy.

However, as data for other Arabic dialects and genres becomes available, we expect that the model’s simplicity and the domain adaptation method we use will allow the system to be applied to these dialects with minimal effort and without a loss of performance in the original domains.

Using domain adaptation alone helps performance on two of the three datasets (with a statistically insignificant decrease on broadcast news), and that our additional features further improve

Third, we show that dialectal data can be handled in the framework of domain adaptation .

We consider a semi-supervised setting for domain adaptation where only unlabeled data is available for the target domain.

As we discussed in the introduction, our goal is to provide a method for domain adaptation based on semi-supervised learning of models with distributed representations.

In this section, we first discuss the shortcomings of domain adaptation with the above-described semi-supervised approach and motivate constraints on inter-domain variability of

Another motivation for the form of regularization we propose originates from theoretical analysis of the domain adaptation problems (Ben-David et al., 2010; Mansour et al., 2009; Blitzer et al., 2007).

One of the most promising research directions on domain adaptation for this setting is based on the idea of inducing a shared feature representation (Blitzer et al., 2006), that is mapping from the initial feature representation to a new representation such that (l) examples from both domains ‘look similar’ and (2) an accurate classifier can be trained in this new representation.

There is a growing body of work on domain adaptation .

Such methods tackle domain adaptation by instance re-weighting (Bickel et al., 2007; Jiang and Zhai, 2007), or, similarly, by feature re-weighting (Sat-pal and Sarawagi, 2007).

Semi-supervised leam-ing with distributed representations and its application to domain adaptation has previously been considered in (Huang and Yates, 2009), but no attempt has been made to address problems specific to the domain-adaptation setting.

In this paper, we propose a domain adaptation framework for sentiment- and topic- lexicon co-extraction in a domain of interest where we do not require any labeled data, but have lots of labeled data in another related domain.

Experimental results show that our domain adaptation framework can extract precise lexicons in the target domain without any annotation.

To address this problem, we propose a two-stage domain adaptation method.

While, most of previous work focused on document level; 2) A new two-step domain adaptation framework, with a novel RAP algorithm for seed expansion, is proposed.

2.2 Domain Adaptation

Even the performance of the pialign-linear is better than the Baseline GIZA-linear’s, which means that phrase pair extraction with hierarchical phrasal ITGs and sampling is more suitable for domain adaptation tasks than the combination GIZA++ and a heuristic method.

In traditional domain adaptation approaches, phrase pairs are extracted together with their probabilities and/or frequencies so that the extracted phrase pairs are merged uniformly or after scaling.

Traditional domain adaption methods for SMT are also not adequate in this scenario.

A number of approaches have been proposed to make use of the full potential of the available parallel sentences from various domains, such as domain adaptation and incremental learning for SMT.

In the case of the previous work on translation modeling, mixed methods have been investigated for domain adaptation in SMT by adding domain information as additional labels to the original phrase table (Foster and Kuhn, 2007).

As a way to choose the right domain for the domain adaption , a classifier-based method and a feature-based method have been proposed.

We also found that accuracy improvements vary when parsers are retrained with domain-specific data, indicating the importance of domain adaptation and the differences in the portability of parser training methods.

2006), the C&C parser (Clark and Curran, 2004), the XLE parser (Kaplan et al., 2004), MINIPAR (Lin, 1998), and Link Parser (Sleator and Temperley, 1993; Pyysalo et al., 2006), but the domain adaptation of these parsers is not straightforward.

We also measure the accuracy improvements obtained by parser retraining with GENIA, to examine the domain portability, and to evaluate the effectiveness of domain adaptation .

Accuracy improvements in this setting indicate the possibility of domain adaptation , and the portability of the training methods of the parsers.

When the parsers are retrained with GENIA (Table 2), the accuracy increases significantly, demonstrating that the WSJ-trained parsers are not sufficiently domain-independent, and that domain adaptation is effective.

It is an important observation that the improvements by domain adaptation are larger than the differences among the parsers in the previous experiment.

A large improvement from ENJU to ENJU-GENIA shows the effectiveness of the specifically designed domain adaptation method, suggesting that the other parsers might also benefit from more sophisticated approaches for domain adaptation .

In general, our evaluation methodology can be applied to English parsers based on any framework; however, in this paper, we chose parsers that were originally developed and trained with the Penn Treebank or its variants, since such parsers can be retrained with GENIA, thus allowing for us to investigate the effect of domain adaptation .

Since we focus on the domain adaptation issue, we incorporate a less domain dependent language resource (i.e.

The same dataset has been used for the domain adaptation track of the CoNLL 2007 Shared Task.

This is the other datasets used in the domain adaptation track of the CoNLL 2007 Shared Task.

It should be noted that domain adaptation also presents a challenge to the disambiguation model of the HP SG parser.

In addition, most of the previous work have been focusing on constituent-based parsing, while the domain adaptation of the dependency parsing has not been fully explored.

This not to say that domain adaptation is

We present a new approach to cross-language text classification that builds on structural correspondence learning, a recently proposed theory for domain adaptation .

Our approach builds upon structural correspondence learning, SCL, a recently proposed theory for domain adaptation in the field of natural language processing (Blitzer et al., 2006).

Domain Adaptation Domain adaptation refers to the problem of adapting a statistical classifier trained on data from one (or more) source domains (e.g., newswire texts) to a different target domain (e.g., legal texts).

In the basic domain adaptation setting we are given labeled data from the source domain and unlabeled data from the target domain, and the goal is to train a classifier for the target domain.

The latter setting is referred to as unsupervised domain adaptation .

We apply our method for the domain adaptation task and the extensive experiments show that our proposed method can substantially improve the translation quality.

Extensive experiments on domain adaptation have shown that our method can significantly outperform previous methods which also focus on exploring the in-domain lexicon and monolingual data.

Our purpose is to induce phrase pairs to improve translation quality for domain adaptation .

Finally, they used the learned translation model directly to translate unseen data (Ravi and Knight, 2011; Nuhn et al., 2012) or incorporated the learned bilingual lexicon as a new in-domain translation resource into the phrase-based model which is trained with out-of-domain data to improve the domain adaptation performance in machine translation (Dou and Knight, 2012).

The induced phrase-based model will be used to help domain adaptation for machine translation.

Section 6 will show the detailed experiments for the task of domain adaptation .

In order to induce the phrase pairs from the in-domain monolingual data for domain adaptation , the probabilistic bilingual lexicon is essential.

Inspired by recent work on transfer learning and domain adaptation , in this paper, we study how we can leverage labeled data of some old relation types to help the extraction of a new relation type in a weakly-supervised setting, where only a few seed instances of the new relation type are available.

Domain adaptation is a special case of transfer learning where the leam-ing task remains the same but the distribution

There has been an increasing amount of work on transfer learning and domain adaptation in natural language processing recently.

(2006) proposed a structural correspondence learning method for domain adaptation and applied it to part-of-speech tagging.

One particularly promising area for further study is the combination of smoothing and instance weighting techniques for domain adaptation .

3.3 Domain Adaptation

For our experiment on domain adaptation , we focus on NP chunking and POS tagging, and we use the labeled training data from the CoNLL 2000 shared task as before.

(2006): the semi-supervised Alternating Structural Optimization (ASO) technique and the Structural Correspondence Learning (SCL) technique for domain adaptation .

One of the benefits of our smoothing technique is that it allows for domain adaptation , a topic that has received a great deal of attention from the NLP community recently.

HMM-smoothing improves on the most closely related work, the Structural Correspondence Learning technique for domain adaptation (Blitzer et al., 2006), in experiments.

In the subproblem of domain adaptation , a model trained over a source domain is generalized to perform well on a related target domain, where the two domains’ data are distributed similarly, but not identically.

We introduce the concept of groups of closely-related domains, called genres, and show how inter-genre adaptation is related to domain adaptation .

When only the type of data being examined is allowed to vary (from news articles to e-mails, for example), the problem is called domain adaptation (Daumé III and Marcu, 2006).

0 domain adaptation , where we assume Y (the set of possible labels) is the same for both DSOWCG and Dtafget, while DSOWCG and Dtafget themselves are allowed to vary between domains.

Domain adaptation can be further distinguished by the degree of relatedness between the source and target domains.

This paper proposes a new approach to domain adaptation in statistical machine translation (SMT) based on a vector space model (VSM).

Domain adaptation is an active topic in the natural language processing (NLP) research community.

The 2012 JHU workshop on Domain Adaptation for MT 1 proposed phrase sense disambiguation (PSD) for translation model adaptation.

In this paper, we propose a new instance weighting approach to domain adaptation based on a vector space model (VSM).

ald, 2008), and is also similar to the Pred baseline for domain adaptation in (Daumé III and Marcu, 2006; Daumé III, 2007).

We are especially grateful to Fernando Pereira and the anonymous reviewers for pointing us to relevant domain adaption references.

The second problem, domain adaptation , is very well-studied, e.g.

This method is very similar to some ideas in domain adaptation (Daume III and Marcu, 2006; Daume III, 2007), but we argue that the underlying problems are quite different.

Domain adaptation assumes the labeling guidelines are preserved between the two domains, e.g., an adjective is always labeled as JJ regardless of from Wall Street Journal (WSJ) or Biomedical texts, and only the distributions are different, e. g., the word “control” is most likely a verb in WSJ but often a noun in Biomedical texts (as in “control experiment”).

Our strategy, therefore, enables us to build a classifier more domain adaptive and up to date.

The classifier performs much worse on the domains of chemistry, physics and machinery, it indicates the importance of domain adaptation for word segmentation (Gao et al., 2004; Ma and Way, 2009; Gao et al., 2010).

What is more, since the text on Internet is wide-coveraged and real-time updated, our strategy also helps a word segmenter be more domain adaptive and up to date.

It probably provides a simple and effective domain adaptation strategy for already trained models.

Additionally, we design a cus-tomizable framework to address the often overlooked concept of domain adaptability , and illustrate that the system allows for transfer to new domains with a minimal amount of data and effort.

This work presents a framework for normalization with an eye towards domain adaptation .

The goal is to evaluate the framework in two aspects: (1) usefulness for downstream applications (specifically dependency parsing), and (2) domain adaptability .

Similarly, our work is the first to prioritize domain adaptation during the new wave of text message normalization.

This causes data-mismatch issues and hence provides a perfect testbed for a domain adaptation task.

To evaluate the domain adaptation (DA) approach and to compare with results reported by (Subramanya et al., 2010), we use the first and second half of QuestionBank (Judge et al., 2006) as our development and test sets (target).

5.3.2 Experiment 2: Domain Adaptation Task.

In future, we intend to apply the proposed method to other domain adaptation tasks.

This can be considered to be a lower bound that does not perform domain adaptation .

Compared to single-domain sentiment classification, which has been studied extensively in previous work (Pang and Lee, 2008; Tumey, 2002), cross-domain sentiment classification has only recently received attention in response to advances in the area of domain adaptation .

Aue and Gammon (2005) report a number of empirical tests into domain adaptation of sentiment classifiers using an ensemble of classifiers.

Recently there have been some works on using multiple treebanks for domain adaptation of parsers, where these treebanks have the same grammar formalism (McClosky et al., 2006b; Roark and Bacchiani, 2003).

Recently there have been some studies addressing how to use treebanks with same grammar formalism for domain adaptation of parsers.

Roark and Bachiani (2003) presented count merging and model interpolation techniques for domain adaptation of parsers.

Their results indicated that both unlabeled in-domain data and labeled out-of-domain data can help domain adaptation .

This study contributes to the research on sentiment tagging, domain adaptation , and the development of ensembles of classifiers (l) by proposing a novel approach for sentiment determination at sentence level and delineating the conditions under which greatest synergies among combined classifiers can be achieved, (2) by describing a precision-based technique for assigning differential weights to classifier results on different categories identified by the classifier (i.e., categories of positive vs. negative sentences), and (3) by proposing a new method for sentiment annotation in situations where the annotated in-domain data is scarce and insufficient to ensure adequate performance of the corpus-based classifier, which still remains the preferred choice when large volumes of annotated data are available for system training.

(2007) applied structural correspondence learning (Drezde et al., 2007) to the task of domain adaptation for sentiment classification of product reviews.

In sentiment tagging and related areas, Aue and Gamon (2005) demonstrated that combining classifiers can be a valuable tool in domain adaptation for sentiment analysis.

The first part of this paper reviews the extant literature on domain adaptation in sentiment analysis and highlights promising directions for research.

5.2 Domain Adaptation Analysis

To understand the domain adaptation issue we compared the nonzero weights in the discriminative phrase table (PT) for Ar—En models tuned on bitext5k and MT05/6/ 8.

Second, large bitexts often comprise many text genres (Haddow and Koehn, 2012), a virtue for classical dense MT models but a curse for high dimensional models: bitext tuning can lead to a significant domain adaptation problem when evaluating on standard test sets.

), we also make initial tries for domain adaptation so that our summarization method does not need human-written abstracts for each new meeting domain (e.g.

Domain Adaptation Evaluation.

We further examine our system in domain adaptation scenarios for decision and problem summarization, where we train the system on AMI for use on ICSI, and vice versa.

Domain adaptation (DA) of sentiment classification becomes extremely challenging when the distributions of words in the source and the target domains are very different, because the features learnt from the source domain labeled reviews might not appear in the target domain reviews that must be classified.

We evaluated the proposed method in two domain adaptation tasks: cross-domain POS tagging and cross-domain sentiment classification.

Our experiments show that without requiring any task-specific customisations to our distribution prediction method, it outperforms competitive baselines and achieves comparable results to the current state-of-the-art domain adaptation methods.

Additionally, we considered a setting including a small amount of training data from the target data (i.e., supervised domain adaptation ).

5.2), which give the possibility to study the domain adaptability of the supervised models by training on one category and testing on the other (and vice versa).

Therefore, rather than trying to build a specialized system for every new target domain, as it has been done in most prior work on domain adaptation (Blitzer et al., 2007; Daume, 2007), the domain adaptation problem boils down to finding a more robust system (Sszsgaard and Johannsen, 2012; Plank and Moschitti, 2013).