To produce character-level trees for Chinese NLP tasks, we develop a character-based parsing model, which can jointly perform word segmentation, POS tagging and phrase-structure parsing.

We make two extensions to their work to enable joint segmentation, POS tagging and phrase-structure parsing from the character level.

First, we split the original SHIFT action into SHIFT—SEPARATE (t) and SHIFT—APPEND, which jointly perform the word segmentation and POS tagging tasks.

Compared to a pipeline system, the advantages of a joint system include reduction of error propagation, and the integration of segmentation, POS tagging and syntax features.

To analyze word structures in addition to phrase structures, our character-based parser naturally performs joint word segmentation, POS tagging and parsing jointly.

We extend their shift-reduce framework, adding more transition actions for word segmentation and POS tagging , and defining novel features that capture character information.

They made use of this information to help joint word segmentation and POS tagging .

In particular, we mark the original nodes that represent POS tags in CTB-style trees with “-t”, and insert our word structures as unary subnodes of the “-t” nodes.

To perform segmentation and tagging simultaneously in a uniform framework, according to Ng and Low (2004), the tag is composed of a word boundary part, and a POS part, e. g., “B _N N” refers to the first character in a word with POS tag “NN”.

As for the POS tag , we shal-1 use the 33 tags in the Chinese tree bank.

The traditional way of segmentation and tagging is performed in a pipeline approach, first segmenting a sentence into words, and then assigning each word a POS tag .

The pipeline approach is very simple to implement, but frequently causes error propagation, given that wrong seg-mentations in the earlier stage harm the subsequent POS tagging (Ng and Low, 2004).

The joint approaches of word segmentation and POS tagging (joint S&T) are proposed to resolve these two tasks simultaneously.

In fact, the sparsity is also a common phenomenon among character-based CWS and POS tagging .

The performance measurement indicators for word segmentation and POS tagging (joint S&T) are balance F-score, F = 2PIU(P+R), the harmonic mean of precision (P) and recall (R), and out-of-vocabulary recall (OOV—R).

There are few explorations of semi-supervised approaches for CWS or POS tagging in previous works.

We induce number of POS tags of a word type at this step.

Furthermore, they will have the same POS tags .

As a result, this method inaccurately induces POS tags for the occurrences of word types with high gold tag perplexity.

In other words, we assume that the number of different POS tags of each word type is equal to 2.

part-of-speech or POS tagging ) is an important preprocessing step for many natural language processing applications because grammatical rules are not functions of individual words, instead, they are functions of word categories.

Unlike supervised POS tagging systems, POS induction systems make use of unsupervised methods.

Type based methods suffer from POS ambiguity because one POS tag is assigned to each word type.

In particular, we extend the monolingual infinite tree model (Finkel et al., 2007) to a bilingual scenario: each hidden state ( POS tag ) of a source-side dependency tree emits a source word together with its aligned target word, either jointly (joint model), or independently (independent model).

Evaluations of J apanese-to-English translation on the NTCIR-9 data show that our induced Japanese POS tags for dependency trees improve the performance of a forest-to-string SMT system.

However, dependency parsing, which is a popular choice for Japanese, can incorporate only shallow syntactic information, i.e., POS tags , compared with the richer syntactic phrasal categories in constituency parsing.

Figure 1: Examples of Existing Japanese POS Tags and Dependency Structures

If we could discriminate POS tags for two cases, we might improve the performance of a Japanese-to-English SMT system.

In this paper, we combine easy-first dependency parsing and POS tagging algorithms with beam search and structured perceptron.

We use the standard split for dependency parsing and the split used by (Ratnaparkhi, 1996) for POS tagging .

For dependency parsing, POS tags of the training set are generated using 10-fold jackknifing.

For dependency parsing, we assume gold segmentation and POS tags for the input.

The proposed solution is general and can also be applied to other algorithms that exhibit spurious ambiguity, such as easy-first POS tagging (Ma et al., 2012) and transition-based dependency parsing with dynamic oracle (Goldberg and Nivre, 2012).

In this paper, we report experimental results on both easy-first dependency parsing and POS tagging (Ma et al., 2012).

We show that both easy-first POS tagging and dependency parsing can be improved significantly from beam search and global learning.

wp denotes the head word of p, tp denotes the POS tag of wp.

Rasooli and Faili (2012) and Bisk and Hockenmaier (2012) made some efforts to boost the verbocentricity of the inferred structures; however, both of the approaches require manual identification of the POS tags marking the verbs, which renders them useless when unsupervised POS tags are employed.

Our dependency model contained a submodel which directly prioritized subtrees that form reducible sequences of POS tags .

Reducibility scores of given POS tag sequences were estimated using a large corpus of Wikipedia articles.

The weakness of this approach was the fact that longer sequences of POS tags are very sparse and no reducibility scores could be estimated for them.

Hereinafter, Psfifzxch, dir) denotes the STOP-probability we want to estimate from a large corpus; ch is the head’s POS tag and dir is the direction in which the STOP probability is estimated.

For each POS tag 0;, in the given corpus, we first compute its left and right “raw” score Sst0p(ch, left) and Sst0p(ch, right) as the relative number of times a word with POS tag 0;, was in the first (or last) position in a reducible sequence found in the corpus.

Their main purpose is to sort the POS tags according to their “reducibility”.

But only sentence boundaries, POS tags and NER labels were kept as the annotation of the corpus.

IR can easily make use of this knowledge: for a when question, IR retrieves sentences with tokens labeled as DATE by NER, or POS tagged as CD.

Moreover, our approach extends easily beyond fixed answer types such as named entities: we are already using POS tags as a demonstration.

We let the trained QA system guide the query formulation when performing coupled retrieval with Indri (Strohman et al., 2005), given a corpus already annotated with POS tags and NER labels.

Since NER and POS tags are not lexicalized they accumulate many more counts (i.e.

NER Types First We found NER labels better indicators of expected answer types than POS tags .

These targeted morphological features are effective during LP because words that share them are much more likely to actually share POS tags .

Since the LP graph contains a node for each corpus token, and each node is labeled with a distribution over POS tags , the graph provides a corpus of sentences labeled with noisy tag distributions along with an expanded tag dictionary.

enized and labeled with POS tags by two linguistics graduate students, each of which was studying one of the languages.

The KIN and MLG data have 12 and 23 distinct POS tags , respectively.

The PTB uses 45 distinct POS tags .

Moreover, since large gains in accuracy can be achieved by spending a small amount of time just annotating word types with POS tags , we are led to conclude that time should be spent annotating types or tokens instead of developing an FST.

Haghighi and Klein (2006) develop a model in which a POS-tagger is learned from a list of POS tags and just three “prototype” word types for each tag, but their approach requires a vector space to compute the distributional similarity between prototypes and other word types in the corpus.

Our approach was superior to previous approaches across 12 multilingual cross-domain POS tagging datasets, with an average error reduction of 4% over a structured perceptron baseline.

POS tagging accuracy is known to be very sensitive to domain shifts.

(2011) report a POS tagging accuracy on social media data of 84% using a tagger that ac-chieves an accuracy of about 97% on newspaper data.

While POS taggers can often recover the part of speech of a previously unseen word from the context it occurs in, this is harder than for previously seen words.

This paper considers the POS tagging problem, i.e.

Several authors have noted how POS tagging performance is sensitive to cross-domain shifts (Blitzer et al., 2006; Daume III, 2007; Jiang and Zhai, 2007), and while most authors have assumed known target distributions and pool unlabeled target data in order to automatically correct cross-domain bias (Jiang and Zhai, 2007; Foster et al., 2010), methods such as feature bagging (Sutton et al., 2006), learning with random adversaries (Globerson and Roweis, 2006) and LOO-regularization (Dekel and Shamir, 2008) have been proposed to improve performance on unknown target distributions.

Section 4 presents experiments on POS tagging and discusses how to evaluate cross-domain performance.

- Part-of-speech (POS) tags and morphological features: POS tags indicate (or counter-indicate) the possible presence of a named entity at word level or at word sequence level.

Benajiba and Rosso (2007) improved their system by incorporating POS tags to improve NE boundary detection.

Benajiba and Rosso (2008) used CRF sequence labeling and incorporated many language specific features, namely POS tagging , base-phrase chunking, Arabic tokenization, and adjectives indicating nationality.

Using POS tagging generally improved recall at the expense of precision, leading to overall improvements in F-measure.

Table 2: Domain Adaptation performance in F-measure on Semantic Tagging on Movie Target domain and POS tagging on QBanszuestionBank.

In (Subramanya et al., 2010) an efficient iterative SSL method is described for syntactic tagging, using graph-based learning to smooth POS tag posteriors.

In (Subramanya et al., 2010), a new SSL method is described for adapting syntactic POS tagging of sentences in newswire articles along with search queries to a target domain of natural language (NL) questions.

The unlabeled POS tag posteriors are then smoothed using a graph-based learning algorithm.

Later, using Viterbi decoding, they select the l-best POS tag sequence, 33'?

2. surrounding: lslex (the previous word / 33:11), rslex (the next word/ fJJ-jill), lspos (lsleX’S POS tag), rspos (rsleX’S POS tag ), lsparent (lsleX’S parent), rsparent

3. nonlocal: lanchorslex (thE: pl‘EDVlOLlS anchor’s word) , ranchorslex (the next an-ChOf’S word), lanchorspos (lanchorslex’s POS tag), ranchorspos (ranchorslex’s POS tag ).

Of mosl_int_spos (mosl_int_sleX’S POS tag ), mosl_ext_spos (mosl_ext_spos’S PQS tag), mosr_int_slex (the actual word.

Performance of POS tagging is an important factor in our methods because they are based on wordfl’OS sequences.

Existing POS taggers might not perform well on nonnative English texts because they are normally developed to analyze native English texts.

In this language model, content words in n-grams are replaced with their corresponding POS tags .

Finally, words are replaced with their corresponding POS tags; for the following words, word tokens are used as their corresponding POS tags : coordinating conjunctions, determiners, prepositions, modals, predeterminers, possessives, pronouns, question adverbs.

At this point, the special POS tags BOS and EOS are added at the beginning and end of each sentence, respectively.

The part-of-speech tags in all datasets were replaced with the universal POS tags of Petrov et al.

This may have a negative effect on the performance of a monolingual model, since most part-of-speech tagsets are more fine-grained than the universal POS tags considered here.

Since the finer-grained POS tags often reflect more language-specific phenomena, however, they would only be useful for very closely related languages in the cross-lingual setting.

If Synt is enabled too, it also uses the POS tags of the argument’s parent, children and siblings.

Cross-lingual annotation projection (Yarowsky et al., 2001) approaches have been applied extensively to a variety of tasks, including POS tagging (Xi and Hwa, 2005; Das and Petrov, 2011), morphology segmentation (Snyder and Barzilay, 2008), verb classification (Merlo et al., 2002), mention detection (Zitouni and Florian, 2008), LFG parsing (Wroblewska and Frank, 2009), information extraction (Kim et al., 2010), SRL (Pado and Lapata, 2009; van der Plas et al., 2011; Annesi and Basili, 2010; Tonelli and Pi-anta, 2008), dependency parsing (Naseem et al., 2012; Ganchev et al., 2009; Smith and Eisner, 2009; Hwa et al., 2005) or temporal relation pre-

Moreover, all POS tag features from English are duplicated with coarse-grained POS tags provided by CoNLL-X.

Before parsing, POS tags were assigned to the training set by using 20-way jackknifing.

For the automatic generation of POS tags , we used the domain-specific model of Choi and Palmer (2012a)’s tagger, which gave 97.5% accuracy on the English evaluation set (0.2% higher than Collins (2002)’s tagger).

Bohnet and Nivre (2012) introduced a transition-based system that jointly performed POS tagging and dependency parsing.

3.1 and the POS tags come from a PCFG.

The standard RNN essentially ignores all POS tags and syntactic categories and each nonterminal node is associated with the same neural network (i.e., the weights across nodes are fully tied).

While this results in a powerful composition function that essentially depends on the words being combined, the number of model parameters explodes and the composition functions do not capture the syntactic commonalities between similar POS tags or syntactic categories.

4Since the dictionary is not explicitly annotated with PoS tags, we firstly took the intersection of the training corpus and the dictionary words, and assigned all the possible PoS tags to the words which appeared in the corpus.

Proper noun performance for the Stanford segmenter is not shown since it does not assign PoS tags .

Here, 111,- and wi_1 denote the current and previous word in question, and ti and til are level-j PoS tags assigned to them.

1The Japanese dictionary and the corpus we used have 6 levels of PoS tag hierarchy, while the Chinese ones have only one level, which is why some of the PoS features are not included in Chinese.

For both English and Chinese data, we used tenfold jackknifing (Collins, 2000) to automatically assign POS tags to the training data.

For English POS tagging, we adopted SVMTool, 3 and for Chinese POS tagging

we employed the Stanford POS tagger .

Word clusters are regarded as lexical intermediaries for dependency parsing (Koo et al., 2008) and POS tagging (Sun and Uszkoreit, 2012).

For this reason, we include POS tag 1, 2, 3, and 4-grams in the set of features we sort in the previous paragraph.

For each error 6,, we select POS tag n-grams from the top thousand features of the information gain sorted list to count toward the Ap+i and Api aggregation features.

This feature type may also help with Confusing Phrasing because the list of POS tag n-grams our annotator generated for its Ap+i contains useful features like DT NNS VBZ VBN (e.g., “these signals has been”), which captures noun-verb disagreement.

the Model 1 probabilities between pairs of words linked in the alignment a, features that inspect source and target POS tags and parses (if available) and features that inspect the alignments of adjacent words in the source and target sentence.

We conjoin the msd (minimum signed distance) with the POS tags to allow the model to capture the fact that the alignment error rate maybe higher for some POS tags than others (e.g., we have observed verbs have a higher error rate in Urdu-English alignments).

where 6 is a learned vector of weights and (I) is a vector of binary feature functions that inspect the words and POS tags of the source sentence at and around positions m and n. We use the features ((1)) described in Visweswariah et al.

As shown in Table 2, we classify the features used in WikiCiKE into three categories: format features, POS tag features and token features.

POS tag POS tag of current token features POS tags of previous 5 tokens

POS tags of

leftmost child label or POS tag rightmost child label or POS tag label or POS tag of the head child the number of child nodes

left-sibling label or POS tag

0 right-sibling label or POS tag

We trained CRFs for opinion entity identification using the following features: indicators for words, POS tags , and lexicon features (the subjectivity strength of the word in the Subjectivity Lexicon).

Words and POS tags: the words contained in the candidate and their POS tags .

For features, we use words, POS tags , phrase types, lexicon and semantic frames (see Section 3.2.1 for details) to capture the properties of the opinion expression, and also features that capture the context of the opinion expression:

When tagging with CTAGS, one can use any statistical POS tagging method such as HMMs, Maximum Entropy Classifiers, Bayesian Networks, CRFs, etc., followed by the CTAG to MSD recovery.

Manual+automatic Tmmmg a POS tagger rules for MSD recovery I I I I Tagging i i Input data Labeling with CTAGS —> MSD Recovery Output data

Also, our POS tagger detected cases where the annotation in the Gold Standard was erroneous.