Trained using annotated word structures, our parser also analyzes the internal structures of Chinese words .

Frequently-occurring character sequences that express certain meanings can be treated as words, while most Chinese words have syntactic structures.

Zhao (2009) studied character-level dependencies for Chinese word segmentation by formalizing segmentsion task in a dependency parsing framework.

They use it as a joint framework to perform Chinese word segmentation, POS tagging and syntax parsing.

They exploit a generative maximum entropy model for character-based constituent parsing, and find that POS information is very useful for Chinese word segmentation, but high-level syntactic information seems to have little effect on segmentation.

Unlike alphabetical languages, Chinese characters convey meanings, and the meaning of most Chinese words takes roots in their character.

Chinese words have internal structures (Xue, 2001; Ma et al., 2012).

Zhang and Clark (2010) found that the first character in a Chinese word is a useful indicator of the word’s POS.

While no segmented corpus of micro-blogs is available to train Chinese word segmentation model, existing Chinese word segmentation tools cannot perform equally well as in ordinary news texts.

In this paper we present an effective yet simple approach to Chinese word segmentation of micro-blog.

We use the benchmark datasets provided by the second International Chinese Word Segmentation Bakeoff2 as the labeled data.

The first two are both famous Chinese word segmentation tools: ICTCLAS3 and Stanford Chinese word segmenter4, which are widely used in NLP related to word segmentation.

Stanford Chinese word segmenter is a CRF-based segmentation tool and its segmentation standard is chosen as the PKU standard, which is the same to ours.

These new features of micro-blogs make the Chinese Word Segmentation (CWS) models trained on the source domain, such as news corpus, fail to perform equally well when transferred to texts from micro-blogs.

Chinese word segmentation problem might be treated as a character labeling problem which gives each character a label indicating its position in one word.

Recent studies show that character sequence labeling is an effective formulation of Chinese word segmentation (Low et al., 2005; Zhao et al., 2006a,b; Chen et al., 2006; Xue, 2003).

(1998) takes advantage of the huge amount of raw text to solve Chinese word segmentation problems.

Besides, Sun and Xu (2011) uses a sequence labeling framework, while unsupervised statistics are used as discrete features in their model, which prove to be effective in Chinese word segmentation.

With Chinese word segmentation as a case study, experiments show that the segmenter enhanced with the Chinese wikipedia achieves significant improvement on a series of testing sets from different domains, even with a single classifier and local features.

Experiments on Chinese word segmentation show that, the enhanced word segmenter achieves significant improvement on testing sets of different domains, although using a single classifier with only local features.

We use the Penn Chinese Treebank 5.0 (CTB) (Xue et al., 2005) as the existing annotated corpus for Chinese word segmentation.

Table 4: Comparison with state-of-the-art work in Chinese word segmentation.

Table 4 shows the comparison with other work in Chinese word segmentation.

Taking Chinese word segmentation for example, the state-of-the-art models (Xue and Shen, 2003; Ng and Low, 2004; Gao et al., 2005; Nakagawa and Uchimoto, 2007; Zhao and Kit, 2008; J iang et al., 2009; Zhang and Clark, 2010; Sun, 2011b; Li, 2011) are usually trained on human-annotated corpora such as the Penn Chinese Treebank (CTB) (Xue et al., 2005), and perform quite well on corresponding test sets.

In the rest of the paper, we first briefly introduce the problems of Chinese word segmentation and the character classification model in section

Li and Sun (2009) extracted character classification instances from raw text for Chinese word segmentation, resorting to the indication of punctuation marks between characters.

Sun and Xu (Sun and Xu, 2011) utilized the features derived from large-scaled unlabeled text to improve Chinese word segmentation.

Finally, all the sentences in the corpus are divided into words using a Chinese word segmentation tool (ICTCLAS)1 with all entities annotated in the corpus kept.

The second issue is that the Chinese word order in a sentence is rather agile for the open

(2012a) find out that sometimes two trigger mentions are within a Chinese word whose morphological structure is Coordination.

The relation between those event mentions whose triggers merge a Chinese word or share the subject and the object are Parallel.

We exploit this reliance as an opportunity: recognizing the relation between informal word recognition and Chinese word segmentation, we propose to model the two tasks jointly.

There is a close dependency between Chinese word segmentation (CWS) and informal word recognition (IWR).

This example illustrates the mutual dependency between Chinese word segmentation (henceforth, CWS) and informal word recognition (IWR) that should be solved jointly.

Given an input Chinese microblog post, our method simultaneously segments the sentences into words (the Chinese Word Segmentation, CWS, task), and marks the component words as informal or formal ones (the Informal Word Re-congition, IWR, task).

0 (*)IkaCk+1(i—4 < k < i+4)isnota Chinese word recorded in dictionaries: CPMI—N@k+i; CPMI—M@k+i; CDifl@k+i; PYPMI—N@k+i; PYPMI—M@k+i; PYD-ifi”@k+i

By analyzing the results, we found out that for both baseline and our model, a large part of missing alignment links involves stop words like English words “the”, “a”, “it” and Chinese words “de”.

As Chinese language lacks morphology, the single form and plural form of a noun in English often correspond to the same Chinese word , thus it is desirable that the two English words should have similar word embeddings.

As shown in example (a) of Figure 1, in word pair {“juda” =>“mammot ”}, the Chinese word “juda” is a common word, but

For example (b) in Figure l, for the word pair {“yibula” => “Yibula”}, both the Chinese word “yibula” and English word “Yibula” are rare name entities, but the words around them are very common, which are {“nongmin”, “shuo”} for Chinese side and {“farmer”, “said”} for the English side.

For example, many Chinese words can act as a verb, noun and adjective without any change, while their English counter parts are distinct words with quite different word embeddings due to their different syntactic roles.

We first predict *pro* and *PRO* with our annotation model for all Chinese sentences in the parallel training data, with *pro* and *PRO* inserted between the original Chinese words .

One of the other frequent ECs, *OP *, appears in the Chinese relative clauses, which usually have a Chinese word “De” aligned to the target side “that” or “which”.

Consequently, “that” is incorrectly aligned to the second to the last Chinese word “De”, due to their high co-occurrence frequency in the training data.

This paper introduces a graph-based semi-supervised joint model of Chinese word segmentation and part-of-speech tagging.

As far as we know, however, these methods have not yet been applied to resolve the problem of joint Chinese word segmentation (CWS) and POS tagging.

This study introduces a novel semi-supervised approach for joint Chinese word segmentation and POS tagging.