For the task of full morphological analysis, the lexicon must provide all possible morphological analyses for any given token.

In this paper, we investigate the characteristics of Hebrew unknowns for full morphological analysis , and propose a new method for handling such unavoidable lack of information.

In our evaluation, these learned distributions include the correct analysis for unknown words in 85% of the cases, contributing an error reduction of over 30% over a competitive baseline for the overall task of full morphological analysis in Hebrew.

model application is a set of possible full morphological analyses for the token — in exactly the same format as the morphological analyzer provides.

Habash and Rambow (2006) used the root+pattern+features representation of Arabic tokens for morphological analysis and generation of Arabic dialects, which have no lexicon.

They report high recall (95%—98%) but low precision (37%—63%) for token types and token instances, against gold-standard morphological analysis .

Unlike Nakagawa, our model does not use any segmented text, and, on the other hand, it aims to select full morphological analysis for each token,

The Input The set of analyses for a token is thus represented as a lattice in which every arc corresponds to a specific lexeme l, as shown in Figure l. A morphological analyzer M : W —> L is a function mapping sentences in Hebrew (W E W) to their corresponding lattices (M = L E L).

212,, and a morphological analyzer , we look for the most probable parse tree 7r s.t.

Since the lattice L for a given sentence W is determined by the morphological analyzer M we have

Morphological Analyzer Ideally, we would use an of-the-shelf morphological analyzer for mapping each input token to its possible analyses.

patible with the one of the Hebrew Treebank.8 For this reason, we use a data-driven morphological analyzer derived from the training data similar to (Cohen and Smith, 2007).

To control for the effect of the HSPELL-based pruning, we also experimented with a morphological analyzer that does not perform this pruning.

We represent all morphological analyses of a given utterance using a lattice structure.

Morphological analyzers for Hebrew that analyze a surface form in isolation have been proposed by Segal (2000), Yona and Wintner (2005), and recently by the knowledge center for processing Hebrew (Itai et al., 2006).

Morphological dis-ambiguators that consider a token in context (an utterance) and propose the most likely morphological analysis of an utterance (including segmentation) were presented by Bar-Haim et a1.

Tsarfaty (2006) used a morphological analyzer (Segal, 2000), a PoS tagger (Bar-Haim et al., 2005), and a general purpose parser (Schmid, 2000) in an integrated framework in which morphological and syntactic components interact to share information, leading to improved performance on the joint task.

Morphological analysis: returns the set of possible morphological analyses Aw = {a1, ..., a”} for w. A morphological analysis a is a vector of categorical values, where each dimension and its possible values are defined by L.

For the morphological analysis operation, we used the same set of morphological features described in (Minkov et al., 2007), that is, seven features for Russian (POS, Person, Number, Gender, Tense, Mood and Case) and 12 for Arabic (POS, Person, Number, Gender, Tense, Mood, Negation, Determiner, Conjunction, Preposition, Object and Possessive pronouns).

The same is true with the operation of morphological analysis .

Work in this area is motivated by two advantages offered by morphological analysis : (1) it provides linguistically motivated clustering of words and makes the data less sparse; (2) it captures morphological constraints applicable on the target side, such as agreement phenomena.

This Bible edition is augmented by gold standard morphological analysis (including segmentation) performed by biblical scholars.

We obtained gold standard segmentations of the Arabic translation with a handcrafted Arabic morphological analyzer which utilizes manually constructed word lists and compatibility rules and is further trained on a large corpus of hand-annotated Arabic data (Habash and Ram-bow, 2005).

The accuracy of this analyzer is reported to be 94% for full morphological analyses , and 98%-99% when part-of-speech tag accuracy is not included.

The underlying assumption of our work is that structural commonality across different languages is a powerful source of information for morphological analysis .

We used MeCab as a morphological analyzer and CaboCha14 (Kudo and Matsumoto, 2002) as the dependency parser to find the boundaries of the bunsetsu.

After preprocessing the first sentence of an article using a morphological analyzer , MeCab9, we extracted the last noun after the appearance of Japanese postpo-sition “Oi (wa)” (% “is”).

Asahara and Motsumoto (2003) proposed using characters instead of morphemes as the unit to alleviate the effect of segmentation errors in morphological analysis and we also used their character-based method.