Conclusion and future work | We showed how adaptor grammars can implement a previously investigated model of unsupervised word segmentation, the unigram word segmentation model . |
Word segmentation with adaptor grammars | (2007a) presented an adaptor grammar that defines a unigram model of word segmentation and showed that it performs as well as the unigram DP word segmentation model presented by (Goldwater et al., 2006a). |
Word segmentation with adaptor grammars | The adaptor grammar that encodes a unigram word segmentation model shown in Figure 1. |
Word segmentation with adaptor grammars | (2007), a unigram word segmentation model tends to undersegment and misanalyse collocations as individual words. |
Experimental SetUp | The probabilistic formulation of this model is close to our monolingual segmentation model , but it uses a greedy search specifically designed for the segmentation task. |
Model | Our segmentation model is based on the notion that stable recurring string patterns within words are indicative of morphemes. |
Model | We note that these single-language morpheme distributions also serve as monolingual segmentation models , and similar models have been successfully applied to the task of word boundary detection (Goldwater et al., 2006). |