Evaluating our algorithm on a subcorpus of the Rondane Treebank , we reduced the mean number of configurations of a sentence from several million to 4.5, in negligible runtime.

In this section, we evaluate the effectiveness and efficiency of our weakest readings algorithm on a treebank .

We compute RTGs for all sentences in the treebank and measure how many weakest readings remain after the intersection, and how much time this computation takes.

For our experiment, we use the Rondane treebank (version of January 2006), a “Redwoods style” (Oepen et al., 2002) treebank containing underspecified representations (USRs) in the MRS formalism (Copestake et al., 2005) for sentences from the tourism domain.

While applications should benefit from these very precise semantic representations, their usefulness is limited by the presence of semantic ambiguity: On the Rondane Treebank (Oepen et al., 2002), the ERG computes an average of several million semantic representations for each sentence, even when the syntactic analysis is fixed.

However, no such approach has been worked out in sufficient detail to support the disambiguation of treebank sentences.

It is of course completely infeasible to compute all readings and compare all pairs for entailment; but even the best known algorithm in the literature (Gabsdil and Striegnitz, 1999) is only an optimization of this basic strategy, and would take months to compute the weakest readings for the sentences in the Rondane Treebank .

always hnc subgraphs of D. In the worst case, GD can be exponentially bigger than D, but in practice it turns out that the grammar size remains manageable: even the RTG for the most ambiguous sentence in the Rondane Treebank , which has about 4.5 x l()12 scope readings, has only about 75 000 rules and can be computed in a few seconds.

We use the syntactic trees from the Penn Treebank to find the dominating nodes,.

But we think that MST algorithm has more potential in discourse dependency parsing, because our converted discourse dependency treebank contains only projective trees and somewhat suppresses the MST algorithm to exhibit its advantage of parsing non-projective trees.

In fact, we observe that some non-projective dependencies produced by the MST algorithm are even reasonable than what they are in the dependency treebank .

Section 2 formally defines discourse dependency structure and introduces how to build a discourse dependency treebank from the existing RST corpus.

2.2 Our Discourse Dependency Treebank

To automatically conduct discourse dependency parsing, constructing a discourse dependency treebank is fundamental.

Manually annotated corpora are valuable but scarce resources, yet for many annotation tasks such as treebanking and sequence labeling there exist multiple corpora with diflerent and incompatible annotation guidelines or standards.

Experiments show that adaptation from the much larger People’s Daily corpus to the smaller but more popular Penn Chinese Treebank results in significant improvements in both segmentation and tagging accuracies (with error reductions of 30.2% and 14%, respectively), which in turn helps improve Chinese parsing accuracy.

Especially, we will pay efforts to the annotation standard adaptation between different treebanks, for example, from HPSG LinGo Redwoods Treebank to PTB, or even from a dependency treebank to PTB, in order to obtain more powerful PTB annotation-style parsers.

Our adaptation experiments are conducted from People’s Daily (PD) to Penn Chinese Treebank 5.0 (CTB).

Much of statistical NLP research relies on some sort of manually annotated corpora to train their models, but these resources are extremely expensive to build, especially at a large scale, for example in treebanking (Marcus et al., 1993).

For example just for English treebanking there have been the Chomskian-style

Penn Treebank (Marcus et al., 1993) the HPSG LinGo Redwoods Treebank (Oepen et al., 2002), and a smaller dependency treebank (Buchholz and Marsi, 2006).

In addition, many efforts have been devoted to manual treebank adaptation, where they adapt PTB to other grammar formalisms, such as such as CCG and LFG (Hockenmaier and Steedman, 2008; Cahill and Mccarthy, 2007).

Table 2: Results for the Penn Treebank development set, sentences of length g 40, for different annotation schemes implemented on top of the X-bar grammar.

Table 3: Final Parseval results for the v = l, h = 0 parser on Section 23 of the Penn Treebank .

Finally, Table 3 shows our final evaluation on Section 23 of the Penn Treebank .

Table 1 shows the results of incrementally building up our feature set on the Penn Treebank development set.

Because constituents in the treebank can be quite long, we bin our length features into 8 buckets, of

Nai've context-free grammars, such as those embodied by standard treebank annotations, do not parse well because their symbols have too little context to constrain their syntactic behavior.

Our parser can be easily adapted to this task by replacing the X-bar grammar over treebank symbols with a grammar over the sentiment values to encode the output variables and then adding n-gram indicators to our feature set to capture the bulk of the lexical effects.

Historically, many annotation schemes for parsers have required language-specific engineering: for example, lexicalized parsers require a set of head rules and manually-annotated grammars require detailed analysis of the treebank itself (Klein and Manning, 2003).

Because the X-bar grammar is so minimal, this grammar does not parse very accurately, scoring just 73 F1 on the standard English Penn Treebank task.

Throughout this and the following section, we will draw on motivating examples from the English Penn Treebank , though similar examples could be equally argued for other languages.

There are a great number of spans in a typical treebank ; extracting features for every possible combination of span and rule is prohibitive.

We address the issue of using heterogeneous treebanks for parsing by breaking it down into two sub-problems, converting grammar formalisms of the treebanks to the same one, and parsing on these homogeneous treebanks .

Then we provide two strategies to refine conversion results, and adopt a corpus weighting technique for parsing on homogeneous treebanks .

Results on the Penn Treebank show that our conversion method achieves 42% error reduction over the previous best result.

The last few decades have seen the emergence of multiple treebanks annotated with different grammar formalisms, motivated by the diversity of languages and linguistic theories, which is crucial to the success of statistical parsing (Abeille et al., 2000; Brants et al., 1999; Bohmova et al., 2003; Han et al., 2002; Kurohashi and Nagao, 1998; Marcus et al., 1993; Moreno et al., 2003; Xue et al., 2005).

Availability of multiple treebanks creates a scenario where we have a treebank annotated with one grammar formalism, and another treebank annotated with another grammar formalism that we are interested in.

a source treebank, and the second a target treebank .

Note that all grammar rules in ERG are either unary or binary, giving us relatively deep trees when compared with annotations such as Penn Treebank .

For these rules, we refer to the conversion of the Penn Treebank into dependency structures used in the CoNLL 2008 Shared Task, and mark the heads of these rules in a way that will arrive at a compatible dependency backbone.

2More recent study shows that with carefully designed retokenization and preprocessing rules, over 80% sentential coverage can be achieved on the WSJ sections of the Penn Treebank data using the same version of ERG.

To evaluate the performance of our different dependency parsing models, we tested our approaches on several dependency treebanks for English in a similar spirit to the CoNLL 2006-2008 Shared Tasks.

Most of them are converted automatically from existing treebanks in various forms.

The larger part is converted from the Penn Treebank Wall Street Journal Sections #2—#21, and is used for training statistical dependency parsing models; the smaller part, which covers sentences from Section #23, is used for testing.

In the meantime, successful continuation of CoNLL Shared Tasks since 2006 (Buchholz and Marsi, 2006; Nivre et al., 2007a; Surdeanu et al., 2008) have witnessed how easy it has become to train a statistical syntactic dependency parser provided that there is annotated treebank .

the Wall Street Journal (WSJ) sections of the Penn Treebank (Marcus et al., 1993) as training set, tests on BROWN Sections typically result in a 6-8% drop in labeled attachment scores, although the average sentence length is much shorter in BROWN than that in WSJ.

This paper proposes an approach to enhance dependency parsing in a language by using a translated treebank from another language.

A simple statistical machine translation method, word-by-word decoding, where not a parallel corpus but a bilingual lexicon is necessary, is adopted for the treebank translation.

The proposed method is evaluated in English and Chinese treebanks .

But, this is not the case as we observe all treebanks in different languages as a whole.

For example, of ten treebanks for CoNLL-2007 shared task, none includes more than 500K

1It is a tradition to call an annotated syntactic corpus as treebank in parsing community.

We study the effect of semantic classes in three dependency parsers, using two types of constituency-to-dependency conversions of the English Penn Treebank .

In addition, we explore parser combinations, showing that the semantically enhanced parsers yield a small significant gain only on the more semantically oriented LTH treebank conversion.

3.1 Treebank conversions

PermZMalt1 performs a simple and direct conversion from the constituency-based PTB to a dependency treebank .

supervised approach that makes use of cluster features induced from unlabeled data, providing significant performance improvements for supervised dependency parsers on the Penn Treebank for English and the Prague Dependency Treebank for Czech.

Most experiments for English were evaluated on the Penn2Malt conversion of the constituency-based Penn Treebank .

tion 3 describes the treebank conversions, parsers and semantic features.

The results showed a signi-cant improvement, giving the first results over both WordNet and the Penn Treebank (PTB) to show that semantics helps parsing.

They demonstrated its effectiveness in dependency parsing experiments on the PTB and the Prague Dependency Treebank .

Looking at table 2, we can say that the differences in baseline parser performance are accentuated when using the LTH treebank conversion, as ZPar clearly outperforms the other two parsers by more than 4 absolute points.

We can also conclude that automatically acquired clusters are specially effective with the MST parser in both treebank conversions, which suggests that the type of semantic information has a direct relation to the parsing algorithm.

Unlike the context-free grammars extracted from the Penn treebank , these allow for the categorial semantics that accompanies any categorial parse and for a more elegant analysis of linguistic structures such as extraction and coordination.

in Petrov’s experiments on the Penn treebank , the syntactic category NP was refined to the more fine-grained N P1 and N P2 roughly corresponding to N Ps in subject and object positions.

In the supertagging literature, POS tagging and supertagging are distinguished — POS tags are the traditional Penn treebank tags (e. g. NN, VBZ and DT) and supertags are CCG categories.

The Petrov parser (Petrov and Klein, 2007) uses latent variables to refine the grammar extracted from a corpus to improve accuracy, originally used to improve parsing results on the Penn treebank (PTB).

These results should not be interpreted as proof that grammars extracted from the Penn treebank and from CCGbank are equivalent.

CCGbank (Hockenmaier and Steedman, 2007) is a corpus of CCG derivations that was semiautomatically converted from the Wall Street J our-nal section of the Penn treebank .

In addition, we present an efficient method for determining whether an arbitrary tree is 2-planar and show that 99% or more of the trees in existing treebanks are 2-planar.

Several constraints on non-projective dependency structures have been proposed recently that seek a good balance between parsing efficiency and coverage of non-projective phenomena present in natural language treebanks .

For example, Kuhlmann and Nivre (2006) and Havelka (2007) have shown that the vast majority of structures present in existing treebanks are well-nested and have a small gap degree (Bodirsky et al., 2005), leading to an interest in parsers for these kinds of structures (Gomez-Rodriguez et al., 2009).

No similar analysis has been performed for m-planar structures, although Yli-Jyr'a (2003) provides evidence that all except two structures in the Danish dependency treebank are at most 3-planar.

Although these proposals seem to have a very good fit with linguistic data, in the sense that they often cover 99% or more of the structures found in existing treebanks , the development of efficient parsing algorithms for these classes has met with more limited success.

First, we present a procedure for determining the minimal number m such that a dependency tree is m-planar and use it to show that the overwhelming majority of sentences in dependency treebanks have a tree that is at most 2-planar.

According to the results by Kuhlmann and Nivre (2006), most non-projective structures in dependency treebanks are also non-planar, so being able to parse planar structures will only give us a modest improvement in coverage with respect to a projective parser.

Once released, treebanks tend to remain unchanged despite any shortcomings in their depth of linguistic analysis or coverage of specific phenomena.

In this paper we show how to improve the quality of a treebank , by integrating resources and implementing improved analyses for specific constructions.

Statistical parsers induce their grammars from corpora, and the corpora for linguistically motivated formalisms currently do not contain high quality predicate-argument annotation, because they were derived from the Penn Treebank (PTB Marcus et al., 1993).

What we suggest in this paper is that a treebank’s grammar need not last its lifetime.

The structure of such compound noun phrases is left underspecified in the Penn Treebank (PTB), because the annotation procedure involved stitching together partial parses produced by the Fid-ditch parser (Hindle, 1983), which produced flat brackets for these constructions.

When Hockenmaier and Steedman (2002) went to acquire a CCG treebank from the PTB, this posed a problem.

The syntactic analysis of punctuation is notoriously difficult, and punctuation is not always treated consistently in the Penn Treebank (Bies et al., 1995).

Treebanking is a difficult engineering task: coverage, cost, consistency and granularity are all competing concerns that must be balanced against each other when the annotation scheme is developed.

The difficulty of the task means that we ought to view treebanking as an ongoing process akin to grammar development, such as the many years of work on the ERG (Flickinger, 2000).

This paper demonstrates how a treebank can be rebanked to incorporate novel analyses and infor-

In addition, when integrated into a 2nd-ordered MST parser, the projected parser brings significant improvement to the baseline, especially for the baseline trained on smaller treebanks .

In this section, we first validate the word-pair classification model by experimenting on human-annotated treebanks .

We experiment on two popular treebanks, the Wall Street Journal (WSJ) portion of the Penn English Treebank (Marcus et al., 1993), and the Penn Chinese Treebank (CTB) 5.0 (Xue et al., 2005).

The constituent trees in the two treebanks are transformed to dependency trees according to the head-finding rules of Yamada and Matsumoto (2003).

Since it is costly and difficult to build human-annotated treebanks , a lot of works have also been devoted to the utilization of unannotated text.

For the 2nd-order MST parser trained on Penn Chinese Treebank (CTB) 5.0, the classifier give an precision increment of 0.5 points.

On the training method, however, our model obviously differs from other graph-based models, that we only need a set of word-pair dependency instances rather than a regular dependency treebank .

We use a sentiment treebank to show that these existing heuristics are poor estimators of sentiment.

Data As described earlier, the Stanford Sentiment Treebank (Socher et al., 2013) has manually annotated, real-valued sentiment values for all phrases in parse trees.

We search these negators in the Stanford Sentiment Treebank and normalize the same negators to a single form; e.g., “is n’t”, “isn’t”, and “is not” are all normalized to “is_not”.

Each occurrence of a negator and the phrase it is directly composed with in the treebank , i.e., (7,071,217), is considered a data point in our study.

Table 1: Mean absolute errors (MAE) of fitting different models to Stanford Sentiment Treebank .

The figure includes five most frequently used negators found in the sentiment treebank .

Below, we take a closer look at the fitting errors made at different depths of the sentiment treebank .

Figure 1: Effect of a list of common negators in modifying sentiment values in Stanford Sentiment Treebank .

Each dot in the figure corresponds to a text span being modified by (composed with) a negator in the treebank .

The recently available Stanford Sentiment Treebank (Socher et al., 2013) renders manually annotated, real-valued sentiment scores for all phrases in parse trees.

We are interested in parsing constituency-based grammars such as HPSG and CCG using a small amount of data specific for the target formalism, and a large quantity of coarse CFG annotations from the Penn Treebank .

While all of the target formalisms share a similar basic syntactic structure with Penn Treebank CFG, they also encode additional constraints and semantic features.

The standard solution to this bottleneck has relied on manually crafted transformation rules that map readily available syntactic annotations (e.g, the Penn Treebank ) to the desired formalism.

In addition, designing these rules frequently requires external resources such as Wordnet, and even involves correction of the existing treebank .

A natural candidate for such coarse annotations is context-free grammar (CFG) from the Penn Treebank , while the target formalism can be any constituency-based grammars, such as Combinatory Categorial Grammar (CCG) (Steedman, 2001), Lexical Functional Grammar (LFG) (Bresnan, 1982) or Head-Driven Phrase Structure Grammar (HPSG) (Pollard and Sag, 1994).

For instance, mappings may specify how to convert traces and functional tags in Penn Treebank to the f-structure in LFG (Cahill, 2004).

For monolingual treebank data we relied on the CoNLL-X and CoNLL-2007 shared tasks on dependency parsing (Buchholz and Marsi, 2006; Nivre et al., 2007).

9We extracted only the words and their POS tags from the treebanks .

(2011) provide a mapping A from the fine-grained language specific POS tags in the foreign treebank to the universal POS tags.

7We used a tagger based on a trigram Markov model (Brants, 2000) trained on the Wall Street Journal portion of the Penn Treebank (Marcus et a1., 1993), for its fast speed and reasonable accuracy (96.7% on sections 22-24 of the treebank , but presumably much lower on the (out-of-domain) parallel cor-

Because there might be some controversy about the exact definitions of such universals, this set of coarse-grained POS categories is defined operationally, by collapsing language (or treebank ) specific distinctions to a set of categories that exists across all languages.

We devise a gold-standard sense- and parse tree-annotated dataset based on the intersection of the Penn Treebank and SemCor, and experiment with different approaches to both semantic representation and disambiguation.

Traditionally, the two parsers have been trained and evaluated over the WSJ portion of the Penn Treebank (PTB: Marcus et al.

We diverge from this norm in focusing exclusively on a sense-annotated subset of the Brown Corpus portion of the Penn Treebank , in order to investigate the upper bound performance of the models given gold-standard sense information.

most closely related research is that of Bikel (2000), who merged the Brown portion of the Penn Treebank with SemCor (similarly to our approach in Section 4.1), and used this as the basis for evaluation of a generative bilexical model for joint WSD and parsing.

The only publicly-available resource with these two characteristics at the time of this work was the subset of the Brown Corpus that is included in both SemCor (Landes et al., 1998) and the Penn Treebank (PTB).2 This provided the basis of our dataset.

2OntoNotes (Hovy et al., 2006) includes large-scale treebank and (selective) sense data, which we plan to use for future experiments when it becomes fully available.

Our approach to exploring the impact of lexical semantics on parsing performance is to take two state-of-the-art statistical treebank parsers and pre-process the inputs variously.

This simple method allows us to incorporate semantic information into the parser without having to reimplement a full statistical parser, and also allows for maximum comparability with existing results in the treebank parsing community.

We provide the first definitive results that word sense information can enhance Penn Treebank parser performance, building on earlier results of Bikel (2000) and Xiong et al.

This paper discusses the construction of a parallel treebank currently involving ten languages from six language families.

The treebank is based on deep LFG (Lexical-Functional Grammar) grammars that were developed within the framework of the ParGram (Parallel Grammar) effort.

This output forms the basis of a parallel treebank covering a diverse set of phenomena.

This paper discusses the construction of a parallel treebank currently involving ten languages that represent several different language families, including non-Indo-European.

The treebank is based on the output of individual deep LFG (Lexical-Functional Grammar) grammars that were developed independently at different sites but within the overall framework of ParGram (the Parallel Grammar project) (Butt et al., 1999a; Butt et al., 2002).

This output forms the basis of the ParGramBank parallel treebank discussed here.

We introduce a novel taxonomy of such approaches and apply it to treebanks across a typologically diverse range of 26 languages.

One of the reasons is the increased availability of dependency treebanks, be they results of genuine dependency annotation projects or converted automatically from previously existing phrase-structure treebanks .

In both cases, a number of decisions have to be made during the construction or conversion of a dependency treebank .

The dominating solution in treebank design is to introduce artificial rules for the encoding of coordination structures within dependency trees using the same means that express dependencies, i.e., by using edges and by labeling of nodes or edges.

0 PS = Prague Dependency Treebank (PDT) style: all conjuncts are attached under the coordinating conjunction (along with shared modifiers, which are distinguished by a special attribute) (Hajic et al., 2006),

Moreover, particular treebanks vary in their contents even more than in their format, i.e.

each treebank has its own way of representing prepositions or different granularity of syntactic labels.

Our analysis of variations in representing coordination structures is based on observations from a set of dependency treebanks for 26 languages.7

The authors are very grateful to Spence Green for his useful help on the treebank , and to Jennifer Thewis-sen for her careful proofreading.

The grammar was trained with a reference treebank where MWEs were annotated with a specific nonterminal node.

The experiments were carried out on the French Treebank (Abeille et al., 2003) where MWEs are annotated.

In our collocation resource, each candidate collocation of the French treebank is associated with its internal syntactic structure and its association score (log-likelihood).

(2011) confirmed these bad results on the French Treebank .

They show a general tagging accuracy of 94% on the French Treebank .

To do so, the MWEs in the training treebank were annotated with specific nonterminal nodes.

The French Treebank is composed of 435,860 lexical units (34,178 types).

In order to compare compounds in these lexical resources with the ones in the French Treebank , we applied on the development corpus the dictionaries and the lexicon extracted from the training corpus.

The authors provided us with a list of 17,315 candidate nominal collocations occurring in the French treebank with their log-likelihood and their internal flat structure.

The vector 6 is estimated during the training stage from a reference treebank and the baseline parser ouputs.

We present a new collection of treebanks with homogeneous syntactic dependency annotation for six languages: German, English, Swedish, Spanish, French and Korean.

This ‘universal’ treebank is made freely available in order to facilitate research on multilingual dependency parsing.1

Research in dependency parsing — computational methods to predict such representations — has increased dramatically, due in large part to the availability of dependency treebanks in a number of languages.

While these data sets are standardized in terms of their formal representation, they are still heterogeneous treebanks .

That is to say, despite them all being dependency treebanks , which annotate each sentence with a dependency tree, they subscribe to different annotation schemes.

(2004) for multilingual syntactic treebank construction.

The second, used only for English and Swedish, is to automatically convert existing treebanks , as in Zeman et al.

For English, we used the Stanford parser (v1.6.8) (Klein and Manning, 2003) to convert the Wall Street J our-nal section of the Penn Treebank (Marcus et al., 1993) to basic dependency trees, including punctuation and with the copula verb as head in copula constructions.

By incorporating this knowledge into Dependency Model with Valence, we managed to considerably outperform the state-of-the-art results in terms of average attachment score over 20 treebanks from CoNLL 2006 and 2007 shared tasks.

The first type are CoNLL treebanks from the year 2006 (Buchholz and Marsi, 2006) and 2007 (Nivre et al., 2007), which we use for inference and for evaluation.

The Wikipedia texts were automatically tokenized and segmented to sentences so that their tokenization was similar to the one in the CoNLL evaluation treebanks .

To evaluate the quality of our estimations, we compare them with P301), the stop probabilities computed directly on the evaluation treebanks .

This is still far below the supervised approaches, but their indisputable advantage is the fact that no annotated treebanks are needed and the induced structures are not burdened by any linguistic conventions.

supervised parsers always only simulate the treebanks they were trained on, whereas unsupervised parsers have an ability to be fitted to different particular applications.

Finally, we obtain the probability of the whole generated treebank as a product over the trees:

no matter how the trees are ordered in the treebank , the Ptreebank is always the same.

stop words in the treebank should be 2/3.

We present a simple and effective framework for exploiting multiple monolingual treebanks with different annotation guidelines for parsing.

Several types of transformation patterns (TP) are designed to capture the systematic annotation inconsistencies among different treebanks .

Our approach can significantly advance the state—of—the—art parsing accuracy on two widely used target treebanks (Penn Chinese Treebank 5.1 and 6.0) using the Chinese Dependency Treebank as the source treebank .

However, the heavy cost of treebanking typically limits one single treebank in both scale and genre.

At present, learning from one single treebank seems inadequate for further boosting parsing accuracy.1

Treebanks # of Words Grammar CTB5 0.51 million Phrase structure CTB6 0.78 million Phrase structure

Therefore, we can hardly obtain a treebank with complete trees through direct projection.

So we extract projected discrete dependency arc instances instead of treebank as training set for the projected grammar induction model.

Then we incorporate projection model into our iterative unsupervised framework, and jointly optimize unsupervised and projection objectives with evolving treebank and constant projection information respectively.

A randomly-initialized monolingual treebank evolves in a self-training iterative procedure, and the grammar parameters are tuned to simultaneously maximize both the monolingual likelihood and bilingually-projected likelihood of the evolving treebank .

And the framework of our unsupervised model builds a random treebank on the monolingual corpus firstly for initialization and trains a discriminative parsing model on it.

Then we use the parser to build an evolved treebank with the l-best result for the next iteration run.

In this way, the parser and treebank evolve in an iterative way until convergence.

In Table l, we show the first four samples of length between 15 and 20 generated from our model and a 5- gram model trained on the Penn Treebank .

For training data, we constructed a large treebank by concatenating the WSJ and Brown portions of the Penn Treebank , the 50K BLLIP training sentences from Post (2011), and the AFP and APW portions of English Gigaword version 3 (Graff, 2003), totaling about 1.3 billion tokens.

We used the human-annotated parses for the sentences in the Penn Treebank , but parsed the Gigaword and BLLIP sentences with the Berkeley Parser.

Figure 2: A sample parse from the Penn Treebank after the tree transformations described in Section 3.

number of transformations of Treebank constituency parses that allow us to capture such dependencies.

Although the Penn Treebank annotates temporal N Ps, most off-the-shelf parsers do not retain these tags, and we do not assume their presence.

There is one additional hurdle in the estimation of our model: while there exist corpora with human-annotated constituency parses like the Penn Treebank (Marcus et al., 1993), these corpora are quite small — on the order of millions of tokens — and we cannot gather nearly as many counts as we can for 77.-grams, for which billions or even trillions (Brants et al., 2007) of tokens are available on the Web.

We propose dealing with the induced word boundaries as soft constraints to bias the continuous learning of a supervised CRFs model, trained by the treebank data (labeled), on the bilingual data (unlabeled).

The monolingual segmented data, trainTB, is extracted from the Penn Chinese Treebank (CTB-7) (Xue et al., 2005), containing 51,447 sentences.

0 Supervised Monolingual Segmenter (SMS): this model is trained by CRFs on treebank training data (trainTB).

The practice in state-of-the-art MT systems is that Chinese sentences are tokenized by a monolingual supervised word segmentation model trained on the hand-annotated treebank data, e.g., Chinese treebank

But one outstanding problem is that these models may leave out some crucial segmentation features for SMT, since the output words conform to the treebank segmentation standard designed for monolingually linguistic intuition, rather than specific to the SMT task.

Crucially, the GP expression with the bilingual knowledge is then used as side information to regularize a CRFs (conditional random fields) model’s learning over treebank and bitext data, based on the posterior regularization (PR) framework (Ganchev et al., 2010).

The input data requires two types of training resources, segmented Chinese sentences from treebank ’ch and parallel unsegmented sentences of Chinese and foreign language “Di and D5.

Algorithm 1 CWS model induction with bilingual constraints Require: Segmented Chinese sentences from treebank ’ch; Parallel sentences of Chinese and foreign

As in conventional GP examples (Das and Smith, 2012), a similarity graph Q = (V, E) is constructed over N types extracted from Chinese training data, including treebank ’ch and bitexts “Di.

Although we restricted ourselves to parsers trainable with Penn Treebank-style treebanks , our methodology can be applied to any English parsers.

It should be noted, however, that this conversion cannot work perfectly with automatic parsing, because the conversion program relies on function tags and empty categories of the original Penn Treebank .

Next, we run parsers retrained with GENIA11 (8127 sentences), which is a Penn Treebank-style treebank of biomedical paper abstracts.

10Some of the parser packages include parsing models trained with extended data, but we used the models trained with WSJ section 2-21 of the Penn Treebank .

This assumes the existence of a gold-standard test corpus, such as the Penn Treebank (Marcus et al., 1994).

Most state-of-the-art parsers for English were trained with the Wall Street Journal (WSJ) portion of the Penn Treebank , and high accuracy has been reported for WSJ text; however, these parsers rely on lexical information to attain high accuracy, and it has been criticized that these parsers may overfit to WSJ text (Gildea, 2001;

When training data in the target domain is available, as is the case with the GENIA Treebank (Kim et al., 2003) for biomedical papers, a parser can be retrained to adapt to the target domain, and larger accuracy improvements are expected, if the training method is sufficiently general.

In general, our evaluation methodology can be applied to English parsers based on any framework; however, in this paper, we chose parsers that were originally developed and trained with the Penn Treebank or its variants, since such parsers can be retrained with GENIA, thus allowing for us to investigate the effect of domain adaptation.

Owing largely to the Penn Treebank , the mainstream of data-driven parsing research has been dedicated to the phrase structure parsing.

ENJU The HPSG parser that consists of an HPSG grammar extracted from the Penn Treebank, and a maximum entropy model trained with an HPSG treebank derived from the Penn Treebank.7

In this paper, we report our preliminary efforts in building an English-Turkish parallel treebank corpus for statistical machine translation.

In the corpus, we manually generated parallel trees for about 5,000 sentences from Penn Treebank .

In recent years, many efforts have been made to annotate parallel corpora with syntactic structure to build parallel treebanks .

A parallel treebank is a parallel corpus where the sentences in each language are syntactically (if necessary morphologically) annotated, and the sentences and words are aligned.

In the parallel treebanks , the syntactic annotation usually follows constituent and/or dependency structure.

Since exact inference is intractable with nonlocal features, we present an approximate algorithm inspired by forest rescoring that makes discriminative training practical over the whole Treebank .

Our final result, an F—score of 91.7, outperforms both 50-best and 100-best reranking baselines, and is better than any previously reported systems trained on the Treebank .

With efficient approximate decoding, perceptron training on the whole Treebank becomes practical, which can be done in about a day even with a Python implementation.

Our final result outperforms both 50-best and 100-best reranking baselines, and is better than any previously reported systems trained on the Treebank .

We compare the performance of our forest reranker against n-best reranking on the Penn English Treebank (Marcus et al., 1993).

We use the standard split of the Treebank : sections 02-21 as the training data (39832 sentences), section 22 as the development set (1700 sentences), and section 23 as the test set (2416 sentences).

Our final result (91.7) is better than any previously reported system trained on the Treebank , although

Although previous work on discriminative parsing has mainly focused on short sentences (3 15 words) (Taskar et al., 2004; Turian and Melamed, 2007), our work scales to the whole Treebank , where

This result is also better than any previously reported systems trained on the Treebank .

However, in this work, we use forests from a Treebank parser (Charniak, 2000) whose grammar is often flat in many productions.

By examining the arguments that the verbal category combines with in the treebank , we can identify the corresponding semantic role for each argument that is marked on the verbal category.

| P | R I F G&H (treebank) 67.5% 60.0% 63.5% Brutus ( treebank ) 88.18% 85.00% 86.56%

Many of the errors made by the Brutus system can be traced directly to erroneous parses, either in the automatic or treebank parse.

However, because in 1956 is erroneously modifying the verb using rather than the verb stopped in the treebank parse, the system trusts the syntactic analysis and places Argl of stopped on using asbestos in 1956.

The same features are extracted for both treebank and automatic parses.

l P l R | F P. et al (treebank) 86.22% 87.40% 86.81% Brutus ( treebank ) 88.29% 86.39% 87.33%

Headword ( treebank ) 88.94% 86.98% 87.95%

Boundary ( treebank ) 88.29% 86.39% 87.33%

Removing them has a strong effect on accuracy when labeling treebank parses, as shown in our feature ablation results in table 4.

For gold-standard parses, we remove functional tag and trace information from the Penn Treebank parses before we extract features over them, so as to simulate the conditions of an automatic parse.

The Penn Treebank features are as follows:

Using a treebank grammar, a data-driven lexicon, and a linguistically motivated unknown-tokens handling technique our model outperforms previous pipelined, integrated or factorized systems for Hebrew morphological and syntactic processing, yielding an error reduction of 12% over the best published results so far.

The overall performance of our joint framework demonstrates that a probability distribution obtained over mere syntactic contexts using a Treebank grammar and a data-driven lexicon outperforms upper bounds proposed by previous joint disambiguation systems and achieves segmentation and parsing results on a par with state-of-the-art standalone applications results.

Data We use the Hebrew Treebank , (Sima’an et a1., 2001), provided by the knowledge center for processing Hebrew, in which sentences from the daily newspaper “Ha’aretz” are morphologically segmented and syntactically annotated.

The treebank has two versions, v1.0 and v2.0, containing 5001 and 6501 sentences respectively.

6Unfortunatley running our setup on the v2.0 data set is currently not possible due to missing tokens-morphemes alignment in the v2.0 treebank .

Morphological segmentation decisions in our model are delegated to a lexeme-based PCFG and we show that using a simple treebank grammar, a data-driven lexicon, and a linguistically motivated unknown-tokens handling our model outperforms (Tsarfaty, 2006) and (Cohen and Smith, 2007) on the joint task and achieves state-of-the-art results on a par with current respective standalone models.2

The development of the very first Hebrew Treebank (Sima’an et al., 2001) called for the exploration of general statistical parsing methods, but the application was at first limited.

Tsarfaty (2006) was the first to demonstrate that fully automatic Hebrew parsing is feasible using the newly available 5000 sentences treebank .

We outline the problem of ad hoc rules in treebanks , rules used for specific constructions in one data set and unlikely to be used again.

Based on a simple notion of rule equivalence and on the idea of finding rules unlike any others, we develop two methods for detecting ad hoc rules in flat treebanks and show they are successful in detecting such rules.

Treebank (Marcus et al., 1993), six of which are errors.

For example, in (2), the daughters list RB TO JJ NNS is a daughters list with no correlates in the treebank ; it is erroneous because close to wholesale needs another layer of structure, namely adjective phrase (ADJP) (Bies et al., 1995, p. 179).

When extracting rules from constituency-based treebanks employing flat structures, grammars often limit the set of rules (e.g., Charniak, 1996), due to the large number of rules (Krotov et al., 1998) and “leaky” rules that can lead to mis-analysis (Foth and Menzel, 2006).

Thus, we need to carefully consider the applicability of rules in a treebank to new text.

For example, when ungrammatical or nonstandard text is used, treebanks employ rules to cover it, but do not usually indicate ungrammaticality in the annotation.

Linguistically annotated treebanks play an essential part in the modern computational linguistics.

The more complex the treebanks become, the more sophisticated tools are required for using them, namely for searching in the data.

We study linguistic phenomena annotated in the Prague Dependency Treebank 2.0 and create a list of requirements these phenomena set on a search tool, especially on its query language.

Searching in a linguistically annotated treebank is a principal task in the modern computational linguistics.

A search tool helps extract useful information from the treebank , in order to study the language, the annotation system or even to search for errors in the annotation.

The more complex the treebank is, the more sophisticated the search tool and its query language needs to be.

However, most evaluations of syntactic treebanks use simple accuracy measures such as bracket F1 scores for constituent trees (NEGRA, Brants, 2000; TIGER, Brants and Hansen, 2002; Cat3LB, Civit et al., 2003; The Arabic Treebank, Maamouri et al., 2008) or labelled or unlabelled attachment scores for dependency syntax (PDT, Hajic, 2004; PCEDT Mikulova and 8tepanek, 2010; Norwegian Dependency Treebank , Skjaerholt, 2013).

In grammar-driven treebanking (or parsebank-ing), the problems encountered are slightly different.

In HPSG and LPG treebanking annotators do not annotate structure directly.

Three of the data sets are dependency treebanks

7We contacted a number of treebank projects, among them the Penn Treebank and the Prague Dependency Treebank , but not all of them had data available.

(NDT, CDT, PCEDT) and one phrase structure treebank (SSD), and of the dependency treebanks the PCEDT contains semantic dependencies, while the other two have traditional syntactic dependencies.

An already annotated corpus, in our case 100 randomly selected sentences from the Norwegian Dependency Treebank (Solberg et al., 2014), are taken as correct and then permuted to produce “annotations” of different quality.

Our training data includes newswire from the English translation treebank (LDC2007T02) and the English-Arabic Treebank (LDC2006T10), which are respectively translations of sections of the Chinese treebank (CTB) and Arabic treebank (ATB).

We also trained the parser on the broadcast-news treebank available in the OntoNotes corpus (LDC2008T04), and added sections 02-21 of the WSJ Penn treebank .

Our other test set is the standard Section 23 of the Penn treebank .

To extract dependencies from treebanks , we used the LTH Penn Converter (ht tp : / / nlp .

We constrain the converter not to use functional tags found in the treebanks , in order to make it possible to use automatically parsed texts (i.e., perform self-training) in future work.

Chinese words were automatically segmented with a conditional random field (CRF) classifier (Chang et al., 2008) that conforms to the Chinese Treebank (CTB) standard.

We perform experiments on three Data sets — Version 1.0 and version 2.0 of Google Universal Dependency Treebanks and Treebanks from CoNLL shared-tasks, across ten languages.

Our experiments rely on two kinds of data sets: (i) Monolingual Treebanks with consistent annotation schema — English treebank is used to train the English parsing model, and the Treebanks for target languages are used to evaluate the parsing performance of our approach.

The monolingual treebanks in our experiments are from the Google Universal Dependency Treebanks (McDonald et al., 2013), for the reason that the treebanks of different languages in Google Universal Dependency Treebanks have consistent syntactic representations.

The treebanks from CoNLL shared-tasks on dependency parsing (Buchholz and Marsi, 2006; Nivre et al., 2007) appear to be another reasonable choice.

Several supervised dependency parsing algorithms (Nivre and Scholz, 2004; McDonald et al., 2005a; McDonald et al., 2005b; McDonald and Pereira, 2006; Carreras, 2007; K00 and Collins, 2010; Ma and Zhao, 2012; Zhang et al., 2013) have been proposed and achieved high parsing accuracies on several treebanks, due in large part to the availability of dependency treebanks in a number of languages (McDonald et al., 2013).

However, the manually annotated treebanks that these parsers rely on are highly expensive to create, in particular when we want to build treebanks for resource-poor languages.

However, most bilingual text parsing approaches require bilingual treebanks — treebanks that have manually annotated tree structures on both sides of source and target languages (Smith and Smith, 2004; Burkett and Klein, 2008), or have tree structures on the source side and translated sentences in the target languages (Huang et

Table 1: Data statistics of two versions of Google Universal Treebanks for the target languages.

Articles in the Penn TreeBank were identified as being reviews, summaries, letters to the editor, news reportage, corrections, wit and short verse, or quarterly profit reports.

All but the latter three were then characterised in terms of features manually annotated in the Penn Discourse TreeBank — discourse connectives and their senses.

This paper has, for the first time, provided genre information about the articles in the Penn TreeBank .

It has characterised each genre in terms of features manually annotated in the Penn Discourse TreeBank , and used this to show that genre should be made a factor in automated sense labelling of discourse relations that are not explicitly marked.

Although the files in the Penn TreeBank (PTB) lack any classificatory meta-data, leading the PTB to be treated as a single homogeneous collection of “news articles”, researchers who have manually examined it in detail have noted that it includes a variety of “financial reports, general interest stories, business-related news, cultural reviews, editorials and letters to the editor” (Carlson et al., 2002, p. 7).

In lieu of any informative meta-data in the PTB filesl, I looked at line-level patterns in the 2159 files that make up the Penn Discourse TreeBank subset of the PTB, and then manually confirmed the text types I found.2 The resulting set includes all the

the Penn TreeBank that aren’t included in the PDTB.

This paper considers differences in texts in the well-known Penn TreeBank (hereafter, PTB) and in particular, how these differences show up in the Penn Discourse TreeBank (Prasad et al., 2008).

After a brief introduction to the Penn Discourse TreeBank (hereafter, PDTB) in Section 4, Sections 5 and 6 show that these four genres display differences in connective frequency and in terms of the senses associated with intra-sentential connectives (eg, subordinating conjunctions), inter-sentential connectives (eg, inter-sentential coordinating conjunctions) and those inter-sentential relations that are not lexically marked.

Genre differences at the level of discourse in the PTB can be seen in the manual annotations of the Penn Discourse TreeBank (Prasad et al., 2008).

Yoshida (2005) proposed methods for extracting a wide-coverage lexicon based on HPSG from a phrase structure treebank of Japanese.

Their treebanks are annotated with dependencies of words, the conversion of which into phrase structures is not a big concern.

Our method integrates multiple dependency-based resources to convert them into an integrated phrase structure treebank .

The obtained treebank is then transformed into CCG derivations.

As we have adopted the method of CCGbank, which relies on a source treebank to be converted into CCG derivations, a critical issue to address is the absence of a Japanese counterpart to PTB.

Our solution is to first integrate multiple dependency-based resources and convert them into a phrase structure treebank that is independent

Next, we translate the treebank into CCG derivations (Step 2).

Our work is basically an extension of a seminal work on CCGbank (Hockenmaier and Steedman, 2007), in which the phrase structure trees of the Penn Treebank (PTB) (Marcus et al., 1993) are converted into CCG derivations and a wide-coverage CCG lexicon is then extracted from these derivations.

In the case study we describe here, the tools, grammars and treebanks we use are taken from work carried out in the DELPH-IN1 collaboration.

We also use the PET parser, and the [incr tsdb()] system profiler and treebanking tool (Oepen, 2001) for evaluation.

The data set used for these experiments is the jh5 section of the treebank released with the ERG.

Since a gold standard treebank for our data set was available, it was possible to evaluate the accuracy of the parser.

Consequently, we developed a Maximum Entropy model for supertagging using the OpenNLP implementation.2 Similarly to Zhang and Kordoni (2006), we took training data from the gold—standard lexical types in the treebank associated with ERG (in our case, the July-07 version).

Four sets are English text: jh5 described in Section 3; tree consisting of questions from TREC and included in the treebanks released with the ERG; a00 which is taken from the BNC and consists of factsheets and newsletters; and depbank, the 700 sentences of the Briscoe and Carroll version of DepBank (Briscoe and Carroll, 2006) taken from the Wall Street Journal.

The last two data sets are German text: clef700 consisting of German questions taken from the CLEF competition and eiche564 a sample of sentences taken from a treebank parsed with the German HPSG grammar, GG and consisting of transcribed German speech data concerning appointment scheduling from the Verbmobil project.

Since the primary effect of adding POS tags is shown with those data sets for which we do not have gold standard treebanks , evaluating accuracy in this case is more difficult.

For these experiments, we use the Wall Street Journal portion of the Penn Treebank (Marcus et al., 1993).

Following the CoNLL shared task from 2000, we use sections 15-18 of the Penn Treebank for our labeled training data for the supervised sequence labeler in all experiments (Tjong et al., 2000).

For the tagging experiments, we train and test using the gold standard POS tags contained in the Penn Treebank .

For POS/parsing annotation, there are also a number of annotation schemes including the Brown tag set, the Claws tag set, and the Penn Treebank tag set.

For instance, there are at least three possibilities for POS-tagging the word sing in the sentence everyone sing together using the Penn Treebank tag set: singN B, sing/VBP, or sing/VBZ.

For similar reasons, to the best of our knowledge, there exists no such learner corpus that is manually shallow-parsed and which is also publicly available, unlike, say, native-speaker corpora such as the Penn Treebank .

We selected the Penn Treebank tag set, which is one of the most widely used tag sets, for our

Similar to the error annotation scheme, we conducted a pilot study to determine what modifications we needed to make to the Penn Treebank scheme.

As a result of the pilot study, we found that the Penn Treebank tag set sufficed in most cases except for errors which learners made.

Both use the Penn Treebank POS tag set.

An obvious cause of mistakes in both taggers is that they inevitably make errors in the POSs that are not defined in the Penn Treebank tag set, that is, UK and CE.

The resulting shift-reduce discourse parser obtains substantial improvements over the previous state-of-the-art in predicting relations and nuclearity on the RST Treebank .

We evaluate DPLP on the RST Discourse Treebank (Carlson et al., 2001), comparing against state-of-the-art results.

Dataset The RST Discourse Treebank (RST-DT) consists of 385 documents, with 347 for train-

We consider the values K E {30,60,90, 150}, A E {1,10,50, 100} and 7' E {1.0, 0.1, 0.01, 0.001}, and search over this space using a development set of thirty document randomly selected from within the RST Treebank training data.

Unfortunately, the performance of discourse parsing is still relatively weak: the state-of-the-art F—measure for text-level relation detection in the RST Treebank is only slightly above 55% (Joty

In addition, we show that the latent representation coheres well with the characterization of discourse connectives in the Penn Discourse Treebank (Prasad et al., 2008).

(2010) show that there is a long tail of alternative lexicalizations for discourse relations in the Penn Discourse Treebank , posing obvious challenges for approaches based on directly matching lexical features observed in the training data.

We apply transition-based (incremental) structured prediction to obtain a discourse parse, training a predictor to make the correct incremental moves to match the annotations of training data in the RST Treebank .

(2009) in the context of the Penn Discourse Treebank (Prasad et al., 2008).

Latent variable parsing assumes that an observed treebank represents a coarse approximation of an underlying, optimally refined grammar which makes more fine-grained distinctions in the syntactic categories.

For example, the noun phrase category NP in a treebank could be viewed as a coarse approximation of two noun phrase categories corresponding to subjects and object, NPS, and NP AVP.

It starts with a simple bi-narized X-bar grammar style backbone, and goes through iterations of splitting and merging nonterminals, in order to maximize the likelihood of the training set treebank .

Incorporating edge label information does not appear to improve performance, possibly because it oversplits the initial treebank and interferes with the parser’s ability to determine optimal splits for refining the grammar.

Hocken-maier (2006) has translated the German TIGER corpus (Brants et al., 2002) into a CCG—based treebank to model word order variations in German.

The corpus-based, stochastic topological field parser of Becker and Frank (2002) is based on a standard treebank PCFG model, in which rule probabilities are estimated by frequency counts.

Ule (2003) proposes a process termed Directed Treebank Refinement (DTR).

We demonstrate the effectiveness of the approach in a series of dependency parsing experiments on the Penn Treebank and Prague Dependency Treebank , and we show that the cluster-based features yield substantial gains in performance across a wide range of conditions.

A natural avenue for further research would be the development of clustering algorithms that reflect the syntactic behavior of words; e.g., an algorithm that attempts to maximize the likelihood of a treebank , according to a probabilistic dependency model.

The English experiments were performed on the Penn Treebank (Marcus et al., 1993), using a standard set of head-selection rules (Yamada and Matsumoto, 2003) to convert the phrase structure syntax of the Treebank to a dependency tree representation.6 We split the Treebank into a training set (Sections 2—21), a development set (Section 22), and several test sets (Sections 0,7 1, 23, and 24).

The Czech experiments were performed on the Prague Dependency Treebank 1.0 (Hajic, 1998; Hajic et al., 2001), which is directly annotated with dependency structures.

9We ensured that the sentences of the Penn Treebank were excluded from the text used for the clustering.

We show that our semi-supervised approach yields improvements for fixed datasets by performing parsing experiments on the Penn Treebank (Marcus et al., 1993) and Prague Dependency Treebank (Hajic, 1998; Hajic et al., 2001) (see Sections 4.1 and 4.3).

This is a significant problem for CCGbank, where binary branching NP derivations are often incorrect, a result of the automatic conversion from the Penn Treebank .

Recently, Vadas and Curran (2007a) annotated internal NP structure for the entire Penn Treebank , providing a large gold-standard corpus for NP bracketing.

We apply one preprocessing step on the Penn Treebank data, where if multiple tokens are enclosed by brackets, then a NML node is placed around those

Since we are applying these to CCGbank NP structures rather than the Penn Treebank , the POS tag based heuristics are sufficient to determine heads accurately.

Vadas and Curran (2007a) experienced a similar drop in performance on Penn Treebank data, and noted that the F-score for NML and JJP brackets was about 20% lower than the overall figure.

This is because their training data, the Penn Treebank (Marcus et al., 1993), does not fully annotate NP structure.

The flat structure described by the Penn Treebank can be seen in this example:

CCGbank (Hockenmaier and Steedman, 2007) is the primary English corpus for Combinatory Categorial Grammar (CCG) (Steedman, 2000) and was created by a semiautomatic conversion from the Penn Treebank .

Using the Penn Treebank sentences associated with each SR Task dependency tree, we will create the two tree sets necessary to support error mining by dividing the set of trees output by the surface realiser into a set of trees (FAIL) associated with overgeneration (the generated sentences do not match the original sentences) and a set of trees (SUCCESS) associated with success (the generated sentence matches the original sentences).

The shallow input data provided by the SR Task was obtained from the Penn Treebank using the LTH Constituent—to—Dependency Conversion Tool for Penn—style Treebanks (Pennconverter, (J ohans—son and Nugues, 2007)).

The chunking was performed by retrieving from the Penn Treebank (PTB), for each phrase type, the yields of the constituents of that type and by using the alignment between words and dependency tree nodes provided by the organisers of the SR Task.

5 In the Penn Treebank , the POS tag is the category assigned to possessive ’s.

(Callaway, 2003) avoids this shortcoming by converting the Penn Treebank to the format expected by his realiser.

In this paper, we investigate various strategies to predict both syntactic dependency parsing and contiguous multiword expression (MWE) recognition, testing them on the dependency version of French Treebank (Abeille and Barrier, 2004), as instantiated in the SPMRL Shared Task (Seddah et al., 2013).

We experimented strategies to predict both MWE analysis and dependency structure, and tested them on the dependency version of French Treebank (Abeille and Barrier, 2004), as instantiated in the SPMRL Shared Task (Seddah et al., 2013).

It contains projective dependency trees that were automatically derived from the latest status of the French Treebank (Abeille and Barrier, 2004), which consists of constituency trees for sentences from the

For instance, in the French Treebank , population active (lit.

The French dataset is the only one containing MWEs: the French treebank has the particularity to contain a high ratio of tokens belonging to a MWE (12.7% of non numerical tokens).

Our representation also resembles that of light-verb constructions (LVC) in the hungarian dependency treebank (Vincze et al., 2010): the construction has regular syntax, and a suffix is used on labels to express it is a LVC (Vincze et al., 2013).

In order to compare the MWEs present in the lexicons and those encoded in the French treebank , we applied the following procedure (hereafter called lexicon

We had to convert the DELA POS tagset to that of the French Treebank .

We use the standard test set for this task, a 24,115-word subset of the Penn Treebank , for which a gold tag sequence is available.

They show considerable improvements in tagging accuracy when using a coarser-grained version (with l7-tags) of the tag set from the Penn Treebank .

In contrast, we keep all the original dictionary entries derived from the Penn Treebank data for our experiments.

Their models are trained on the entire Penn Treebank data (instead of using only the 24,115-token test data), and so are the tagging models used by Goldberg et al.

ing data from the 24,115-t0ken set to the entire Penn Treebank (973k tokens).

Their systems were shown to obtain considerable improvements in accuracy when using a l7-tagset (a coarser-grained version of the tag labels from the Penn Treebank ) instead of the 45-tagset.

The accuracy numbers reported for Init-HMM and LDA+AC are for models that are trained on all the available unlabeled data from the Penn Treebank .

The IP+EM models used in the l7-tagset experiments reported here were not trained on the entire Penn Treebank , but instead used a smaller section containing 77,963 tokens for estimating model parameters.

We explore the extent to which high-resource manual annotations such as treebanks are necessary for the task of semantic role labeling (SRL).

To compare with prior approaches that use semantic supervision for grammar induction, we utilize Section 23 of the WSJ portion of the Penn Treebank (Marcus et al., 1993).

We contrast low-resource (D) and high-resource settings (E), where latter uses a treebank .

We therefore turn to an analysis of other approaches to grammar induction in Table 8, evaluated on the Penn Treebank .

However, richly annotated data such as that provided in parsing treebanks is expensive to produce, and may be tied to specific domains (e.g., newswire).

(2012) observe that syntax may be treated as latent when a treebank is not available.

(2011) require an oracle CCG tag dictionary extracted from a treebank .

There has not yet been a comparison of techniques for SRL that do not rely on a syntactic treebank , and no exploration of probabilistic models for unsupervised grammar induction within an SRL pipeline that we have been able to find.

As a result, many structures that in other treebanks would be prepositional phrases with embedded noun phrases — and thus nonlocal constituents — are flat prepositional phrases here.

3For the Penn Treebank tagset, see Marcus et a1.

1999); for German, the Negra corpus V2 (Krenn et al., 1998); for Chinese, the Penn Chinese Treebank V5.0 (CTB, Palmer et al., 2006).

Sentence segmentation and tok-enization from the treebank is used.

Their output is not evaluated directly using treebanks , but rather applied to several information retrieval problems.

Examples of constituent chunks extracted from treebank constituent trees are in Fig.

One study by Cramer (2007) found that none of the three performs particularly well under treebank evaluation.

21,938 total examples, 15,330 come from sections 2—21 of the Penn Treebank (Marcus et al., 1993).

For the Penn Treebank , we extracted the examples using the provided gold standard parse trees, whereas, for the latter cases, we used the output of an open source parser (Tratz and Hovy, 2011).

The accuracy figures for the test instances from the Penn Treebank , The Jungle Book, and The History of the Decline and Fall of the Roman Empire were 88.8%, 84.7%, and 80.6%, respectively.

The NomBank project (Meyers et al., 2004) provides coarse annotations for some of the possessive constructions in the Penn Treebank , but only those that meet their criteria.

Penn Treebank , respectively.

portion of the Penn Treebank .

The Penn Treebank and The History of the Decline and Fall of the R0-man Empire were substantially more similar, although there are notable differences.

The Ancient Greek treebank comprises both archaic texts, before the development of a definite article, and later classic Greek, which has a definite article; Hungarian has both a definite and an indefinite article.

More importantly, the Latin Dependency Treebank has grown from about 30K at the time of the previous work to 53K at present, resulting in significantly different training and testing material.

Our evaluation focused on the Latin Dependency Treebank (Bamman and Crane, 2006), created at the Perseus Digital Library by tailoring the Prague Dependency Treebank guidelines for the Latin language.

We randomly divided the 53K—word treebank into 10 folds of roughly equal sizes, with an average of 5314 words (347 sentences) per fold.

Their respective datasets consist of 8000 sentences from the Ancient Greek Dependency Treebank (Bamman et al., 2009), 5800 from the Hungarian Szeged Dependency Treebank (Vincze et al., 2010), and a subset of 3100 from the Prague Dependency Treebank (Bohmova et al., 2003).

Similarly, the English POS tags in the Penn Treebank combine word class information with morphologi-

Ad hoc rules are CFG productions extracted from a treebank which are “used for specific constructions and unlikely to be used again,” indicating annotation errors and rules for ungram-maticalities (see also Dickinson and Foster, 2009).

Each method compares a given CFG rule to all the rules in a treebank grammar.

This procedure is applicable whether the rules in question are from a new data set—as in this paper, where parses are compared to a training data grammar—or drawn from the treebank grammar itself (i.e., an internal consistency check).

Furthermore, parsing accuracy degrades unless sufficient amounts of labeled training data from the same domain are available (e.g., Gildea, 2001; Sekine, 1997), and thus we need larger and more varied annotated treebanks , covering a wide range of domains.

However, there is a bottleneck in obtaining annotation, due to the need for manual intervention in annotating a treebank .

We have proposed different methods for flagging the errors in automatically-parsed corpora, by treating the problem as one of looking for anomalous rules with respect to a treebank grammar.

In practice, we can extract the best reading of the most ambiguous sentence in the Rondane treebank (4.5 x 1012 readings, 75 000 grammar rules) with random soft edges in about a second.

For instance, the following sentence from the Rondane treebank is analyzed as having six quantifiers and 480 readings by the ERG grammar; these readings fall into just two semantic equivalence classes, characterized by the relative scope of “the lee of” and “a small hillside”.

To measure the extent to which the new algorithm improves upon KT06, we compare both algorithms on the USRs in the Rondane treebank (version of January 2006).

The Rondane treebank is a “Redwoods style” treebank (Oepen et al., 2002) containing MRS-based underspecified representations for sentences from the tourism domain, and is distributed together with the English Resource Grammar (ERG) (Copestake and Flickinger, 2000).

2 compares the average number of configurations and the average number of RTG production rules for USRs of increasing sizes in the Rondane treebank (see Sect.

Computing the charts for all 999 MRS-nets in the treebank takes about 45 seconds.

These predicates are among the most frequent in the TreeBank and are likely to require approaches that differ from the ones we pursued.

Implicit arguments have not been annotated within the Penn TreeBank , which is the textual and syntactic basis for NomBank.

Thus, to facilitate our study, we annotated implicit arguments for instances of nominal predicates within the standard training, development, and testing sections of the TreeBank .

Consider the following abridged sentences, which are adjacent in their Penn TreeBank document:

Starting with a wide range of features, we performed floating forward feature selection (Pudil et al., 1994) over held-out development data comprising implicit argument annotations from section 24 of the Penn TreeBank .

Throughout our study, we used gold-standard discourse relations provided by the Penn Discourse TreeBank (Prasad et al., 2008).

However, as shown by the following example from the Penn TreeBank (Marcus et al., 1993), this restriction excludes extra-sentential arguments:

Two models are the volunteers who scan documents and correct OCR output in Project Gutenberg, or the undergraduate volunteers who have constructed Greek and Latin treebanks within Project Perseus (Crane, 2010).

It is natural to think in terms of replicating the body of resources available for well-documented languages, and the preeminent resource for any language is a treebank .

Producing a treebank involves a staggering amount of manual effort.

The idea of producing treebanks for 6,900 languages is quixotic, to put it mildly.

We compare the CCG parser of Clark and Curran (2007) with a state-of-the-art Penn Treebank (PTB) parser.

The first approach, which began in the mid-90$ and now has an extensive literature, is based on the Penn Treebank (PTB) parsing task: inferring skeletal phrase-structure trees for unseen sentences of the W8], and evaluating accuracy according to the Parseval metrics.

The second approach is to apply statistical methods to parsers based on linguistic formalisms, such as HPSG, LFG, TAG, and CCG, with the grammar being defined manually or extracted from a formalism-specific treebank .

Evaluation is typically performed by comparing against predicate-argument structures extracted from the treebank , or against a test set of manually annotated grammatical relations (GRs).

However, there are a number of differences between the two treebanks which make the conversion back far from trivial.

First, the corresponding derivations in the treebanks are not isomorphic: a CCG derivation is not simply a relabelling of the nodes in the PTB tree; there are many constructions, such as coordination and control structures, where the trees are a different shape, as well as having different labels.

Our SR-TSG parser achieves an F 1 score of 92.4% in the Wall Street Journal (WSJ) English Penn Treebank parsing task, which is a 7.7 point improvement over a conventional Bayesian TSG parser, and better than state-of-the-art discriminative reranking parsers.

We ran experiments on the Wall Street Journal (WSJ) portion of the English Penn Treebank data set (Marcus et al., 1993), using a standard data split (sections 2—21 for training, 22 for development and 23 for testing).

The treebank data is right-binarized (Matsuzaki et al., 2005) to construct grammars with only unary and binary productions.

This result suggests that the conventional TSG model trained from the vanilla treebank is insufficient to resolve

Probabilistic context-free grammar (PCFG) underlies many statistical parsers, however, it is well known that the PCFG rules extracted from treebank data Via maximum likelihood estimation do not perform well due to unrealistic context freedom assumptions (Klein and Manning, 2003).

Symbol refinement is a successful approach for weakening context freedom assumptions by dividing coarse treebank symbols (e.g.

Our SR-TSG parser achieves an F1 score of 92.4% in the WSJ English Penn Treebank parsing task, which is a 7.7 point improvement over a conventional Bayesian TSG parser, and superior to state-of-the-art discriminative reranking parsers.

This paper introduces a new technique for phrase-structure parser analysis, categorizing possible treebank structures by integrating regular expressions into derivation trees.

We analyze the performance of the Berkeley parser on OntoNotes WSJ and the English Web Treebank .

The high coverage (%) reinforces the point that there is a limited number of core structures in the treebank .

1We refer only to the WSJ treebank portion of OntoNotes, which is roughly a subset of the Penn Treebank (Marcus et al., 1999) with annotation revisions including the addition of NML nodes.

We derived the regexes via an iterative process of inspection of tree decomposition on dataset (a), together with taking advantage of the treebanking experience from some of the coauthors.

Second, we use a set of regular expressions (henceforth “regexes”) that categorize the possible structures in the treebank .

After describing in more detail the basic framework, we show some aspects of the resulting analysis of the performance of the Berkeley parser (Petrov et al., 2008) on three datasets: (a) OntoNotes WSJ sections 2-21 (Weischedel et al., 2011)1, (b) OntoNotes WSJ section 22, and (c) the “Answers” section of the English Web Treebank (Bies et al., 2012).

Our sentiment analysis datasets consist of movie reviews from the Stanford sentiment treebank (Socher et al., 2013),11 and floor speeches by US.

Congressmen alongside “yea”/“nay” votes on the bill under discussion (Thomas et al., 2006).12 For the Stanford sentiment treebank , we only predict binary classifications (positive or negative) and exclude neutral reviews.

Figure 1: An example of a parse tree from the Stanford sentiment treebank , which annotates sentiment at the level of every constituent (indicated here by —|— and ++; no marking indicates neutral sentiment).

The Stanford sentiment treebank has an annotation of sentiments at the constituent level.

Figure 1 illustrates the group structures derived from an example sentence from the Stanford sentiment treebank (Socher et al., 2013).

On the other hand, consider the annotation guideline of English Treebank (Marcus et a1., 1993) instead.

Following this POS representation, there are as many as 10 possible POS tags that may occur in between the—0f, as estimated from the WSJ corpus of Penn Treebank .

To explore determinacy in the distribution of POS tags in Penn Treebank , we need to consider that a POS tag marks the basic syntactic category of a word as well as its morphological inflection.

The most well-known multiply-annotated and validated corpus of English is the one million word Wall Street Journal corpus known as the Penn Treebank (Marcus et al., 1993), which over the years has been fully or partially annotated for several phenomena over and above the original part-of-speech tagging and phrase structure annotation.

More recently, the OntoNotes project (Pradhan et al., 2007) released a one million word English corpus of newswire, broadcast news, and broadcast conversation that is annotated for Penn Treebank syntax, PropBank predicate argument structures, coreference, and named entities.

words Token Validated 1 18 222472 Sentence Validated 1 18 222472 POS/lemma Validated 1 18 222472 Noun chunks Validated 1 18 222472 Verb chunks Validated 1 18 222472 Named entities Validated 1 18 222472 FrameNet frames Manual 21 17829 HSPG Validated 40* 30106 Discourse Manual 40* 30106 Penn Treebank Validated 97 873 83 PropB ank Validated 92 50165 Opinion Manual 97 47583 TimeB ank Validated 34 5434 Committed belief Manual 13 4614 Event Manual 13 4614 Coreference Manual 2 1 877

Annotations produced by other projects and the FrameNet and Penn Treebank annotations produced specifically for MASC are semiautomatically and/or manually produced by those projects and subjected to their internal quality controls.

All of the first 80K increment is annotated for Penn Treebank syntax.

The second 120K increment includes 5.5K words of Wall Street Journal texts that have been annotated by several projects, including Penn Treebank, PropBank, Penn Discourse Treebank , TimeML, and the Pittsburgh Opinion project.

6We replicated the Treebank for the 100,000 sentences pass.

Table 1: Performance numbers for computing Viterbi inside charts on 20,000 sentences of length $40 from the Penn Treebank .

As with other grammars with a parse/derivation distinction, the grammars of Petrov and Klein (2007) only achieve their full accuracy using minimum-Bayes-risk parsing, with improvements of over 1.5 F1 over best-derivation Viterbi parsing on the Penn Treebank (Marcus et al., 1993).

Table 2: Performance numbers for computing max constituent (Goodman, 1996) trees on 20,000 sentences of length 40 or less from the Penn Treebank .

Therefore, in the fine pass, we normalize the inside scores at the leaves to sum to 1.0.5 Using this slight modification, no sentences from the Treebank under- or overflow.

We measured parsing accuracy on sentences of length g 40 from section 22 of the Penn Treebank .

structure treebank , namely CTB.

Our treebank conversion algorithm borrows key insights from Lexical Functional Grammar (LFG; Bresnan and Kaplan, 1982; Dalrymple, 2001).

There are two sources of errors in treebank conversion: (1) inadequate conversion rules and (2) wrong or inconsistent original annotations.

To acquire high-quality GR corpus, we propose a linguistically-motivated algorithm to translate a Government and Binding (GB; Chomsky, 1981; Camie, 2007) grounded phrase structure treebank , i.e.

Chinese Treebank (CTB; Xue et al., 2005) to a deep dependency bank where GRs are explicitly represented.

The availability of large-scale treebanks has contributed to the blossoming of statistical approaches to build accurate shallow constituency and dependency parsers.

Though PoS induction was not their aim, this restriction is largely validated by empirical analysis of treebanked data, and moreover conveys the significant advantage that all the tags for a given word type can be updated at the same time, allowing very efficient inference using the exchange algorithm.

Recent work on unsupervised PoS induction has focussed on encouraging sparsity in the emission distributions in order to match empirical distributions derived from treebank data (Goldwater and Griffiths, 2007; Johnson, 2007; Gao and Johnson, 2008).

Treebank (Marcus et al., 1993), while for other languages we use the corpora made available for the CoNLL-X Shared Task (Buchholz and Marsi, 2006).

Treebank , along with a number of state-of—the-art results previously reported (Table 1).

The former shows that both our models and mkcl s induce a more uniform distribution over tags than specified by the treebank .

In Proceedings of the 5th International Workshop on Treebanks and Linguistic Theories.

Recognizing Implicit Discourse Relations in the Penn Discourse Treebank .

The Penn Discourse TreeBank 2.0.

We directly use the golden standard parse trees in Penn TreeBank .

The experiment shows that tree kernel is able to effectively incorporate syntactic structural information and produce statistical significant improvements over flat syntactic path feature for the recognition of both explicit and implicit relation in Penn Discourse Treebank (PDTB; Prasad et al., 2008).

The Penn Discourse Treebank (PDTB) is the largest available annotated corpora of discourse relations (Prasad et al., 2008) over 2,312 Wall Street Journal articles.

Empty categories (EC) are artificial elements in Penn Treebanks motivated by the govemment-binding (GB) theory to explain certain language phenomena such as pro-drop.

The empty categories in the Chinese Treebank (CTB) include trace markers for A’- and A-movement, dropped pronoun, big PRO etc.

Our effort of recovering ECs is a two-step process: first, at training time, ECs in the Chinese Treebank are moved and preserved in the portion of the tree structures pertaining to surface words only.

We use Chinese Treebank (CTB) V7.0 to train and test the EC prediction model.

In order to account for certain language phenomena such as pro-drop and wh-movement, a set of special tokens, called empty categories (EC), are used in Penn Treebanks (Marcus et al., 1993; Bies and Maamouri, 2003; Xue et al., 2005).

The experiments were performed on the Penn Treebank (PTB) (Marcus et al., 1993), using a standard set of head-selection rules (Yamada

and Matsumoto, 2003) to convert the phrase structure syntax of the Treebank into a dependency tree representation, dependency labels were obtained via the ”Malt” hard-coded setting.8 We split the Treebank into a training set (Sections 2-2l), a development set (Section 22), and several test sets (Sections 0,9 l, 23, and 24).

The results show that our second order model incorporating the N-gram features (92.64) performs better than most previously reported discriminative systems trained on the Treebank .

With the availability of large-scale annotated corpora such as Penn Treebank (Marcus et al., 1993), it is easy to train a high-performance dependency parser using supervised learning methods.

We conduct the experiments on the English Penn Treebank (PTB) (Marcus et al., 1993).

We perform parsing experiments the Penn Treebank and draw comparisons to Tree-Substitution Grammars and between different variations in probabilistic model design.

As a proof of concept, we investigate OSTAG in the context of the classic Penn Treebank statistical parsing setup; training on section 2-21 and testing on section 23.

Furthermore, the various parameteri-zations of adjunction with OSTAG indicate that, at least in the case of the Penn Treebank , the finer grained modeling of a full table of adjunction probabilities for each Goodman index OSTAG3 overcomes the danger of sparse data estimates.

We evaluate OSTAG on the familiar task of parsing the Penn Treebank .

We propose a simple but empirically effective heuristic for grammar induction for our experiments on Penn Treebank data.

Experiments are conducted on the Wall Street Journal portion of the English Penn Treebank .

We prepare three training sets: the complete training set of 39,832 sentences from the treebank (sections 2 through 21), a smaller training set, consisting of the first 3000 sentences, and an even smaller set of the first 300.

test on the French treebank (the “French” column).

Example: the infrequently-occurring treebank tag UH dominates greetings (among other interjections).

Example: individual first names are also rare in the treebank , but tend to cluster together in distributional representations.

These constraints are learned from relation-head-modifier co-occurrence counts estimated from a dependency treebank £.

Algorithm 1 SentenceGeneration(U, 9, 73, [2): U is the user specification, 8 is a set of meta-parameters; 73 and £3 are two dependency treebanks .

We estimate the probability of a modifier word m and its head h to be in the relation r as Mb, m) = Cr(h, m)/(Zh, Em, CAM» 77%)), where cr(-) is the number of times that m depends on h in the dependency treebank £ and hi, m,- are all the head/modifier pairs observed in £.

BRAINSUP makes heavy use of dependency parsed data and statistics collected from dependency treebanks to ensure the grammaticality of the generated sentences, and to trim the search space while seeking the sentences that maximize the user satisfaction.

As discussed in Section 3 we use two different treebanks to learn the syntactic patterns (’P) and the dependency operators (£).

This paper focuses on two representative popular corpora for Chinese lexical processing: (1) the Penn Chinese Treebank (CTB) and (2) the PKU’s People’s Daily data (PPD).

Penn Chinese Treebank (CTB) and PKU’s People’s Daily (PPD), on manually mapped data, and show that their linguistic annotations are systematically different and highly compatible.

A well known work is transforming Penn Treebank into resources for various deep linguistic processing, including LTAG (Xia, 1999), CCG (Hockenmaier and Steedman, 2007), HP SG (Miyao et al., 2004) and LFG (Cahill et al., 2002).

For example, the Penn Treebank is popular to train PCFG-based parsers, while the Redwoods Treebank is well known for HP SG research; the Propbank is favored to build general semantic role labeling systems, while the FrameNet is attractive for predicate-specific labeling.

Penn Chinese Treebank (CTB) and PKU’s People’s Daily (PPD).

More than 25 treebanks (in 22 languages) can be automatically mapped to this tag set, which includes “Noun” (nominals), “Verb” (verbs), “Adj” (adjectives), and “ADP” (pre-and postpositions).

Many of these treebanks also contain per-token morphological annotations.

We trained a simple add-1 smoothed bigram language model over gold class sequences in the same treebank training data:

The model can be implemented with a standard CRF package, trained on existing treebanks for many languages, and integrated easily with many MT feature APIs.

Experimental Setup All experiments use the Penn Arabic Treebank (ATB) (Maamouri et al., 2004) parts 1—3 divided into training/dev/test sections according to the canonical split (Rambow et al., 2005).7

(2005), trained on sections 2—21 of the WSJ Penn Treebank , transformed to dependency trees following Yamada and Matsumoto (2003).

(The same treebank data were also to estimate many of the parameters of our model, as discussed in the text.)

4 is estimated in our model using the transformed treebank (see footnote 4).

The 9 documents include 3 texts from the RST literature2 , 3 online product reviews from Epinions.com, and 3 Wall Street Journal articles taken from the Penn Treebank .

Many of our differences with Carlson and Marcu (2001), who defined EDUs for the RST Discourse Treebank (Carlson et al., 2002), are due to the fact that we adhere closer to the original RST proposals (Mann and Thompson, 1988), which defined as ‘spans’ adjunct clauses, rather than complement (subject and object) clauses.

SPADE is trained on the RST Discourse Treebank (Carlson et al., 2002).

(2004) construct a rule-based segmenter, employing manually annotated parses from the Penn Treebank .

High F—score in the Treebank data can be attributed to the parsers having been trained on Treebank .

We used three corpora for experiments: WSJ from Penn Treebank , Wikipedia, and the general Web.

In contrast, TextRunner was trained with 91,687 positive examples and 96,795 negative examples generated from the WSJ dataset in Penn Treebank .

We used three parsing options on the WSJ dataset: Stanford parsing, C] 50 parsing (Charniak and Johnson, 2005), and the gold parses from the Penn Treebank .

For example, TextRunner uses a small set of handwritten rules to heuristically label training examples from sentences in the Penn Treebank .

In both cases, however, we generate training data from Wikipedia by matching sentences with infoboxes, while TextRunner used a small set of handwritten rules to label training examples from the Penn Treebank .

For instance, both the PTB and the Prague Dependency Treebank (Bo'hmova et al., 2003) employed annotators with extensive linguistic background.

In fact, the annotations of (a) and (c) are identical under the most widely-used schemes for English, the Penn Treebank (PTB) (Marcus et al., 1993) and CoNLL-style dependencies (Surdeanu et al., 2008) (see Figure l).

The most prominent annotation scheme in NLP for English syntax is the Penn Treebank .

Examples include the Groningen Meaning bank (Basile et al., 2012), Treebank Semantics (Butler and Yoshi-moto, 2012) and the Lingo Redwoods treebank (Oepen et al., 2004).

3This data sparsity problem is quite severe — for example, the Penn treebank (Marcus et a1., 1993) has a total number of 43,498 sentences, with 42,246 unique POS tag sequences, averaging to be 1.04.

For English we use the Penn treebank (Marcus et al., 1993), with sections 2—21 for training and section 23 for final testing.

For German and Chinese we use the Ne-gra treebank and the Chinese treebank respectively and the first 80% of the sentences are used for training and the last 20% for testing.

Lastly, in step (i) of Figure 1, we run k-means clustering method on the S-CODE sphere and split word-substitute word pairs into 45 clusters because the treebank we worked on uses 45 part-of—speech tags.

The experiments are conducted on Penn Treebank Wall Street Journal corpus.

Because we are trying to improve (Yatbaz et al., 2012), we select the experiment on Penn Treebank Wall Street Journal corpus in that work as our baseline and replicate it.

For instance,the gold tag perplexity of word “offers” in the Penn Treebank Wall Street Journal corpus we worked on equals to 1.966.

It achieves 87.8% labelled attachment score and 88.8% unlabeled attachment score on the standard Penn Treebank test set.

On the standard Penn Treebank test set, it achieves an F-score of 89.5%.

The parser preprocesses the Penn Treebank training data through binarization.

For example, Penn Treebank annotations are often flat at the phrase level.

Using parse accuracy in a simple reranking strategy for self-monitoring, we find that with a state-of-the-art averaged perceptron realization ranking model, BLEU scores cannot be improved with any of the well-known Treebank parsers we tested, since these parsers too often make errors that human readers would be unlikely to make.

A limitation of the experiments reported in this paper is that OpenCCG’s input semantic dependency graphs are not the same as the Stanford dependencies used with the Treebank parsers, and thus we have had to rely on the gold parses in the PTB to derive gold dependencies for measuring accuracy of parser dependency recovery.

With this simple reranking strategy and each of three different Treebank parsers, we find that it is possible to improve BLEU scores on Penn Treebank development data with White & Rajkumar’s (2011; 2012) baseline generative model, but not with their averaged perceptron model.

We ran two OpenCCG surface realization models on the CCGbank dev set (derived from Section 00 of the Penn Treebank ) and obtained n-best (n = 10) realizations.

In a syntactic tree, each node indicates a clause/phrase/word and is only labeled with a Treebank tag (Marcus et al., 1993).

The Treebank tag, unfortunately, is usually too coarse or too general to capture semantic information.

where Ln is its phrase label (i.e., its Treebank tag), and F7, is a feature vector which indicates the characteristics of node n, which is represented as:

The first group comes from the English Web Treebank (EWT),4 also used in the Parsing the Web shared task (Petrov and McDonald, 2012).

We train our tagger on Sections 2—21 of the WSJ data in the Penn-III Treebank (PTB), Ontonotes 4.0 release.

Finally we do experiments with the Danish section of the Copenhagen Dependency Treebank (CDT).

Labeled English data employed in this paper were derived from the Wall Street Journal (WSJ) corpus of the Penn Treebank (Marcus et al., 1993).

For labeled Chinese data, we used the version 5 .1 of the Penn Chinese Treebank (CTB) (Xue et al., 2005).

In addition, we removed from the unlabeled English data the sentences that appear in the WSJ corpus of the Penn Treebank .

On standard evaluations using both the Penn Treebank and the Penn Chinese Treebank , our parser gave higher accuracies than the Berkeley parser (Petrov and Klein, 2007), a state-of-the-art chart parser.

EXperiments on English and Chinese treebanks confirm its advantage over its first-order version.

Our parsing models are evaluated on both English and Chinese treebanks, i.e., the WSJ section of Penn Treebank 3.0 (LDC99T42) and the Chinese Treebank 5.1 (LDC2005T01U01).

For parser combination, we follow the setting of Fossum and Knight (2009), using Section 24 instead of Section 22 of WSJ treebank as development set.

Evaluated on the PTB WSJ and Chinese Treebank , it achieves its best Fl scores of 91.86% and 85.58%, respectively.

Our final verb features were the part of speech tags (gold-standard from the Penn Treebank ) of the main verb.

For our experiments, we use the Penn Discourse Treebank , the largest existing corpus of discourse annotations for both implicit and explicit relations.

For our experiments, we use the Penn Discourse Treebank (PDTB; Prasad et al., 2008), the largest available annotated corpora of discourse relations.

The PDTB contains discourse annotations over the same 2,312 Wall Street Journal (WSJ) articles as the Penn Treebank .

Data The Penn Korean Treebank (Han et al., 2002) consists of 5,083 Korean sentences translated into English for the purposes of language training in a military setting.

The English-Urdu parallel corpus3 consists of 4,325 sentences from the first three sections of the Penn Treebank and their Urdu translations annotated at the part-of-speech level.

We use the remaining sections of the Penn Treebank for English testing.

Our scheme, inspired by the Penn Discourse TreeBank (PDTB), adopts the lexically grounded approach; at the same time, it makes adaptations based on the linguistic and statistical characteristics of Chinese text.

According to a rough count on 20 randomly selected files from Chinese Treebank (Xue et al., 2005), 82% are tokens of implicit relation, compared to 54.5% in the PDTB 2.0.

The data set consists of 98 files taken from the Chinese Treebank (Xue et al., 2005).

In the realm of discourse annotation, the Penn Discourse TreeBank (PDTB) (Prasad et al., 2008) separates itself by adopting a lexically grounded approach: Discourse relations are lexically anchored by discourse connectives (e.g., because, but, therefore), which are viewed as predicates that take abstract objects such as propositions, events and states as their arguments.

Trained on 8,975 dependency structures of a Chinese Dependency Treebank , the realizer achieves a BLEU score of 0.8874.

And for training the headword model, we use both the HIT-CDT and the HIT Chinese Skeletal Dependency Treebank (HIT-CSDT).

The grammar rules are either developed by hand, such as those used in LinGo (Carroll et al., 1999), OpenCCG (White, 2004) and XLE (Crouch et al., 2007), or extracted automatically from annotated corpora, like the HPSG (Nakanishi et al., 2005), LFG (Cahill and van Genabith, 2006; Hogan et al., 2007) and CCG (White et al., 2007) resources derived from the Penn-II Treebank,

The input to our sentence realizer is a dependency structure as represented in the HIT Chinese Dependency Treebank (HIT-CDT)1.

CCGbank was created by semiautomatically converting the Penn Treebank to CCG derivations (Hockenmaier and Steedman, 2007).

CCG-TUT was created by semiautomatically converting dependencies in the Italian Turin University Treebank to CCG derivations (Bos et al., 2009).

Most work has focused on POS-tagging for English using the Penn Treebank (Marcus et al., 1993), such as (Banko and Moore, 2004; Goldwater and Griffiths, 2007; Toutanova and J ohn-son, 2008; Goldberg et al., 2008; Ravi and Knight, 2009).

This generally involves working with the standard set of 45 POS-tags employed in the Penn Treebank .

By exploiting an existing syntactic parser trained on a large treebank , our approach produces improved results on standard corpora, particularly when training data is limited or sentences are long.

Experiments on CLANG and GEOQUERY showed that the performance can be greatly improved by adding a small number of treebanked examples from the corresponding training set together with the WSJ corpus.

Listed together with their PARSEVAL F-measures these are: gold-standard parses from the treebank (GoldSyn, 100%), a parser trained on WSJ plus a small number of in-domain training sentences required to achieve good performance, 20 for CLANG (Syn20, 88.21%) and 40 for GEOQUERY (Syn40, 91.46%), and a parser trained on no in-domain data (Syn0, 82.15% for CLANG and 76.44% for GEOQUERY).

This demonstrates the advantage of utilizing existing syntactic parsers that are learned from large open domain treebanks instead of relying just on the training data.

Broad-coverage annotated treebanks necessary to train parsers do not exist for many resource-poor languages.

In our experiments we evaluate the learned models on dependency treebanks (Nivre et al., 2007).

(2005) with projective decoding, trained on sections 2-21 of the Penn treebank with dependencies extracted using the head rules of Yamada and Matsumoto (2003b).

We evaluate our approach by transferring from an English parser trained on the Penn treebank to Bulgarian and Spanish.

We run all experiments on the WSJ treebank (Marcus et al., 1999) using the standard splits: section 2-21 for training, section 22 for development, and section 23 for testing.

We preprocess the treebank by removing empty nodes, temporal labels, and spurious unary productions (X—>X), as is standard in published works on syntactic parsing.

To achieve state-of-the-art accuracy levels, we parse with the Berkeley SM6 latent-variable grammar (Petrov and Klein, 2007b) where the original treebank non-terminals are automatically split into subclasses to optimize parsing accuracy.

We simply parse sections 2-21 of the WSJ treebank and train our search models from the output of these trees, with no prior knowledge of the nonterminal set or other grammar characteristics to guide the process.

We investigate the models using the 2009 edition of the parallel treebank from UFAL (Bojar and Zabokrtsky, 2009), containing 8,029,801 sentence pairs from various genres.

The English-side annotation follows the standards of the Penn Treebank and includes dependency parses and structural role labels such as subject and object.

The Czech tags follow the standards of the Prague Dependency Treebank 2.0.

The inflection for number is particularly easy to model in translating from English, since it is generally marked on the source side, and POS taggers based on the Penn treebank tag set attempt to infer it in cases where it is not.

We use the WSJ 10 corpus (as processed by Smith (2006)), which is comprised of English sentences of ten words or fewer (after stripping punctuation) from the WSJ portion of the Penn Treebank .

It is our hope that this method will permit more effective leveraging of linguistic insight and resources and enable the construction of parsers in languages and domains where treebanks are not available.

While such supervised approaches have yielded accurate parsers (Chamiak, 2001), the syntactic annotation of corpora such as the Penn Treebank is extremely costly, and consequently there are few treebanks of comparable size.

The above methods can be applied to small seed corpora, but McDonald1 has criticized such methods as working from an unrealistic premise, as a significant amount of the effort required to build a treebank comes in the first 100 sentences (both because of the time it takes to create an appropriate rubric and to train annotators).

2.1 The RST Discourse Treebank

The RST Discourse Treebank (RST-DT) (Carlson et al., 2001), is a corpus annotated in the framework of RST.

2.2 The Penn Discourse Treebank

This corpus contains text of a similar domain to the TIGER treebank .

We train the log-linear ranking model on 7759 F-structures from the TIGER treebank .

We generate strings from each F-structure and take the original treebank string to be the labelled example.

We evaluate the string chosen by the log-linear model against the original treebank string in terms of exact match and BLEU score (Papineni et al.,

The training corpus for Mo-ME model consists of the Chinese Peen Treebank and the Chinese part of the LDC parallel corpus with about 2 million sentences.

According to our survey on the measure word distribution in the Chinese Penn Treebank and the test datasets distributed by Linguistic Data Consortium (LDC) for Chinese-to-English machine translation evaluation, the average occurrence is 0.505 and 0.319 measure

Table 1 shows the relative position’s distribution of head words around measure words in the Chinese Penn Treebank , where a negative position indicates that the head word is to the left of the measure word and a positive position indicates that the head word is to the right of the measure word.

According to our survey, about 70.4% of measure words in the Chinese Penn Treebank need

One obvious direction is to use the whole Penn Treebank as labeled data and use some other unannotated data source as unlabeled data for semi-supervised training.

For experiment on English, we used the English Penn Treebank (PTB) (Marcus et al., 1993) and the constituency structures were converted to dependency trees using the same rules as (Yamada and Matsumoto, 2003).

For Chinese, we experimented on the Penn Chinese Treebank 4.0 (CTB4) (Palmer et al., 2004) and we used the rules in (Bikel, 2004) for conversion.

We evaluate parsing accuracy by comparing the directed dependency links in the parser output against the directed links in the treebank .

Experiments on the Penn Chinese Treebank demonstrate the importance of both paradigmatic and syntagmatic relations.

We conduct experiments on the Penn Chinese Treebank and Chinese Gigaword.

Their evaluations on the Chinese Treebank show that Chinese POS tagging obtains an accuracy of about 93-94%.

Penn Chinese Treebank (CTB) (Xue et al., 2005) is a popular data set to evaluate a number of Chinese NLP tasks, including word segmentation (Sun and

It is compatible with the broader range of DELPH-IN tools, e. g., for machine translation (Lonning and Oepen, 2006), treebanking (Oepen et al., 2004) and parse selection (Toutanova et al., 2005).

treebanks .

With no prior knowledge of this language beyond its most general typological properties, we were able to develop in under 5.5 person-weeks of development time (210 hours) a grammar able to assign appropriate analyses to 91% of the examples in the development set.4 The 210 hours include 25 hours of an RA’s time entering lexical entries, 7 hours spent preparing the development test suite, and 15 hours treebanking (using the LinGO Redwoods software (Oepen et al., 2004) to annotate the intended parse for each item).

The resulting treebank was used to select appropriate parameters by 10-fold cross-validation, applying the experimentation environment and feature templates of (Velldal, 2007).

In this set, the 75 relations originally used in the RST Discourse Treebank (RST-DT) corpus (Carlson et al., 2001) are partitioned into 18 classes according to rhetorical similarity (e.g.

directly from the Penn Treebank corpus (which covers a superset of the RST-DT corpus), then “lexicalized” (i.e.

Figure 1: Example of a simple RST tree (Source: RST Discourse Treebank (Carlson et al., 2001),

Experiments on the translated portion of the Chinese Treebank show that our system outperforms monolingual parsers by 2.93 points for Chinese and 1.64 points for English.

All the bilingual data were taken from the translated portion of the Chinese Treebank (CTB) (Xue et al., 2002; Bies et al., 2007), articles 1-325 of CTB, which have English translations with gold-standard parse trees.

Experiments on the translated portion of the Chinese Treebank (Xue et al., 2002; Bies et al., 2007) show that our system outperforms state-of-the-art monolingual parsers by 2.93 points for Chinese and 1.64 points for English.

We classify 7 as typos and 26 as annotation inconsistencies, although the distinction between the two is murky: typos are intentionally preserved in the treebank data, but segmentation of typos varies depending on how well they can be reconciled with standard Arabic orthography.

The first example is segmented in the Egyptian treebank but is left unsegmented by our system; the second is left as a single token in the treebank but is split into the above three segments by our system.

We train and evaluate on three corpora: parts 1—3 of the newswire Arabic Treebank (ATB),1 the Broadcast News Arabic Treebank (BN),2 and parts 1—8 of the BOLT Phase 1 Egyptian Arabic Treebank (ARZ).3 These correspond respectively to the domains in section 2.2.

We conducted experiments on the Penn Chinese Treebank (CTB) version 5.1 (Xue et al., 2005): Articles 001-270 and 400-1151 were used as the training set, Articles 301-325 were used as the development set, and Articles 271-300 were used

To check whether more labeled data can further improve our parsing system, we evaluated our N0nlocal&Cluster system on the Chinese TreeBank version 6.0 (CTB6), which is a super set of CTB5 and contains more annotated data.

However, parse trees in Treebanks often contain an arbitrary number of branches.

These models have in common that they explicitly or implicitly use a context-free grammar induced from a treebank , with the exception of Chelba and J elinek (2000).

In order to compute the probability of a parse tree, it is transformed to a flat dependency tree similar to the syntax graph representation used in the TIGER treebank Brants et al (2002).

The resulting probability distributions were trained on the German TIGER treebank which consists of about 50000 sentences of newspaper text.

We describe an efficient framework for training our model based on the Margin Infused Relaxed Algorithm (MIRA), evaluate our approach on the Penn Chinese Treebank , and show that it achieves superior performance compared to the state-of-the-art approaches reported in the literature.

Previous studies on joint Chinese word segmentation and POS tagging have used Penn Chinese Treebank (CTB) (Xia et al., 2000) in experiments.

We conducted our experiments on Penn Chinese Treebank (Xia et al., 2000) and compared our approach with the best previous approaches reported in the literature.

(2007) describe an ongoing effort to engineer a grammar from the CCGbank (Hockenmaier and Steedman, 2007) — a corpus of CCG derivations derived from the Penn Treebank — suitable for realization with OpenCCG.

Our approach follows Langkilde-Geary (2002) and Callaway (2003) in aiming to leverage the Penn Treebank to develop a broad-coverage surface realizer for English.

However, while these earlier, generation-only approaches made use of converters for transforming the outputs of Treebank parsers to inputs for realization, our approach instead employs a shared bidirectional grammar, so that the input to realization is guaranteed to be the same logical form constructed by the parser.

Experimental results on the Chinese Treebank demonstrate improved performances over word-based parsing methods.

We use the Chinese Penn Treebank 5 .0, 6.0 and 7.0 to conduct the experiments, splitting the corpora into training, development and test sets according to previous work.

Their results on the Chinese Treebank (CTB) showed that character-level constituent parsing can bring increased performances even with the pseudo word structures.

Note that, because these domains are considerably different from the RST Treebank , the parser fails to produce a tree on a large number of answer candidates: 6.2% for YA, and 41.1% for Bio.

RST Treebank

performance on a small sample of seven WSJ articles drawn from the RST Treebank (Carlson et al., 2003).

Proposition Bank (Palmer et al., 2005) adds Levin’s style predicate-argument annotation and indication of verbs’ alternations to the syntactic structures of the Penn Treebank (Marcus et al.,

Verbal predicates in the Penn Treebank (PTB) receive a label REL and their arguments are annotated with abstract semantic role labels A0-A5 or AA for those complements of the predicative verb that are considered arguments, while those complements of the verb labelled with a semantic functional label in the original PTB receive the composite semantic role label AM-X, where X stands for labels such as LOC, TMP or ADV, for locative, temporal and adverbial modifiers respectively.

SemLink1 provides mappings from PropB ank to VerbNet for the WSJ portion of the Penn Treebank .

For English, we used the Penn Treebank (Marcus et al., 1993) in our experiments.

For Chinese, we used the Chinese Treebank (CTB) version 4.04 in the experiments.

3 We ensured that the text used for extracting subtrees did not include the sentences of the Penn Treebank .

We use the Penn treebank for our experiments and find that our proposal Bayesian TIG model not only has competitive parsing performance but also finds compact yet linguistically rich TIG representations of the data.

We use the standard Penn treebank methodology of training on sections 2—21 and testing on section 23.

carried out a small treebank experiment where we train on Section 2, and a large one where we train on the full training set.

We trained our model on sections 2—21 of the WSJ part of the Penn Treebank (Marcus et al., 1999).

Unfortunately, marking for argument/modifiers in the Penn Treebank is incomplete, and is limited to certain adverbials, e.g.

This corpus adds annotations indicating, for each node in the Penn Treebank , whether that node is a modifier.

We present a reformulation of the word pair features typically used for the task of disambiguating implicit relations in the Penn Discourse Treebank .

Previous work has relied on features based on the gold parse trees of the Penn Treebank (which overlaps with PDTB) and on contextual information from relations preceding the one being disambiguated.

More recently, implicit relation prediction has been evaluated on annotated implicit relations from the Penn Discourse Treebank (Prasad et al., 2008).

We use the Penn Chinese Treebank 5.0 (CTB) (Xue et al., 2005) as the existing annotated corpus for Chinese word segmentation.

Taking Chinese word segmentation for example, the state-of-the-art models (Xue and Shen, 2003; Ng and Low, 2004; Gao et al., 2005; Nakagawa and Uchimoto, 2007; Zhao and Kit, 2008; J iang et al., 2009; Zhang and Clark, 2010; Sun, 2011b; Li, 2011) are usually trained on human-annotated corpora such as the Penn Chinese Treebank (CTB) (Xue et al., 2005), and perform quite well on corresponding test sets.

In parsing, Pereira and Schabes (1992) proposed an extended inside-outside algorithm that infers the parameters of a stochastic CFG from a partially parsed treebank .

However, the treebanks necessary for training a high-accuracy parsing model are expensive to build for new domains.

An unlexicalized parser cannot distinguish these based just on POS tags, while a lexicalized parser requires a large treebank .

These typically rely on language-specific knowledge, either directly through heuristics, or indirectly through parsing models trained on treebanks .

For example, Higgins and Sadock (2003) find fewer than 1000 sentences with two or more explicit quantifiers in the Wall Street journal section of Penn Treebank .

Plurals form 18% of the NPs in our corpus and 20% of the nouns in Penn Treebank .

Explicit universals, on the other hand, form less than 1% of the determiners in Penn Treebank .

Using sections 00-21 of the treebanks , we handcrafted instructions for 527 lexical categories, a process that took under 100 hours, and includes all the categories used by the C&C parser.

Figure 3: For each sentence in the treebank , we plot the converted parser output against gold conversion (left), and the original parser evaluation against gold conversion (right).

Converting the Penn Treebank (PTB, Marcus et al., 1993) to other formalisms, such as HPSG (Miyao et al., 2004), LFG (Cahill et al., 2008), LTAG (Xia, 1999), and CCG (Hockenmaier, 2003), is a complex process that renders linguistic phenomena in formalism-specific ways.

In experiments using the Chinese Treebank (CTB), we show that the accuracies of the three tasks can be improved significantly over the baseline models, particularly by 0.6% for POS tagging and 2.4% for dependency parsing.

We perform experiments using the Chinese Treebank (CTB) corpora, demonstrating that the accuracies of the three tasks can be improved significantly over the pipeline combination of the state-of-the-art joint segmentation and POS tagging model, and the dependency parser.

’e use the Chinese Penn Treebank ver.

Adding the swapping operation changes the time complexity for deterministic parsing from linear to quadratic in the worst case, but empirical estimates based on treebank data show that the expected running time is in fact linear for the range of data attested in the corpora.

When building practical parsing systems, the oracle can be approximated by a classifier trained on treebank data, a technique that has been used successfully in a number of systems (Yamada and Matsumoto, 2003; Nivre et al., 2004; Attardi, 2006).

These languages have been selected because the data come from genuine dependency treebanks , whereas all the other data sets are based on some kind of conversion from another type of representation, which could potentially distort the distribution of different types of structures in the data.

Performance evaluation is carried out on the Penn Treebank (Marcus et al., 1993) converted to Stanford basic dependencies (De Marneffe etal., 2006).

This development is probably due to many factors, such as the increased availability of dependency treebanks and the perceived usefulness of dependency structures as an interface to downstream applications, but a very important reason is also the high efficiency offered by dependency parsers, enabling web-scale parsing with high throughput.

While the classical approach limits training data to parser states that result from oracle predictions (derived from a treebank ), these novel approaches allow the classifier to explore states that result from its own (sometimes erroneous) predictions (Choi and Palmer, 2011; Goldberg and Nivre, 2012).

, 36 (the number Penn treebank POS tags, used for the ‘POS’ models, is 36).6 For ‘Clust’, we see a comfortably wide plateau of nearly-identical scores from N = 7,. .

Label-based approaches have resulted in improvements in translation quality over the single X label approach (Zollmann et al., 2008; Mi and Huang, 2008); however, all the works cited here rely on stochastic parsers that have been trained on manually created syntactic treebanks .

These treebanks are difficult and expensive to produce and exist for a limited set of languages only.

Current state-of-the art syntactic parsers have achieved accuracies in the range of 90% F1 on the Penn Treebank , but a range of errors remain.

Figure l: A PP attachment error in the parse output of the Berkeley parser (on Penn Treebank ).

We use the standard splits of Penn Treebank into training (sections 2-21), development (section 22) and test (section 23).

Compared with the widely used Penn TreeBank annotation, the new criterion utilizes some different grammar tags and is able to effectively describe some rare language phenomena in Chinese.

The annotator still uses Penn TreeBank annotation on the English side.

In addition, HIT corpus is not applicable for MT experiment due to the problems of domain divergence, annotation discrepancy (Chinese parse tree employs a different grammar from Penn Treebank annotations) and degree of tolerance for parsing errors.

This method has been used effectively to improve parsing performance on newspaper text (McClosky et al., 2006a), as well as adapting a Penn Treebank parser to a new domain (McClosky et al., 2006b).

We have used Sections 02-21 of CCGbank (Hock-enmaier and Steedman, 2007), the CCG version of the Penn Treebank (Marcus et al., 1993), as training data for the newspaper domain.

Since the CCG lexical category set used by the supertagger is much larger than the Penn Treebank POS tag set, the accuracy of supertagging is much lower than POS tagging; hence the CCG supertagger assigns multiple supertags1 to a word, when the local context does not provide enough information to decide on the correct supertag.

We evaluate our parsers on the Penn Treebank and Prague Dependency Treebank , achieving unlabeled attachment scores of 93.04% and 87.38%, respectively.

We evaluate our parsers on the Penn WSJ Treebank (Marcus et al., 1993) and Prague Dependency Treebank (Hajic et al., 2001), achieving unlabeled attachment scores of 93.04% and 87.38%.

In order to evaluate the effectiveness of our parsers in practice, we apply them to the Penn WSJ Treebank (Marcus et al., 1993) and the Prague Dependency Treebank (Hajic et al., 2001; Hajic, 1998).6 We use standard training, validation, and test splits7 to facilitate comparisons.

5.4 Penn Treebank data set

The Penn Treebank 2 data set is available through LDC license at http: / /WWW .

edu/ ~treebank/ and contains 251,854 sentences with a total of 6,080,493 tokens and 45 different parts-of-speech.

2.1 Penn Discourse Treebank

In our work we use the Penn Discourse Treebank (PDTB) (Prasad et al., 2008), the largest public resource containing discourse annotations.

The syntactic features we used were extracted from the gold standard Penn Treebank (Marcus et al., 1994) parses of the PDTB articles: