This paper further explores linguistic features that explain why certain relations are preserved in English writing, and which contribute to related tasks such as native language identification.

1Recently, native language identification has drawn the attention of NLP researchers.

native language identification took place at an NAACL—HLT 2013 workshop.

This tendency in the length of noun-noun compounds provides us with a crucial insight for native language identification, which we will

Because some of the writers had more than one native language, we excluded essays that did not meet the following three conditions: (i) the writer has only one native language; (ii) the writer has only one language at home; (iii) the two languages in (i) and (ii) are the same as the native language of the subcorpus to which the essay belongs3.

Native language # of essays # of tokens

(2005) work on native language identification and show that machine learning-based methods are effective.

Related to this, other researchers (Koppel and Ordan, 2011; van Halteren, 2008) show that machine learning-based methods can also predict the source language of a given translated text although it should be emphasized that it is a different task from native language identification because translation is not typically performed by nonnative speakers but rather native speakers of the target language“.

The experimental results show that n-grams containing articles are predictive for identifying native languages .

This becomes important in native language identification1 which is useful for improving grammatical error correction systems (Chodorow et al., 2010) or for providing more targeted feedback to language learners.

6, this paper reveals several crucial findings that contribute to improving native language identification.

This opinion paper discusses subjective natural language problems in terms of their motivations, applications, characterizations, and implications.

Subjective natural language problems extend well beyond sentiment and opinion analysis.

Affective semantics is difficult for many automatic techniques to capture because rather than simple text-derived ‘surface’ features, it requires sophisticated, ‘deep’ natural language understanding that draws on subjective human knowledge, interpretation, and experience.

0 Nontraditional intersubjectivity Subjective natural language processing problems are generally problems of meaning or communication where so-called intersubjective agreement does not apply in the same way as in traditional tasks.

In computational linguistics and natural language processing (NLP), current efforts on subjective natural language problems are concentrated on the vibrant field of opinion mining and sentiment analysis (Liu, 2010; Tackstrom, 2009), and ACL-HLT 2011 lists Sentiment Analysis, Opinion Mining and Text Classification as a subject area.

The purpose of this opinion paper is not to provide a survey of subjective natural language prob-

Rather, it intends to launch discussions about how subjective natural language problems have a vital role to play in computational linguistics and in shaping fundamental questions in the field for the future.

Subjective natural language processing problems represent exciting frontier areas that directly relate to advances in artificial natural language behavior, improved intelligent access to information, and more agreeable and comfortable language-based human-computer interaction.

From a practical, application-oriented point of View, dedicating more resources and efforts to subjective natural language problems is a natural step, given the wealth of available written, spoken or multimodal texts and information associated with creativity, socializing, and subtle interpretation.

Learning a semantic lexicon is often an important first step in building a system that learns to interpret the meaning of natural language .

Formally, the system is given training data in the form: {(el,a1,w1),(eg,a2,w2),...,(en,an,wn)}, where 67; is a written natural language instruction, at, is an observed action sequence, and w, is a description of the virtual world.

There are two types of data we are interested in collecting: natural language navigation instructions and follower data.

It verified that we are indeed collecting useful information and that non-experts are fully capable of training the system by demonstrating how to use natural language in relevant contexts.

Learning to understand the semantics of human languages has been one of the ultimate goals of natural language processing (NLP).

Traditional learning approaches have relied on access to parallel corpora of natural language sentences paired with their meanings (Mooney, 2007; Zettlemoyer and Collins, 2007; Lu et al., 2008; Kwiatkowski et al., 2010).

By using a MRG that correlates better to the structure of natural language , we further improve the performance on the navigation task.

KRISP learns string-kernel classifiers that maps natural language substrings to MRG production rules.

Consequently, it is important that the production rules in the MRG mirror the structure of natural language (Kate, 2008).

While these rules are quite expressive, they often do not correspond well to any words or phrases in natural language .

We use a Bayesian generative model to capture relevant natural language phenomena and translate the English specification into a specification tree, which is then translated into a C++ input parser.

The general problem of translating natural language specifications into executable code has been around since the field of computer science was founded.

Figure 1: An example of (a) one natural language specification describing program input data; (b) the corresponding specification tree representing the program input structure; and (c) two input examples

Recent advances in this area include the successful translation of natural language commands into database queries (Wong and Mooney, 2007; Zettlemoyer and Collins, 2009; Poon and Domingos, 2009; Liang et al., 2011) and the successful mapping of natural language instructions into Windows command sequences (Branavan et al., 2009; Branavan et al., 2010).

Finally, it generates natural language feature observations conditioned on the hidden specification trees.

We define a range of features that capture the correspondence between the input format and its description in natural language .

NLP in Software Engineering Researchers have recently developed a number of approaches that apply natural language processing techniques to software engineering problems.

This research analyzes natural language in documentation or comments to better understand existing application programs.

Our mechanism, in contrast, automatically generates parser programs from natural language input format descriptions.

Dependency-based Compositional Semantics (DCS) is a framework of natural language semantics with easy-to-process structures as well as strict semantics.

The pursue of a logic more suitable for natural language inference is not new.

Much work has been done in mapping natural language into database queries (Cai and Yates, 2013; Kwiatkowski et al., 2013; Poon, 2013).

can thus be considered as an attempt to characterize a fragment of FOL that is suited for both natural language inference and transparent syntax-semantics mapping, through the choice of operations and relations on sets.

It is expressive enough to represent complex natural language queries on a relational database, yet simple enough to be latently learned from question-answer pairs.

In this section we describe the idea of representing natural language semantics by DCS trees, and achieving inference by computing logical relations among the corresponding abstract denotations.

DCS trees has been proposed to represent natural language semantics with a structure similar to dependency trees (Liang et al., 2011) (Figure 1).

2.4.1 Natural language to DCS trees

In this paper, we consider a new zero-shot learning task of extracting entities specified by a natural language query (in place of seeds) given only a single web page.

In our case, we only have the natural language query, which presents the more difficult problem of associating the entity class in the query (e.g., hiking trails) to concrete entities (e.g., Avalon Super Loop).

Another related line of work is information extraction from text, which relies on natural language patterns to extract categories and relations of entities.

In future work, we would like to explore the issue of compositionality in queries by aligning linguistic structures in natural language with the relative position of entities on web pages.

In this paper, we propose a novel task, zero-shot entity extraction, where the specification of the desired entities is provided as a natural language query.

In our setting, we take as input a natural language query and extract entities from a single web page.

For evaluation, we created the OPENWEB dataset comprising natural language queries from the Google Suggest API and diverse web pages returned from web search.

We define the zero-shot entity extraction task as follows: let at be a natural language query (e.g., hiking trails near Baltimore), and to be a web page.

One goal of natural language generation is to produce coherent text that presents information in a logical order.

Then, we incorporate the model enhanced with topological fields into a natural language generation system that generates constituent orders for German text, and show that the added coherence component improves performance slightly, though not statistically significantly.

Local coherence modelling has been shown to be useful for tasks like natural language generation and summarization, (Barzilay and Lee, 2004) and genre classification (Barzilay and Lapata, 2008).

We then embed these topological field annotations into a natural language generation system to show the utility of local coherence information in an applied setting.

Filippova and Strube (2007c) also examine the role of the VF in local coherence and natural language generation, focusing on the correlation between VFs and sentential topics.

Given an input utterance, we first use a simple method to deterministically generate a set of candidate logical forms with a canonical realization in natural language for each.

Utterance generation While mapping general language utterances to logical forms is hard, we observe that it is much easier to generate a canonical natural language utterances of our choice given a logical form.

use a KB over natural language extractions rather than a formal KB and so querying the KB does not require a generation step — they paraphrase questions to KB entries directly.

We consider the semantic parsing problem of mapping natural language utterances into logical forms to be executed on a knowledge base (KB) (Zelle and Mooney, 1996; Zettlemoyer and Collins, 2005; Wong and Mooney, 2007; Kwiatkowski et al., 2010).

Semantic parsers need to somehow associate natural language phrases with logical predicates, e.g., they must learn that the constructions “What

To learn these map-tings, traditional semantic parsers use data which airs natural language with the KB.

date logical forms Zac, and then for each 2 E 236 generate a small set of canonical natural language utterances Cz.

Second, natural language utterances often do not express predicates explicitly, e. g., the question “What is Italy’s money?” expresses the binary predicate CurrencyOf with a possessive construction.

Our framework accommodates any paraphrasing method, and in this paper we propose an association model that learns to associate natural language phrases that co-occur frequently in a monolingual parallel corpus, combined with a vector space model, which learns to score the similarity between vector representations of natural language utterances (Section 5).

This paper presents and evaluates several original techniques for the latent classification of biographic attributes such as gender, age and native language , in diverse genres (conversation transcripts, email) and languages (Arabic, English).

(including true speaker gender, age, native language , etc.)

Corpus details for Age and Native Language : For age, we used the same training and test speakers from Fisher corpus as explained for gender in section 3 and binarized into greater-than or less-than-or-equal-to 40 for more parallel binary evaluation.

Based on the prior distribution, always guessing the most likely class for age ( age less-than-or—equal-to 40) results in 62.59% accuracy and always guessing the most likely class for native language (nonnative) yields 50.59% accuracy.

Speaker attributes such as gender, age, dialect, native language and educational level may be (a) stated overtly in metadata, (b) derivable indirectly from metadata such as a speaker’s phone number or userid, or (c) derivable from acoustic properties of the speaker, including pitch and f0 contours (Bocklet et al., 2008).

In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and

Computational Natural Language Learning, pages 92—101.

Convolution Kernels for Natural Language .

The ability of recognizing such relations between text units including identifying and classifying provides important information to other natural language processing systems, such as language genenuknn docunnnu sunnnafizafion, and question answering.

In particular, more research is needed to handle more complex matches between database and textual relations, and to handle more complex natural language queries.

Semantic parsing is the task of translating natural language utterances to a formal meaning representation language (Chen et al., 2010; Liang et al., 2009; Clarke et al., 2010; Liang et al., 2011; Artzi and Zettlemoyer, 2011).

Two existing systems translate between natural language questions and database queries over large-scale databases.

(2012) report on a system for translating natural language queries to SPARQL queries over the Yago2 (Hoffart et al., 2013) database.

The manual extraction patterns predefine a link between natural language terms and Yago2 relations.

The textual schema matching task is to identify natural language words and phrases that correspond with each relation and entity in a fixed schema for a relational database.

The problem would be greatly simplified if M were a 1-1 function, but in practice most database relations can be referred to in many ways by natural language users: for instance, f i lm_actor can be referenced by the English verbs “played,” “acted,” and “starred,” along with morphological variants of them.

Problem Our goal is to learn a function that will map a natural language question cc to a query 2 over a database D. The database D is a collection of assertions in the form r(el, 62) where 7“ is a bi-

The lexicon L associates natural language patterns to database concepts, thereby defining the space of queries that can be derived from the input question (see Table 2).

To answer questions, we must find the best query for a given natural language question.

To define the space of possible queries, PARALEX uses a lexicon L that encodes mappings from natural language to database concepts (entities, relations, and queries).

Our work builds upon two major threads of research in natural language processing: information extraction (IE), and natural language interfaces to databases (NLIDB).

While much progress has been made in converting text into structured knowledge, there has been little work on answering natural language questions over these databases.

However, we use a paraphrase corpus for extracting lexical items relating natural language patterns to database concepts, as opposed to relationships between pairs of natural language utterances.

Eventually, we hope to extend the techniques to synthesize even more complex structures, such as computer programs, from natural language .

As the questions are posted to a web forum, the posts often contained additional comments which were not part of the word problems and the solutions are embedded in long freeform natural language descriptions.

This allows for a tighter mapping between the natural language and the system template, where the words aligned to the first equation in the template come from the first two sentences, and the words aligned to the second equation come from the third.

Document level features Oftentimes the natural language in ac will contain words or phrases which are indicative of a certain template, but are not associated with any of the words aligned to slots in the template.

Single Slot Features The natural language cc always contains one or more questions or commands indicating the queried quantities.

Situated Semantic Interpretation There is a large body of research on learning to map natural language to formal meaning representations, given varied forms of supervision.

Answering natural language questions using the Freebase knowledge base has recently been explored as a platform for advancing the state of the art in open domain semantic parsing.

One challenge for natural language querying against a KB is the relative informality of queries as compared to the grammar of a KB.

To compensate for the problem of domain mismatch or overfitting, we exploited ClueWeb, mined mappings between KB relations and natural language text, and showed that it helped both relation prediction and answer extraction.

However, most Freebase relations are framed in a way that is not commonly addressed in natural language questions.

Question answering (QA) from a knowledge base (KB) has a long history within natural language processing, going back to the 1960s and 1970s, with systems such as Baseball (Green Jr et al., 1961) and Lunar (Woods, 1977).

These systems were limited to closed-domains due to a lack of knowledge resources, computing power, and ability to robustly understand natural language .

We present a system that learns to follow navigational natural language directions.

We presented a reinforcement learning system which learns to interpret natural language directions.

While our results are still preliminary, we believe our model represents a significant advance in learning natural language meaning, drawing its supervision from human demonstration rather than word distributions or hand-labeled semantic tags.

Learning exactly which words influence decision making is difficult; reinforcement learning algorithms have problems with the large, sparse feature vectors common in natural language processing.

However, they do not learn these representations from text, leaving natural language processing as an open problem.

Additionally, the instruction giver has a path drawn on her map, and must communicate this path to the instruction follower in natural language .

However, in natural language , some facts are implicit, and identifying them requires “reading between the lines”.

It involves learning uncertain commonsense knowledge (in the form of probabilistic first-order rules) from natural language text by mining a large corpus of automatically extracted facts.

We have introduced a novel approach using Bayesian Logic Programs to learn to infer implicit information from facts extracted from natural language text.

To the best of our knowledge, this is the first paper that employs BLPs for inferring implicit information from natural language text.

We demonstrate that it is possible to learn the structure and the parameters of BLPs automatically using only noisy extractions from natural language text, which we then use to infer additional facts from text.

In our task, the supervised training data consists of facts that are extracted from the natural language text.

Potential applications include learnable models for aspects of natural language and cognition.

For theoretical linguistics, it appears that the string extension function f = (LR13,P2), which defines a class of languages which obey restrictions on both contiguous subsequences of length 3 and on discontiguous subsequences of length 2, provides a good first approximation to the segmental phonotactic patterns in natural languages (Heinz, 2007).

Finally, since the stochastic counterpart of k:-SL class is the n-gram model, it is plausible that probabilistic string extension language classes can form the basis of new natural language processing techniques.

One notable case is the Strictly Piecewise (SP) languages, which was originally motivated for two reasons: the leamability properties discussed here and its ability to describe long-distance dependencies in natural language phonology (Heinz, 2007; Heinz, to appear).

Another example is the Strictly Local (SL) languages which are the categorical, symbolic version of n-gram models, which are widely used in natural language processing (Jurafsky and Martin, 2008).

Heinz (2007,2009a) shows that consonantal harmony patterns in natural language are describable by such SP2 languages and hypothesizes that humans learn them in the way suggested by $3122.

We will apply our proposed R2NN to other tree structure learning tasks, such as natural language parsing.

Applying DNN to natural language processing (NLP), representation or embedding of words is usually learnt first.

Recursive neural networks, which have the ability to generate a tree structured output, are applied to natural language parsing (Socher et al., 2011), and they are extended to recursive neural tensor networks to explore the compositional aspect of semantics (Socher et al., 2013).

To generate a tree structure, recursive neural networks are introduced for natural language parsing (Socher et al., 2011).

For example, for nature language parsing, sum] is the representation of the parent node, which could be a NP or VP node, and it is also the representation of the whole subtree covering from Z to n .

We present a novel hierarchical prior structure for supervised transfer learning in named entity recognition, motivated by the common structure of feature spaces for this task across natural language data sets.

In particular, we develop a novel prior for named entity recognition that exploits the hierarchical feature space often found in natural language domains (§l.2) and allows for the transfer of information from labeled datasets in other domains (§l.3).

Representing feature spaces with this kind of tree, besides often coinciding with the explicit language used by common natural language toolkits (Cohen, 2004), has the added benefit of allowing a model to easily back-off, or smooth, to decreasing levels of specificity.

We used a standard natural language toolkit (Cohen, 2004) to compute tens of thousands of binary features on each of these tokens, encoding such information as capitalization patterns and contextual information from surrounding words.

In this work, we will base our work on Conditional Random Fields (CRF’s) (Lafferty et al., 2001), which are now one of the most preferred sequential models for many natural language processing tasks.

We describe a joint model for understanding user actions in natural language utterances.

We inject information extracted from unstructured web search query logs as prior information to enhance the generative process of the natural language utterance understanding model.

However data sources in VPA systems pose new challenges, such as variability and ambiguities in natural language , or short utterances that rarely contain contextual information, etc.

Experimental results using the new Bayesian model indicate that we can effectively learn and discover meta-aspects in natural language utterances, outperforming the supervised baselines, especially when there are fewer labeled and more unlabeled utterances.

The contributions of this paper are as follows: (i) construction of a novel Bayesian framework for semantic parsing of natural language (NL) utterances in a unifying framework in §4, (ii) representation of seed labeled data and information from web queries as informative prior to design a novel utterance understanding model in §3 & §4, (iii) comparison of our results to supervised sequential and joint learning methods on NL utterances in §5.

Experimental results verify the effectiveness of our approach for both short keyword queries, and verbose natural language queries.

Automatic markup of textual documents with linguistic annotations such as part-of-speech tags, sentence constituents, named entities, or semantic roles is a common practice in natural language processing (NLP).

Instead of just focusing our attention on keyword queries, as is often done in previous work (Barr et al., 2008; Bergsma and Wang, 2007; Tan and Peng, 2008; Guo et al., 2008), we also explore the performance of our annotations with more complex natural language search queries such as verbal phrases and wh-questions, which often pose a challenge for IR applications (Bendersky et al., 2010; Kumaran and Allan, 2007; Kumaran and Carvalho, 2009; Lease, 2007).

Instead, we are interested in annotation of queries of different types, including verbose natural language queries.

An additional research area which is relevant to this paper is the work on joint structure modeling (Finkel and Manning, 2009; Toutanova et al., 2008) and stacked classification (Nivre and McDonald, 2008; Martins et al., 2008) in natural language processing.

Our proposal performs distant supervised learning to extract a set of relations from a natural language corpus, and anchors each of them to an interval of temporal validity, aggregating evidence from documents supporting the relation.

This paper introduces the problem of extracting, from unrestricted natural language text, relational knowledge anchored to a temporal span, aggregating temporal evidence from a collection of documents.

The Temporally anchored relation extraction problem consists in, given a natural language text document corpus, C, a target entity, 6, and a target

their relation to events in natural language , the complete problem of temporally anchored relation extraction remains relatively unexplored.

This sharp temporal interval fails to capture the imprecision of temporal boundaries conveyed in natural language text.

Semantic parsing is the problem of deriving a structured meaning representation from a natural language utterance.

We have presented a semantic parser which uses techniques from machine translation to learn mappings from natural language to variable-free meaning representations.

Semantic parsing (SP) is the problem of transforming a natural language (NL) utterance into a machine-interpretable meaning representation (MR).

cityid, which in some training examples is unary) to align with different natural language strings depending on context.

Other work which generalizes from variable-free meaning representations to A-calculus expressions includes the natural language generation procedure described by Lu and Ng (2011).

Similar studies in parsing natural language sen-

Context-free phrase structure grammars are widely used for parsing natural language .

There are however natural languages that are free word order.

Although very intuitive, ID/LP rules are not widely used in the area of natural language processing.

We have proposed STRUCT, a general-purpose natural language generation system which is comparable to current state-of-the-art generators.

Natural language generation (NLG) develops techniques to extend similar capabilities to automated systems.

to Natural Language Generation

This is then used by a surface realization module which encodes the enriched semantic representation into natural language .

In the MDP we use for NLG, we must define each element of the tuple in such a way that a plan in the MDP becomes a sentence in a natural language .

Second, the measure makes sense theoretically, both from algorithmic and native language acquisition points of view.

We also make an interesting observation that the impressionistic evaluation of syntactic complexity is better approximated by the presence or absence of grammar and usage patterns (and not by their frequency of occurrence), an idea supported by studies in native language acquisition.

Studies in native language acquisition, have considered multiple grammatical developmental indices that represent the grammatical levels reached at various stages of language acquisition.

0 In the domain of native language acquisition, the presence or absence of a grammatical structure indicates grammatical development.

The inductive classifier we use here is the maximum-entropy model (MaxEnt) which has been used to solve several statistical natural language processing problems with much success (Berger et al., 1996; Borthwick et al., 1998; Borthwick, 1999; Pang et al., 2002; Klein et al., 2003; Rosenfeld, 2005).

In this paper, we present a reinforcement learning approach for mapping natural language instructions to sequences of executable actions.

The problem of interpreting instructions written in natural language has been widely studied since the early days of artificial intelligence (Winograd, 1972; Di Eugenio, 1992).

This form of supervision allows us to learn interpretations of natural language instructions when standard supervised techniques are not applicable, due to the lack of human-created annotations.

Our formulation is unique in how it represents natural language in the reinforcement learning framework.

These systems converse with a human user by taking actions that emit natural language utterances.

Finding concepts in natural language utterances is a challenging task, especially given the scarcity of labeled data for learning semantic ambiguity.

Semantic tagging is used in natural language understanding (NLU) to recognize words of semantic importance in an utterance, such as entities.

Our SSL approach uses probabilistic clustering method tailored for tagging natural language utterances.

In (Subramanya et al., 2010), a new SSL method is described for adapting syntactic POS tagging of sentences in newswire articles along with search queries to a target domain of natural language (NL) questions.

As our input consists of natural language attributes, the model would infer textual attributes given visual attributes and vice versa.

The two modalities are encoded as vectors of natural language attributes and are obtained automatically from decoupled text and image data.

Recent years have seen a surge of interest in single word vector spaces (Turney and Pantel, 2010; Collobert et al., 2011; Mikolov et al., 2013) and their successful use in many natural language applications.

The visual and textual modalities on which our model is trained are decoupled in that they are not derived from the same corpus (we would expect co-occurring images and text to correlate to some extent) but unified in their representation by natural language attributes.

Natural language is diverse.

Information extraction projects aim to distill relational facts from natural language text (Auer et al., 2007; Bollacker et al,2008;(knlynietal,2010;Faderetal,2011; Nakashole et al., 2011; Del Corro and Gemulla, 2013).

However, such scores are often tied to the extractor’s ability to read and understand natural language text.

The focus is on understanding natural language , including the use of negation.

Negative expressions are common in natural language text and play a critical role in information extraction.

Negation expressions are common in natural language text.

Horn, 1989; van der Wouden, 1997), and there were only a few in natural language processing with focus on negation recognition in the biomedical domain.

Due to the increasing demand on deep understanding of natural language text, negation recognition has been drawing more and more attention in recent years, with a series of shared tasks and workshops, however, with focus on cue detection and scope resolution, such as the BioNLP 2009 shared task for negative event detection (Kim et al., 2009) and the ACL 2010 Workshop for scope resolution of negation and speculation (Morante and Sporleder, 2010), followed by a special issue of Computational Linguistics (Morante and Sporleder, 2012) for modality and negation.

We describe all the linguistic resources and natural language processing techniques that we have exploited for this task.

naming agencies and naive generators) that can be used for obtaining name suggestions, we propose a system which combines several linguistic resources and natural language processing (NLP) techniques to generate creative names, more specifically neologisms based on homophonic puns and metaphors.

Accordingly, we have made a systematic attempt to replicate these processes, and implemented a system which combines methods and resources used in various areas of Natural Language Processing (NLP) to create neologisms based on homophonic puns and metaphors.

This resource consists of nodes representing concepts which are in the form of words or short phrases of natural language , and labeled relations between them.

We present a holistic data—driven approach to image description generation, exploiting the vast amount of (noisy) parallel image data and associated natural language descriptions available on the web.

Automatically describing images in natural language is an intriguing, but complex AI task, requiring accurate computational visual recognition, comprehensive world knowledge, and natural language generation.

By judiciously exploiting the correspondence between image content elements and phrases, it is possible to generate natural language descriptions that are substantially richer in content and more linguistically interesting than previous work.

For a query image, we retrieve relevant candidate natural language phrases by visually comparing the query image to database images from the SBU Captioned Photo Collection (Ordonez et al., 2011) (1 million photographs with associated human-composed descriptions).

Our unsupervised model brings together familiar components in natural language processing (like parsers and topic models) with contextual political information—temporal and dyad dependence—to infer latent event classes.

(This highlights how better natural language processing could help the model, and the dangers of false positives for this type of data analysis, especially in small-sample drilldowns.)

6.2 Events in Natural Language Processing

Political event extraction from news has also received considerable attention within natural language processing in part due to government-funded challenges such as MUC-3 and MUC-4 (Lehnert, 1994), which focused on the extraction of terrorist events, as well as the more recent ACE program.

Natural language parsing has typically been done with small sets of discrete categories such as NP and VP, but this representation does not capture the full syntactic nor semantic richness of linguistic phrases, and attempts to improve on this by lexicalizing phrases or splitting categories only partly address the problem at the cost of huge feature spaces and sparseness.

Syntactic parsing is a central task in natural language processing because of its importance in mediating between linguistic expression and meaning.

In many natural language systems, single words and n-grams are usefully described by their distributional similarities (Brown et al., 1992), among many others.

However, their model is lacking in that it cannot represent the recursive structure inherent in natural language .

Semantic parsing is the problem of mapping natural language strings into meaning representations.

Although it does not encode quantifiers, tense, or modality, the set of semantic phenomena included in AMR were selected with natural language applications—in particular, machine translation—in mind.

While all semantic parsers aim to transform natural language text to a formal representation of its meaning, there is wide variation in the meaning representations and parsing techniques used.

Natural language interfaces for querying databases have served as another driving application (Zelle and Mooney, 1996; Kate et al., 2005; Liang et al., 2011, inter alia).

Bayesian inference methods have become popular in natural language processing (Goldwater and Grif-fiths, 2007; Finkel et al., 2005; Blunsom et al., 2009; Chiang et al., 2010).

A common phenomenon observed while modeling natural language problems is sparsity.

We use natural language processing techniques to attack letter substitution ciphers.

In a letter substitution cipher, every letter p in the natural language (plaintext) sequence is replaced by a cipher token 0, according to some substitution key.

This paper investigates the class of Tree-Tuple MCTAG with Shared Nodes, TT-MCTAG for short, an extension of Tree Adjoining Grammars that has been proposed for natural language processing, in particular for dealing with discontinuities and word order variation in languages such as German.

Some others generate only polynomial languages but their generative capacity is too limited to deal with all natural language phenomena.

However, from a first inspection of the MCTAG analyses proposed for natural languages (see Chen-Main and Joshi (2007) for an overview), it seems that there are no important natural language phenomena that can be described by LCFRS and not by TT-MCTAG.

As a result, one obtains a slight degree of locality that can be exploited for natural language phenomena that are unbounded only in a limited domain.

We built a system that interprets natural language utterances much more accurately than existing systems, despite using no annotated logical forms.

Think of DCS as a higher-level programming language tailored to natural language , which results in programs (DCS trees) which are much simpler than the logically-equivalent lambda calculus formulae.

The integration of natural language with denotations computed against a world (grounding) is becoming increasingly popular.

We now turn to the task of mapping natural language utterances to DCS trees.

These findings have important implications for search results presentation, especially for natural language queries.

There are a disproportionally large number of natural language queries in this set compared with query sets from typical keyword engines.

Such queries are often complete questions and are sometimes grammatical fragments (e.g., “date of next US election”) and so are likely to be amenable to interesting natural language processing algorithms, which is an area of in-

This is substantially longer than the current average for web search query, which was approximately 2.8 in 2005 (Jansen et al., 2007); this is due to the existence of natural language queries.

It is a well-known fact that natural language is highly ambiguous: a correct and seemingly unambiguous sentence may have an enormous number of readings.

It has repeatedly been pointed out that N-grams model natural language only superficially: an Nth-order Markov chain is a very crude model of the complex dependencies between words in an utterance.

More accurate statistical models of natural language have mainly been developed in the field of statistical parsing, e.g.

On the other hand, they are not suited to reliably decide on the grammaticality of a given phrase, as they do not accurately model the linguistic constraints inherent in natural language .

Much of the work in the field of Natural Language Processing is founded on an assumption of semantic compositionality: that there are identifiable, separable components of an unspecified inference process that will develop as research in NLP progresses.

While many have (nearly) immediate application to real world tasks like search, many are also motivated by their potential contribution to more ambitious Natural Language tasks.

But there is no clear process for identifying potential tasks (other than consensus by a sufficient number of researchers), nor for quantifying their potential contribution to existing NLP tasks, let alone to Natural Language Understanding.

For natural language words and

Over the past decade, supervised learning algorithms have gained widespread acceptance in natural language processing (NLP).

The long-tailed distribution of natural language words implies that most of the word types will be either unseen or seen very few times in the labeled training data, even if the data set is a relatively large one (e. g., the Penn Treebank).

Out of context, natural language words are often ambiguous.

Of course, at the end of the day, the issue that is more relevant to computational linguistics than a formalism’s ability to generate artificial languages such as L3 is how useful it is for modeling natural languages .

In this sense, our formal result can also be understood as a contribution to a discussion about the expressive power that is needed to model natural languages .

It is well-known that CCG can generate languages that are not context-free (which is necessary to capture natural languages ), but can still be parsed in polynomial time.

On the other hand, as pure multi-modal CCG has been successfully applied to model the syntax of a variety of natural languages, another way to read our results is as contributions to a discussion about the exact expressiveness needed to model natural language .

Given an image I , and a related knowledge database K, create a natural language description C which captures the main content of the image under K. Specifically, in the news story scenario, we will generate a caption C for an image I and its accompanying document D. The training data thus consists of document-image-caption tu-

The picture is first analyzed using image processing techniques into an abstract representation, which is then rendered into a natural language description with a text generation engine.

They extract features of human motion and interleave them with a concept hierarchy of actions to create a case frame from which a natural language sentence is generated.

Within natural language processing most previous efforts have focused on generating captions to accompany complex graphical presentations (Mittal et al., 1998; Corio and Lapalme, 1999; Fas-ciano and Lapalme, 2000; Feiner and McKeown, 1990) or on using the captions accompanying information graphics to infer their intended message, e.g., the author’s goal to convey ostensible increase or decrease of a quantity of interest (Elzer et al., 2005).

Several constraints on non-projective dependency structures have been proposed recently that seek a good balance between parsing efficiency and coverage of non-projective phenomena present in natural language treebanks.

However, we have found this not to be a problem when measuring multiplanarity in natural language treebanks, since the effective problem size can be reduced by noting that each connected component of the crossings graph can be treated separately, and that nodes that are not part of a cycle need not be considered.5 Given that non-projective sentences in natural language tend to have a small proportion of non-projective links (Nivre and Nilsson, 2005), the connected components of their crossings graphs are very small, and k-colourings for them can quickly be found by brute-force search.

Dependency-based syntactic parsing has become a widely used technique in natural language processing, and many different parsing models have been proposed in recent years (Yamada and Matsumoto, 2003; Nivre et al., 2004; McDonald et al., 2005a; Titov and Henderson, 2007; Martins et al., 2009).

Like context-free grammars, projective dependency trees are not sufficient to represent all the linguistic phenomena observed in natural languages , but they have the advantage of being efficiently parsable: their parsing problem can be solved in cubic time with chart parsing techniques (Eisner, 1996; Gomez-Rodriguez et al., 2008), while in the case of general non-projective dependency forests, it is only tractable under strong independence assumptions (McDonald et al., 2005b; McDonald and Satta, 2007).

In various natural language processing tasks, relevant statements appearing in a speculative context are treated as false positives.

(Hyland, 1994)), speculative language from a Natural Language Processing perspective has only been studied in the past few years.

What makes this iterative method efficient is that, as we said earlier, hedging is expressed via keywords in natural language texts; and often several keywords are present in a single sentence.

We present a new approach to learning a semantic parser (a system that maps natural language sentences into logical form).

The resulting system produces improved results on standard corpora on natural language interfaces for database querying and simulated robot control.

Semantic parsing is the task of mapping a natural language (NL) sentence into a completely formal meaning representation (MR) or logical form.

Learning natural language in an unsupervised way commonly involves the expectation-maximization (EM) algorithm to optimize the parameters of a generative model, often a probabilistic grammar (Pereira and Schabes, 1992).

Later approaches include variational EM in a Bayesian setting (Beal and Gharamani, 2003), which has been shown to obtain even better results for various natural language tasks over EM (e.g., Cohen et al., 2008).

For example, Smith and Eisner (2006) have penalized the approximate posterior over dependency structures in a natural language grammar induction task to avoid long range dependencies between words.

In all these approaches, grammar and lexicon are developed manually and it is assumed that the lexicon associates semantic sub-formulae with natural language expressions.

As discussed in (Power and Third, 2010), one important limitation of these approaches is that they assume a simple deterministic mapping between knowledge representation languages and some controlled natural language (CNL).

(Lu and Ng, 2011) focuses on generating natural language sentences from logical form (i.e., lambda terms) using a synchronous context-free grammar.

Through an investigation of Category-2 non-English queries, we find out that this is mostly due to the usage of some common internet or computer terms such as ”download”, ”software”, ”flash player”, among other native language query terms.

The language identification problem refers to the task of deciding in which natural language a given text is written.

Although the problem is heavily studied by the Natural Language Processing community, most of the research carried out to date has been concerned with relatively long texts such as articles or web pages which usually contain enough text for the systems built for this task to reach almost perfect accuracy.

Using textual entailment analysis, we obtain entailment scores between a natural language question posed by the user and the candidate sentences returned from search engine.

Open domain natural language question answering (QA) is a process of automatically finding answers to questions searching collections of text files.

Recent research indicates that using labeled and unlabeled data in semi-supervised learning (SSL) environment, with an emphasis on graph-based methods, can improve the performance of information extraction from data for tasks such as question classification (Tri et al., 2006), web classification (Liu et al., 2006), relation extraction (Chen et al., 2006), passage-retrieval (Otterbacher et al., 2009), various natural language processing tasks such as part-of-speech tagging, and named-entity recognition (Suzuki and Isozaki, 2008), word-sense disam-

Another direction is to apply the semi-supervised algorithm to other natural language problems, such as machine translation, topic segmentation and chunking.

Unfortunately, although significant recent progress has been made in the area of semi-supervised learning, the performance of semi-supervised learning algorithms still fall far short of expectations, particularly in challenging real-world tasks such as natural language parsing or machine translation.

plied to several natural language processing tasks (Yarowsky, 1995; Charniak, 1997; Steedman et al., 2003).

Two of the fundamental components of a natural language communication are word sense discovery (Jones, 1986) and word sense disambiguation (Ide and Veronis, 1998).

two aspects are not only important from the perspective of developing computer applications for natural languages but also form the key components of language evolution and change.

Word sense disambiguation as well as word sense discovery have both remained key areas of research right from the very early initiatives in natural language processing research.

Recent years have seen increased interest in grounded language acquisition, where the goal is to extract representations of the meaning of natural language tied to the physical world.

The language grounding problem has assumed seVeral guises in the literature such as semantic parsing (Zelle and Mooney, 1996; Zettlemoyer and Collins, 2005; Kate and Mooney, 2007; Lu et al., 2008; Bo'rschinger et al., 2011), mapping natural language instructions to executable actions (Branavan et al., 2009; Tellex et al., 2011), associating simplified language to perceptual data such as images or Video (Siskind, 2001; Roy and Pent-land, 2002; Gorniak and Roy, 2004; Yu and Ballard, 2007), and learning the meaning of words based on linguistic and perceptual input (Bruni et al., 2012b; Feng and Lapata, 2010; Johns and Jones, 2012; Andrews et al., 2009; Silberer and Lapata, 2012).

Since our goal is to develop distributional models that are applicable to many words, it contains a considerably larger number of concepts (i.e., more than 500) and attributes (i.e., 412) based on a detailed taxonomy which we argue is cognitively plausible and beneficial for image and natural language processing tasks.

Given a pair of entities (A,B) in S, the first step is to express the relation between A and B with some feature representation using a feature extraction scheme x. Lexical or syntactic patterns have been successfully used in numerous natural language processing tasks, including relation extraction.

Each node is augmented with relevant part-of-speech (POS) using the Python Natural Language Processing Tool Kit.

Because not-a-relation is a background or default relation type in the relation classification task, and because it has rather high variation when manifested in natural language , we have found it difficult to obtain a distance metric W that allows the not-a-relation samples to form clusters naturally using transductive inference.

This underlying intuition, adopted from formal semantics of natural language , motivated the creation of the lexical function model of composition (lf) (Baroni and Zamparelli, 2010; Co-ecke et al., 2010).

The lf model can be seen as a projection of the symbolic Montagovian approach to semantic composition in natural language onto the domain of vector spaces and linear operations on them (Baroni et al., 2013).

The full range of semantic types required for natural language processing, including those of adverbs and transitive verbs, has to include, however, tensors of greater rank.

We present a new method for generating linguistic variation projecting multiple personality traits continuously, by combining and extending previous research in statistical natural language generation (Paiva and Evans, 2005; Rambow et al., 2001; Isard et al., 2006; Mairesse and Walker, 2007).

Over the last 20 years, statistical language models (SLMs) have been used successfully in many tasks in natural language processing, and the data available for modeling has steadily grown (Lapata and Keller, 2005).

Langkilde and Knight (1998) first applied SLMs to statistical natural language generation (SNLG), showing that high quality paraphrases can be generated from an underspecified representation of meaning, by first applying a very undercon-strained, rule-based overgeneration phase, whose outputs are then ranked by an SLM scoring phase.

The EBDM framework is a simple and powerful approach to rapidly develop natural language interfaces for multi-domain dialog processing (Lee et al., 2006b).

Since the performance in human language technologies such as Automatic Speech Recognition (ASR) and Natural Language Understanding (NLU)1 have been improved, this advance has made it possible to develop spoken dialog systems for many different application domains.

1Through this paper, we will use the term natural language to include both spoken language and written language

We describe a system capable of translating native language (L1) fragments to foreign language (L2) fragments in an L2 context.

The type of translation assistance system under investigation here encourages language learners to write in their target language while allowing them to fall back to their native language in case the correct word or expression is not known.

Whereas machine translation generally concerns the translation of whole sentences or texts from one language to the other, this study focusses on the translation of native language (henceforth L1) words and phrases, i.e.

This hypothesis states that natural languages tend to use consistent spoken and written styles.

hypothesis assumes that in natural languages , there is a tendency for an author to be consistent in the same discourse or article.

The proposed system, preserves its portability between languages and domains because it does not use any natural language processing (NLP) subsystem (e.g.

Formalisms like HPSG (Pollard and Sag, 1994), LFG (Kaplan and Bresnan, 1982), and CCG (Steedman, 2000) are linguistically motivated in the sense that they attempt to explain and predict the limited variation found in the grammars of natural languages .

Research in natural language understanding is driven by the datasets that we have available.

Progress in natural language processing relies on direct comparison on shared data, discouraging improvements to the evaluation data.

This phenomenon has been observed in several natural language synthesis tasks such as generation and summarization, in which a single gold standard is inadequate to fully assess performance.

This notion of preferential ordering of discourse relations is observed in natural language in general,

This task, discourse parsing, has been a recent focus of study in the natural language processing (NLP) community, largely enabled by the availability of large-scale discourse annotated corpora (Wellner and Pustejovsky, 2007; Elwell and Baldridge, 2008; Lin et al., 2009; Pitler et al., 2009; Pitler and Nenkova, 2009; Lin et al., 2010; Wang et al., 2010).

Understanding the meaning of text is a long term goal in the natural language processing community.

Within natural language processing, negation has drawn attention mainly in sentiment analysis (Wilson et al., 2009; Wiegand et al., 2010) and the biomedical domain.

None of the above references aim at detecting or annotating the focus of negation in natural language .

cQA websites are becoming an increasingly popular complement to search engines: overnight, a user can expect a human-crafted, natural language answer tailored to her specific needs.

(2009) with a system that makes use of semantic-aware Natural Language Preprocessing techniques.

BEs are a strong theoretical instrument to tackle the ambiguity inherent in natural language that find successful practical applications in real-world query-based summarization systems.

and Collins (2007) instead, which combines the utterances of a single user in one scenario and contains 5,426 scenarios in total; each scenario corresponds to a (manually annotated) formal meaning representation (it-expression) and its translation in natural language .

Here, the records provide a structured representation of the flight details (e. g., departure and arrival time, location), and the text renders some of this information in natural language .

Specifically, we define a probabilistic context-free grammar (PCFG) that captures the structure of the database and its correspondence to natural language .

Their system, however, does not take natural language sentences as input, but logical expressions that are representations of small discourse fragments.

natural language computation, whereby manually crafted rules gradually give way to more robust corpus-based statistical methods.

The use of metaphor is ubiquitous in natural language text and it is a serious bottleneck in automatic text understanding.

Note that we only consider uniform membership, since grammars for natural languages are typically considerably larger than input sentences, and their influence can hardly be neglected.

A conclusion with a more immediate linguistic value is that MLIGs and UVG—dls hardly qualify as formalisms for mildly context-sensitive languages, claimed by Joshi (1985) to be adequate for modeling natural languages , and “roughly” defined as the extensions of context-free languages that display

Natural languages are known for displaying some limited cross-serial dependencies, as witnessed in linguistic analyses, e.g.

However, the study of the phenomenon in natural language is relatively nascent, and its incorporation into language technologies is almost nonexistent.

Moreover, applications of natural language processing generally lack the ability to recognize and interpret metalanguage (Anderson et al.

Applications of natural language understanding cannot process metalanguage without detecting it, especially when upstream components (such as parsers) mangle its structure.

Three graduate students in natural language processing annotate intruding paths.

Following (Yao et al., 2011), we filter out noisy documents and use natural language packages to annotate the documents, including NER tagging (Finkel et al., 2005) and dependency parsing (Nivre et al., 2004).

Here, the relation extractor simultaneously discovers facts expressed in natural language , and the ontology into which they are assigned.

Open-domain question answering (QA), which fulfills a user’s information need by outputting direct answers to natural language queries, is a challenging but important problem (Etzioni, 2011).

Due to the variety of word choices and inherent ambiguities in natural languages , bag-of-words approaches with simple surface-form word matching tend to produce brittle results with poor prediction accuracy (Bilotti et al., 2007).

While the task of question answering has a long history dated back to the dawn of artificial intelligence, early systems like STUDENT (Winograd, 1977) and LUNAR (Woods, 1973) are typically designed to demonstrate natural language understanding for a small and specific domain.

Automatically identifying the polarity of words is a very important task in Natural Language Processing.

Predicting the semantic orientation of words is a very interesting task in Natural Language Processing and it has a wide variety of applications.

Identifying emotions and attitudes from unstructured text is a very important task in Natural Language Processing.

Part-of-speech tagging is a crucial preliminary process in many natural language processing applications.

Because many words in natural languages have more than one part-of-speech tag, resolving part-of-speech ambiguity is an important task.

part-of-speech or POS tagging) is an important preprocessing step for many natural language processing applications because grammatical rules are not functions of individual words, instead, they are functions of word categories.

In recent years, research in natural language processing has increasingly focused on normalizing SMS messages.

Whatever their causes, these deviations considerably hamper any standard natural language processing (NLP) system, which stumbles against so many Out-Of-Vocabulary words.

In natural language processing, a word is commonly defined as “a sequence of alphabetic characters between separators”, and an IV word is simply a word that belongs to the lexicon in use.

We use as our corpus the 4.5 million word Intema—tional Corpus of Learner English (ICLE) (Granger et al., 2009), which consists of more than 6000 essays written by university undergraduates from 16 countries and 16 native languages who are learners of English as a Foreign Language.

Fifteen native languages are represented in the set of essays selected for annotation.

Automated essay scoring, the task of employing computer technology to evaluate and score written text, is one of the most important educational applications of natural language processing (NLP) (see Shermis and Burstein (2003) and Shermis et al.

This means that researchers do not have a common development and test set for natural language processing of learner English such as for grammatical error detection.

The availability of learner corpora is still somewhat limited despite the obvious usefulness of such data in conducting research on natural language processing of learner English in recent years.

This is one of the most active research areas in natural language processing of learner English.

Some businesses have recently allowed users to formulate queries in natural language using SMS.

These systems generally adopt one of the following three approaches: Human intervention based, Information Retrieval based, or Natural language processing based.

The natural language processing based system tries to fully parse a question to discover semantic structure and then apply logic to formulate the answer (Molla et al., 2003).

We address the challenge of generating natural language abstractive summaries for spoken meetings in a domain-independent fashion.

0 To the best of our knowledge, our system is the first fully automatic system to generate natural language abstracts for spoken meetings.

In this section, we describe surface realization, which renders the relation instances into natural language abstracts.