In this paper we propose a method to automatically label multilingual data with named entity tags.

Of these, we manually annotated 91 English-Bulgarian and 79 English-Korean sentence pairs with source and target named entities as well as word-alignment links among named entities in the two languages.

The named entity annotation scheme followed has the labels GPE (Geopolitical entity), PER (Person), ORG (Organization), and DATE.

The other is that the same information might be expressed using a named entity in one language, and using a nonentity phrase in the other language (e. g. “He is from Bulgaria” versus “He is Bulgarian”).

Named Entity Recognition (NER) is a frequently needed technology in NLP applications.

Tweets represent a critical source of fresh information, in which named entities occur frequently with rich variations.

We study the problem of named entity normalization (N EN ) for tweets.

Two main challenges are the errors propagated from named entity recognition (NER) and the dearth of information in a single tweet.

As a result, the task of named entity recognition (NER) for tweets, which aims to identify mentions of rigid designators from tweets belonging to named-entity types such as persons, organizations and locations (2007), has attracted increasing research interest.

entities and then uses a distantly supervised approach based on LabeledLDA to classify named entities .

However, named entity normalization (NEN) for tweets, which transforms named entities mentioned in tweets to their unambiguous canonical forms, has not been well studied.

(2010) use Amazons Mechanical Turk service 3 and CrowdFlower 4 to annotate named entities in tweets and train a CRF model to evaluate the effectiveness of human labeling.

(2011) rebuild the NLP pipeline for tweets beginning with POS tagging, through chunking, to NER, which first exploits a CRF model to segment named entities and then uses a distantly supervised approach based on LabeledLDA to classify named entities .

Unlike this work, our work detects the boundary and type of a named entity simultaneously using sequential labeling techniques.

In Figure 2, these relations are denoted by double-line boundary, including location, person, title, date and time; every tuple of these relations maps to a named entity mention.1

Figure 3 demonstrates the correct mapping of named entity mentions to tuples, as well as tuple unification, for the example shown in Figure 1.

In addition to proper nouns ( named entity mentions) that are considered in this work, they also account for nominal and pronominal noun mentions.

Interestingly, several researchers have attempted to model label consistency and high-level relational constraints using state-of-the-art sequential models of named entity recognition (NER).

In the proposed framework we take a bottom-up approach to identifying entity mentions in text, where given a noisy set of candidate named entities , described using rich semantic and surface features, discriminative learning is applied to label these mentions.

We used a set of rules to extract candidate named entities per the types specified in Figure 2.4 The rules encode information typically used in NER, including content and contextual patterns, as well as lookups in available dictionaries (Finkel et al., 2005; Minkov et al., 2005).

recall for the named entities of type date and time is near perfect, and is estimated at 96%, 91% and 90% for location, speaker and title, respectively.

Another feature encodes the size of the most semantically detailed named entity that maps to a field; for example, the most detailed entity mention of type stime in Figure l is “3:30”, comprising of two attribute values, namely hour and minutes.

Named entity recognition.

The candidate tuples generated using this procedure are structured entities, constructed using typed named entity recognition, unification, and hierarchical assignment of field values (Figure 3).

Figure 2: A DL from iteration 5 of Yarowsky on the named entity task.

The task of Collins and Singer (1999) is named entity classification on data from New York Times text.7 The data set was preprocessed by a statistical parser (Collins, 1997) and all noun phrases that are potential named entities were extracted from the parse tree.

The test data additionally contains some noise examples which are not in the three named entity categories.

We present a statistical model for canonicalizing named entity mentions into a table whose rows represent entities and whose columns are attributes (or parts of attributes).

We collected named entity mentions from two corpora: political blogs and sports news.

Due to the large size of the corpora, we uniformly sampled a subset of documents for each corpus and ran the Stanford NER tagger (Finkel et al., 2005), which tagged named entities mentions as person, location, and organization.

We used named entity of type person from the political blogs corpus, while we are interested in person and organization entities for the sports news corpus.

tributes through transductive learning from named entity mentions with a small number of seeds (see

by a named entity recognizer, along with their con-

As a result, the model discovers parts of names—(Mrs., Michelle, Obama)—while simultaneously performing coreference resolution for named entity mentions.

In our experiments these are obtained by running a named entity recognizer.

Other features: 1) Lucene’s best ranking of the date 2) Number of times where the date is absolute in the text 3) Number of times where the date is relative (but normalized) in the text 4) Total number of keywords of the query in the title, sentence and named entities of retrieved documents 5) Number of times where the date modifies a reported speech verb or is extracted from reported speech.

(Swan and Allen, 2000) present an approach to generating graphical timelines that involves extracting clusters of noun phrases and named entities .

It also performs named entity recog-

nition (NER) of the most usual named entity categories and recognizes temporal expressions.

4.2 Named Entity Recognition

In addition to this shallow parsing method, we also use named entity recognition (NER) to identify more entities.

We use the Stanford Named Entity Recognizer (Finkel et al., 2005) for this purpose.

7) We only use named entity recognition to identify entity targets; i.e.

Although using both named entity recognition (NER) and noun phrase chunking achieves better results, it

In our work, we extract as targets frequent noun phrases and named entities that are used by two or more different discussants.

Sequential modeling has been widely used in a variety of important applications including named entity recognition and shallow parsing.

We apply 1-best and k-best sequential decoding algorithms to five NLP tagging tasks: Penn TreeBank (PTB) POS tagging, CoNLLZOOO joint POS tagging and chunking, CoNLL 2003 joint POS tagging, chunking and named entity tagging, HPSG supertag-ging (Matsuzaki et al., 2007) and a search query named entity recognition (NER) dataset.

Similarly we combine the POS tags, chunk tags, and named entity tags to form joint tags for CoNLL 2003 dataset, e.g., PRP$|I-NP|O.

Note that by such tag joining, we are able to offer different tag decodings (for example, chunking and named entity tagging) simultaneously.

For example to pursue a high recall ratio, a named entity recognition system may have to adopt k-best sequences in case the true entities are not recognized at the best one.

In Wikipedia articles, named entities were identified by anchor text linking to another article and starting with a capital letter (Yan et al., 2009).

than one named entity .

An entity is mentioned as a named entity in text.

Since two entities mentioned in a sentence do not always have a relation, we select entity pairs from a corpus when: (i) the path of the dependency parse tree between the corresponding two named entities in the sentence is no longer than 4 and (ii) the path does not contain a sentence-like boundary, such as a relative clause1 (Banko et al., 2007; Banko and Etzioni, 2008).

2If we use a standard named entity tagger, the entity types are Person, Location, and Organization.

We enforce the Named Entity type of entity and value to match a expected type, predefined for the relation.

There are three families of types: Events (verbs that describe an action, annotated with tense, polarity and aspect); standardized Time Expressions; and Named Entities , with additional annotations such as gender or age.

1Most chunks consist in one word; we join words into a chunk (and a node) in two cases: a multi—word named entity and a verb and its auxiliaries.

The processing includes dependency parsing, named entity recognition and coreference resolution, done with the Stanford CoreNLP software (Klein and Manning, 2003); and events and temporal information extraction, via the TARSQI Toolkit (Verhagen et al., 2005).

We extract dependency paths for each pair of named entities in one sentence.

To determine entity types, we link named entities to Wikipedia pages using the Wikifier (Rati-nov et al., 2011) package and extract categories from the Wikipedia page.

Many relation discovery methods rely exclusively on the notion of either shallow or syntactic patterns that appear between two named entities (B ollegala et al., 2010; Lin and Pantel, 2001).

(2004) cluster pairs of named entities according to the similarity of context words intervening between them.

Named entities are important to document dating due to the nature of people and places coming in and out of the news at precise moments in time.

Named Entities : Although not directly related to time expressions, we also include n— grams of tokens that are labeled by an NER system using Person, Organization, or Location.

Collecting named entity mentions will differentiate between an article discussing a bill and one discussing the US President, Bill Clinton.