Let cc represent the feature vector of a candidate relation instance , and y 6 {+1, —1} represent a class label.

After data cleaning, we obtained 4290 positive instances among 48614 candidate relation instances .

In order to concentrate on the classification accuracy for the target relation type, we removed the positive instances of the auxiliary relation types from the test set, although in practice we need to extract these auxiliary relation instances using learned classifiers for these relation types.

We focus on extracting binary relation instances between two relation arguments occurring in the same sentence.

Some example relation instances and their corresponding relation types as defined by ACE can be found in Table 1.

Each pair of entities within a single sentence is considered a candidate relation instance , and the task becomes predicting whether or not each candidate is a true instance of T. We use feature-based logistic regression classifiers.

This time, every pair of entities appearing together in a sentence is considered a potential relation instance , and whenever those entities appear together, features are extracted on the sentence and added to a feature vector for that entity pair.

We refer to individual ordered pairs in this relation as ‘relation instances’ .

We use relations and relation instances from Freebase, a freely available online database of structured semantic data.

This means that 900,000 Freebase relation instances are used in training, and 900,000 are held out.

For human evaluation experiments, all 1.8 million relation instances are used in training.

For all our experiments, we only extract relation instances that do not appear in our training data, i.e., instances that are not already in Freebase.

The NIST Automatic Content Extraction (ACE) RDC 2003 and 2004 corpora, for example, include over 1,000 documents in which pairs of entities have been labeled with 5 to 7 major relation types and 23 to 24 subrelations, totaling 16,771 relation instances .

Thus whereas the supervised training paradigm uses a small labeled corpus of only 17,000 relation instances as training data, our algorithm can use much larger amounts of data: more text, more relations, and more instances.

Table 1 shows examples of relation instances extracted by our system.

Approaches based on WordNet have often only looked at the hypernym (isa) or meronym (part-of) relation (Girju et al., 2003; Snow et al., 2005), while those based on the ACE program (Doddington et al., 2004) have been restricted in their evaluation to a small number of relation instances and corpora of less than a million words.