Abstract | Most current event extraction systems rely on local information at the phrase or sentence level. |
Conclusion and Future Work | Experiments show that document-level information can improve the performance of a sentence-level baseline event extraction system . |
Cross-event Approach | Our event extraction system is a two-pass system where the sentence-level system is first applied to make decisions based on local information. |
Introduction | Most current event extraction systems are based on phrase or sentence level extraction. |
Introduction | Several recent studies use high-level information to aid local event extraction systems . |
Introduction | We extend these approaches by introducing cross-event information to enhance the performance of multi-event-type extraction systems . |
Related Work | Almost all the current ACE event extraction systems focus on processing one sentence at a time (Grishman et al., 2005; Ahn, 2006; Hardy et al. |
Related Work | They used this technique to augment an information extraction system with long-distance dependency models, enforcing label consistency and extraction template consistency constraints. |
Data | To train the extractive system described in Section 2, we use as our labels y* the extractions with the largest bigram recall values relative to the sets of references. |
Experiments | But, importantly, the gains achieved by the joint extractive and compressive system in content-based metrics do not come at the cost of linguistic quality when compared to purely extractive systems . |
Experiments | The joint extractive and compressive system fits more word types into a summary than the extractive systems , but also produces longer sentences on average. |
Experiments | Reading the output summaries more carefully suggests that by learning to extract and compress jointly, our joint system has the flexibility to use or create reasonable, medium-length sentences, whereas the extractive systems are stuck with a few valuable long sentences, but several less productive shorter sentences. |
Introduction | For example, Zajic et al (2006) use a pipeline approach, preprocessing to yield additional candidates for extraction by applying heuristic sentence compressions, but their system does not outperform state-of-the-art purely extractive systems . |
Introduction | A second contribution of the current work is to show a system for jointly learning to jointly compress and extract that exhibits gains in both ROUGE and content metrics over purely extractive systems . |
Introduction | learns parameters for compression and extraction jointly using an approximate training procedure, but his results are not competitive with state-of-the-art extractive systems , and he does not report improvements on manual content or quality metrics. |
Joint Model | Learning weights for Objective 1 where Y(:c) is the set of extractive summaries gives our LEARNED EXTRACTIVE system . |
Experiments | In our study, we compared the characteristics of summaries generated by the eight human summarizers with those generated by the peer summaries, which are basically extractive systems . |
Experiments | Purely extractive systems would thus be expected to score 1.0, as would systems that perform text compression by remov- |
Experiments | Peer 2 shows a relatively high level of aggregation despite being an extractive system . |
Introduction | In automatic summarization, centrality has been one of the guiding principles for content selection in extractive systems . |
Related Work | Domain-dependent template-based summarization systems have been an alternative to extractive systems which make use of rich knowledge about a domain and information extraction techniques to generate a summary, possibly using a natural language generation system (Radev and McKeown, 1998; White et al., 2001; McKeown et al., 2002). |
Related Work | Several studies complement this paper by examining the best possible extractive system using current evaluation measures, such as ROUGE (Lin and Hovy, 2003; Conroy et al., 2006). |
Related Work | They find that the best possible extractive systems score higher or as highly than human summarizers, but it is unclear whether this means the oracle summaries are actually as useful as human ones in an extrinsic setting. |
Conclusions | Results show that the state of the art is improved in automatic and manual metrics, with speeds close to extractive systems . |
Experiments | The bottom rows show the results achieved by our implementation of a pure extractive system (similar to the learned extractive summarizer of Berg-Kirkpatrick et al., 2011); a system that post-combines extraction and compression components trained separately, as in Martins and Smith (2009); and our compressive summarizer trained as a single task, and in the multitask setting. |
Experiments | The ROUGE and Pyramid scores show that the compressive summarizers (when properly trained) yield considerable benefits in content coverage over extractive systems , confirming the results of Berg-Kirkpatrick et al. |
Experiments | Our ROUGE-2 score (12.30%) is, to our knowledge, the highest reported on the TAG-2008 dataset, with little harm in grammaticality with respect to an extractive system that preserves the original sentences. |
Introduction | Up to now, extractive systems have been the most popular in multi-document summarization. |
Introduction | However, extractive systems are rather limited in the summaries they can produce. |
Introduction | All approaches above are based on integer linear programming (ILP), suffering from slow runtimes, when compared to extractive systems . |
Experimental Setup | We randomly selected 12 document-image pairs from the test set and generated captions for them using the best extractive system , and two abstractive systems (word-based and phrase-based). |
Results | Table 3 reports mean ratings for the output of the extractive system (based on the KL divergence), the two abstractive systems, and the human-authored gold standard caption. |
Results | It is significantly worse than the phrase-based abstractive system (0c < 0.01), the extractive system (0c < 0.01), and the gold standard (0c < 0.01). |
Results | Unsurprisingly, the phrase-based system is significantly less grammatical than the gold standard and the extractive system , whereas the latter is perceived as equally grammatical as the gold standard (the difference in the means is not significant). |
Analysis | Although a few researchers have presented a sample of their systems’ output and the corresponding gold keyphrases to show the differences between them (Witten et al., 1999; Nguyen and Kan, 2007; Medelyan et al., 2009), a systematic analysis of the major types of errors made by state-of-the-art keyphrase extraction systems is missing. |
Analysis | To fill this gap, we ran four keyphrase extraction systems on four commonly-used datasets of varying sources, including Inspec abstracts (Hulth, 2003), DUC-2001 news articles (Over, 2001), scientific papers (Kim et al., 2010b), and meeting transcripts (Liu et al., 2009a). |
Corpora | Automatic keyphrase extraction systems have been evaluated on corpora from a variety of |
Evaluation | In this section, we describe metrics for evaluating keyphrase extraction systems as well as state-of-the-art results on commonly-used datasets. |
Evaluation | To score the output of a keyphrase extraction system , the typical approach, which is also adopted by the SemEval—2010 shared task on keyphrase extraction, is (1) to create a mapping between the keyphrases in the gold standard and those in the system output using exact match, and then (2) score the output using evaluation metrics such as precision (P), recall (R), and F-score (F). |
Keyphrase Extraction Approaches | A keyphrase extraction system typically operates in two steps: (1) extracting a list of words/phrases that serve as candidate keyphrases using some |
Baseline | As the baseline, we choose a state-of-the-art Chinese event extraction system , as described in Li et al. |
Experimentation | For fair comparison, we adopt the same experimental settings as the state-of-the-art event extraction system (Li et al. |
Experimentation | Besides, all the experiments on argument extraction are done on the output of the trigger extraction system as described in Li et al. |
Experimentation | Table 3 shows the performance of the baseline trigger extraction system and Line 1 in Table 4 illustrates the results of argument identification and role determination based on this system. |
Introduction | Section 3 describes a state-of-the-art Chinese argument extraction system as the baseline. |
Abstract | However, it is not straightforward to adapt the existing event extraction systems since texts in social media are fragmented and noisy. |
Experiments | In this section, we first describe the Twitter corpus used in our experiments and then present how we build a baseline based on the previously proposed TwiCal system (Ritter et al., 2012), the state-of-the-art open event extraction system on tweets. |
Experiments | To evaluate the performance of the propose approach, we use preeison, recall, and F —measure as in general information extraction systems (Makhoul et al., 1999). |
Experiments | It can be observed from Table 2 that by using NW—NER, the performance of event extraction system is improved significantly by 7.5% and 3% respectively on F-measure when evaluated on 3-tuples (without keywords) or 4-tuples (with keywords). |
Introduction | We have conducted experiments on a Twitter corpus and the results show that our proposed approach outperforms TwiCal, the state-of-the-art open event extraction system , by 7.7% in F-measure. |
Experimental Setup | Moreover, we compare our abstractive system with the first part of our framework (utterance extraction in Figure l), which can be presented as an extractive query-based summarization system (our extractive system ). |
Experimental Setup | We also show the results of the version we use in our pipeline (our pipeline extractive system ). |
Experimental Setup | In contrast, in the stand alone version ( extractive system ) we limit the number of retrieved sentences to the desired length of the summary. |
Abstract | This paper proposes a novel graph-based projection approach and demonstrates the merits of it by using a Korean relation extraction system based on projected dataset from an English—Korean parallel corpus. |
Conclusions | The feasibility of our approach was demonstrated by our Korean relation extraction system . |
Implementation | To demonstrate the effectiveness of the graph-based projection approach for relation extraction, we developed a Korean relation extraction system that was trained with projected annotations from English resources. |
Implementation | We obtained 155,409 positive instances from the English sentences using an off-the-shelf relation extraction system , ReVerb 2 (Fader et al., 201 l). |
Introduction | To obtain training examples in the resource-poor target language, this approach exploited a cross-lingual annotation projection by propagating annotations that were generated by a relation extraction system in a resource-rich source language. |
Abstract | We present a simple semi-supervised relation extraction system with large-scale word clustering. |
Conclusion and Future Work | We have described a semi-supervised relation extraction system with large-scale word clustering. |
Feature Based Relation Extraction | (2005), a state-of—the-art feature based relation extraction system . |
Introduction | For example, a relation extraction system needs to be able to extract an Employment relation between the entities US soldier and US in the phrase US soldier. |
Introduction | The performance of a supervised relation extraction system is usually degraded by the sparsity of lexical features. |
Background: Never-Ending Language Learner | NELL is an information extraction system that has been running 24x7 for over a year, using coupled semi-supervised learning to populate an ontology from unstructured text found on the web. |
Background: Never-Ending Language Learner | As in other information extraction systems , the category and relation instances extracted by NELL contain polysemous and synonymous noun phrases. |
Discussion | In order for information extraction systems to accurately represent knowledge, they must represent noun phrases, concepts, and the many-to-many mapping from noun phrases to concepts they denote. |
Introduction | Many information extraction systems construct knowledge bases by extracting structured assertions from free text (e.g., NELL (Carlson et al., 2010), TextRunner (Banko et al., 2007)). |
Introduction | Instead, it can be more cost-effective to adapt an existing relation extraction system to the new domain using a small set of labeled data. |
Introduction | This paper considers relation adaptation, where a relation extraction system trained on many source domains is adapted to a new target domain. |
Introduction | There are at least three challenges when adapting a relation extraction system to a new domain. |
Related Work | Among these studies, Plank and Moschitti’s is the closest to ours because it adapts relation extraction systems to new domains. |
Abstract | In order to extract entities of a fine- grained category from semistructured data in web pages, existing information extraction systems rely on seed examples or redundancy across multiple web pages. |
Approach | We represent each web page 212 as a DOM tree, a common representation among wrapper induction and web information extraction systems (Sahuguet and Azavant, 1999; Liu et al., 2000; Crescenzi et al., 2001). |
Approach | In the literature, many information extraction systems employ more versatile extraction predicates (Wang and Cohen, 2009; Fumarola et al., 2011). |
Discussion | In contrast to information extraction systems that extract homogeneous records from web pages (Liu et al., 2003; Zheng et al., 2009), our system must choose the correct field from each record and also identify the relevant part of the page based on the query. |
Abstract | Our approach is to measure the impact of each parser when it is used as a component of an information extraction system that performs protein—protein interaction (PPI) identification in biomedical papers. |
Evaluation Methodology | In our approach to parser evaluation, we measure the accuracy of a PPI extraction system , in which |
Experiments | In the following experiments, we used AImed (Bunescu and Mooney, 2004), which is a popular corpus for the evaluation of PPI extraction systems . |
Identifying Key Medical Relations | The first step in building a relation extraction system for medical domain is to identify the relations that are important for clinical decision making. |
Introduction | To construct a medical relation extraction system , several challenges have to be addressed: |
Introduction | The medical corpus underlying our relation extraction system contains 80M sentences (ll gigabytes pure text). |
Abstract | Second, when the number of target relations is small, and their names are known in advance, we show that O-CRF is able to match the precision of a traditional extraction system , though at substantially lower recall. |
Relation Extraction | The unique nature of the open extraction task has led us to develop O-CRF, an open extraction system that uses the power of graphical models to identify relations in text. |
Relation Extraction | The set of features used by O-CRF is largely similar to those used by O-NB and other state-of-the-art relation extraction systems , They include part-of-speech tags (predicted using a separately trained maximum-entropy model), regular expressions (e. g.detecting capitalization, punctuation, eta), context words, and conjunctions of features occurring in adjacent positions within six words to the left and six words to the right of the current word. |