A Comparative Study on Generalization of Semantic Roles in FrameNet
Matsubayashi, Yuichiroh and Okazaki, Naoaki and Tsujii, Jun'ichi

Article Structure

Abstract

A number of studies have presented machine-learning approaches to semantic role labeling with availability of corpora such as FrameNet and PropBank.

Introduction

Semantic Role Labeling (SRL) is a task of analyzing predicate-argument structures in texts.

Related Work

Moschitti et al.

Role Classification

SRL is a complex task wherein several problems are intertwined: frame-evoking word identification, frame disambiguation (selecting a correct frame from candidates for the evoking word), role-phrase identification (identifying phrases that fill semantic roles), and role classification (assigning correct roles to the phrases).

Design of Role Groups

We formalize the generalization of semantic roles as the act of grouping several roles into a class.

Role classification method

5.1 Traditional approach

Experiment and Discussion

We used the training set of the Semeval-2007 Shared task (Baker et al., 2007) in order to ascertain the contributions of role groups.

Conclusion

We have described different criteria for generalizing semantic roles in FrameNet.

Topics

semantic roles

Appears in 41 sentences as: Semantic Role (1) semantic role (5) semantic roles (37)
In A Comparative Study on Generalization of Semantic Roles in FrameNet
  1. A number of studies have presented machine-learning approaches to semantic role labeling with availability of corpora such as FrameNet and PropBank.
    Page 1, “Abstract”
  2. These corpora define the semantic roles of predicates for each frame independently.
    Page 1, “Abstract”
  3. Thus, it is crucial for the machine-learning approach to generalize semantic roles across different frames, and to increase the size of training instances.
    Page 1, “Abstract”
  4. This paper explores several criteria for generalizing semantic roles in FrameNet: role hierarchy, human-understandable descriptors of roles, semantic types of filler phrases, and mappings from FrameNet roles to thematic roles of VerbNet.
    Page 1, “Abstract”
  5. Semantic Role Labeling (SRL) is a task of analyzing predicate-argument structures in texts.
    Page 1, “Introduction”
  6. More specifically, SRL identifies predicates and their arguments with appropriate semantic roles .
    Page 1, “Introduction”
  7. These corpora define a large number of frames and define the semantic roles for each frame independently.
    Page 1, “Introduction”
  8. PropBank defines a frame for each sense of predicates (e.g., buy.01), and semantic roles are defined in a frame-specific manner (e.g., buyer and seller for buy.
    Page 1, “Introduction”
  9. Some recent studies have addressed alternative approaches to generalizing semantic roles across different frames (Gordon and Swanson, 2007; Zapi-
    Page 1, “Introduction”
  10. FrameNet designs semantic roles as frame specific, but also defines hierarchical relations of semantic roles among frames.
    Page 2, “Introduction”
  11. Although the role hierarchy was expected to generalize semantic roles , no positive results for role classification have been reported (B aldewein et al., 2004).
    Page 2, “Introduction”

See all papers in Proc. ACL 2009 that mention semantic roles.

See all papers in Proc. ACL that mention semantic roles.

Back to top.

F1 scores

Appears in 5 sentences as: F1 score (2) F1 scores (4)
In A Comparative Study on Generalization of Semantic Roles in FrameNet
  1. Table 1 shows the micro and macro averages of F1 scores .
    Page 6, “Experiment and Discussion”
  2. Moreover, the macro-averaged F1 scores clearly showed improvements resulting from using role groups.
    Page 6, “Experiment and Discussion”
  3. In Table 2, we show that the micro-averaged F1 score for roles having 10 instances or less was improved (by 15.46 points) when all role groups were used.
    Page 6, “Experiment and Discussion”
  4. Table 3 shows the micro-averaged F1 scores of all
    Page 6, “Experiment and Discussion”
  5. Table 6 reports the precision, recall, and micro-averaged F1 scores of semantic roles with respect to each coreness type.4 In general, semantic roles of the core coreness were easily identified by all of the grouping criteria; even the baseline system obtained an F1 score of 91.93.
    Page 7, “Experiment and Discussion”

See all papers in Proc. ACL 2009 that mention F1 scores.

See all papers in Proc. ACL that mention F1 scores.

Back to top.

role labels

Appears in 4 sentences as: Role Labeling (1) role labeling (1) role labels (2)
In A Comparative Study on Generalization of Semantic Roles in FrameNet
  1. A number of studies have presented machine-learning approaches to semantic role labeling with availability of corpora such as FrameNet and PropBank.
    Page 1, “Abstract”
  2. Semantic Role Labeling (SRL) is a task of analyzing predicate-argument structures in texts.
    Page 1, “Introduction”
  3. We define a role group as a set of role labels grouped by a criterion.
    Page 3, “Design of Role Groups”
  4. We confirmed that modeling the role generalization at feature level was better than the conventional approach that replaces semantic role labels .
    Page 8, “Conclusion”

See all papers in Proc. ACL 2009 that mention role labels.

See all papers in Proc. ACL that mention role labels.

Back to top.

baseline system

Appears in 3 sentences as: baseline system (3)
In A Comparative Study on Generalization of Semantic Roles in FrameNet
  1. The baseline system achieved 89.00% with respect to the micro-averaged F1.
    Page 6, “Experiment and Discussion”
  2. Table 6 reports the precision, recall, and micro-averaged F1 scores of semantic roles with respect to each coreness type.4 In general, semantic roles of the core coreness were easily identified by all of the grouping criteria; even the baseline system obtained an F1 score of 91.93.
    Page 7, “Experiment and Discussion”
  3. Since we used the latest release of FrameNet in order to use a greater number of hierarchical role-to-role relations, we could not make a direct comparison of performance with that of existing systems; however we may say that the 89.00% F1 micro-average of our baseline system is roughly comparable to the 88.93% value of Bejan and Hathaway (2007) for SemEval-2007 (Baker et al., 2007).
    Page 8, “Conclusion”

See all papers in Proc. ACL 2009 that mention baseline system.

See all papers in Proc. ACL that mention baseline system.

Back to top.

content word

Appears in 3 sentences as: content word (3)
In A Comparative Study on Generalization of Semantic Roles in FrameNet
  1. The characteristics of x are: frame, frame evoking word, head word, content word (Surdeanu et al., 2003), first/last word, head word of left/right sister, phrase type, position, voice, syntactic path (di-rected/undirected/partial), governing category (Gildea and Jurafsky, 2002), WordNet supersense in the phrase, combination features of frame evoking word & headword, combination features of frame evoking word & phrase type, and combination features of voice & phrase type.
    Page 6, “Experiment and Discussion”
  2. associations with lexical and structural characteristics such as the syntactic path, content word , and head word.
    Page 8, “Experiment and Discussion”
  3. or hr rl st vn frame 0 4 0 1 0 evoking word 3 4 7 3 0 ew & hw stem 9 34 20 8 0 ew & phrase type 11 7 11 3 1 head word 1 3 19 8 3 1 hw stem 1 1 17 8 8 1 content word 7 19 12 3 0 cw stem 1 1 26 13 5 0 cw P08 4 5 14 15 2 directed path 19 27 24 6 7 undirected path 21 35 17 2 6 partial path 15 18 16 13 5 last word 15 1 8 12 3 2 first word 1 1 23 53 26 10 supersense 7 7 35 25 4 position 4 6 30 9 5 others 27 29 3 3 19 6 total 188 298 313 152 50
    Page 8, “Conclusion”

See all papers in Proc. ACL 2009 that mention content word.

See all papers in Proc. ACL that mention content word.

Back to top.

Semantic Role Labeling

Appears in 3 sentences as: Semantic Role Labeling (1) semantic role labeling (1) semantic role labels (1)
In A Comparative Study on Generalization of Semantic Roles in FrameNet
  1. A number of studies have presented machine-learning approaches to semantic role labeling with availability of corpora such as FrameNet and PropBank.
    Page 1, “Abstract”
  2. Semantic Role Labeling (SRL) is a task of analyzing predicate-argument structures in texts.
    Page 1, “Introduction”
  3. We confirmed that modeling the role generalization at feature level was better than the conventional approach that replaces semantic role labels .
    Page 8, “Conclusion”

See all papers in Proc. ACL 2009 that mention Semantic Role Labeling.

See all papers in Proc. ACL that mention Semantic Role Labeling.

Back to top.