Classifying French Verbs Using French and English Lexical Resources
Falk, Ingrid and Gardent, Claire and Lamirel, Jean-Charles

Article Structure

Abstract

We present a novel approach to the automatic acquisition of a Verbnet like classification of French verbs which involves the use (i) of a neural clustering method which associates clusters with features, (ii) of several supervised and unsupervised evaluation metrics and (iii) of various existing syntactic and semantic lexical resources.

Introduction

Verb classifications have been shown to be useful both from a theoretical and from a practical perspective.

Lexical Resources Used

Our aim is to accquire a classification which covers the core verbs of French, could be used to support semantic role labelling and is similar in spirit to the English Verbnet.

Clustering Methods, Evaluation Metrics and Experimental Setup

3.1 Clustering Methods

Features and Data

Features In the simplest case the features are the subcategorisation frames (scf) associated to the verbs by our lexicon.

Topics

F-measure

Appears in 14 sentences as: F-measure (14) f-Measure (1)
In Classifying French Verbs Using French and English Lexical Resources
  1. Their approach achieves a F-measure of 55.1 on 116 verbs occurring at least 150 times in Lexschem.
    Page 1, “Introduction”
  2. The best performance is achieved when restricting the approach to verbs occurring at least 4000 times (43 verbs) with an F-measure of 65.4.
    Page 1, “Introduction”
  3. We show that the approach yields promising results ( F-measure of 70%) and that the clustering produced systematically associates verbs with syntactic frames and thematic grids thereby providing an interesting basis for the creation and evaluation of a Verbnet-like classification.
    Page 2, “Introduction”
  4. Feature maximisation is a cluster quality metric which associates each cluster with maximal features i.e., features whose Feature F-measure is maximal.
    Page 3, “Clustering Methods, Evaluation Metrics and Experimental Setup”
  5. Feature F-measure is the harmonic mean of Feature Recall and Feature Precision which in turn are defined as:
    Page 3, “Clustering Methods, Evaluation Metrics and Experimental Setup”
  6. represents the weight of the feature f for element :10 and FC designates the set of features associated with the verbs occuring in the cluster c. A feature is then said to be maximal for a given cluster iff its Feature F-measure is higher for that cluster than for any other cluster.
    Page 3, “Clustering Methods, Evaluation Metrics and Experimental Setup”
  7. As we shall see, this permits distinguishing between clusterings with similar F-measure but lower “linguistic plausibility” (cf.
    Page 3, “Clustering Methods, Evaluation Metrics and Experimental Setup”
  8. Following (Sun et al., 2010), we use modified purity (mPUR); weighted class accuracy (ACC) and F-measure to evaluate the clusterings produced.
    Page 3, “Clustering Methods, Evaluation Metrics and Experimental Setup”
  9. Finally, F-measure is the harmonic mean of mPUR and ACC.
    Page 3, “Clustering Methods, Evaluation Metrics and Experimental Setup”
  10. 3.3 Cluster display, feature f-Measure and confidence score
    Page 4, “Clustering Methods, Evaluation Metrics and Experimental Setup”
  11. Features are displayed in decreasing order of Feature F-measure (cf.
    Page 4, “Clustering Methods, Evaluation Metrics and Experimental Setup”

See all papers in Proc. ACL 2012 that mention F-measure.

See all papers in Proc. ACL that mention F-measure.

Back to top.

gold standard

Appears in 6 sentences as: Gold Standard (2) gold standard (5)
In Classifying French Verbs Using French and English Lexical Resources
  1. ate the resulting classification on a slightly modified version of the gold standard provided by (Sun et al., 2010).
    Page 2, “Introduction”
  2. 3The training data consists of the verbs and Verbnet classes used in the gold standard presented in (Sun et al., 2010).
    Page 2, “Clustering Methods, Evaluation Metrics and Experimental Setup”
  3. to a gold standard .
    Page 4, “Clustering Methods, Evaluation Metrics and Experimental Setup”
  4. French Gold Standard To evaluate our approach, we use the gold standard proposed by Sun et al.
    Page 5, “Features and Data”
  5. (2010)’s Gold Standard to 11 Verbnet classes thereby associating each class with a thematic grid.
    Page 5, “Features and Data”
  6. Verbs For our clustering experiments we use the 2183 French verbs occurring in the translations of the 11 classes in the gold standard (cf.
    Page 5, “Features and Data”

See all papers in Proc. ACL 2012 that mention gold standard.

See all papers in Proc. ACL that mention gold standard.

Back to top.

evaluation metrics

Appears in 5 sentences as: Evaluation metrics (1) evaluation metrics (4)
In Classifying French Verbs Using French and English Lexical Resources
  1. We present a novel approach to the automatic acquisition of a Verbnet like classification of French verbs which involves the use (i) of a neural clustering method which associates clusters with features, (ii) of several supervised and unsupervised evaluation metrics and (iii) of various existing syntactic and semantic lexical resources.
    Page 1, “Abstract”
  2. 3.2 Evaluation metrics
    Page 3, “Clustering Methods, Evaluation Metrics and Experimental Setup”
  3. We use several evaluation metrics which bear on different properties of the clustering.
    Page 3, “Clustering Methods, Evaluation Metrics and Experimental Setup”
  4. As pointed out in (Lamirel et al., 2008; Attik et al., 2006), unsupervised evaluation metrics based on cluster labelling and feature maximisation can prove very useful for identifying the best clustering strategy.
    Page 3, “Clustering Methods, Evaluation Metrics and Experimental Setup”
  5. Moreover, for this data set, the unsupervised evaluation metrics (cf.
    Page 5, “Features and Data”

See all papers in Proc. ACL 2012 that mention evaluation metrics.

See all papers in Proc. ACL that mention evaluation metrics.

Back to top.

confidence score

Appears in 4 sentences as: confidence score (4)
In Classifying French Verbs Using French and English Lexical Resources
  1. 3.3 Cluster display, feature f-Measure and confidence score
    Page 4, “Clustering Methods, Evaluation Metrics and Experimental Setup”
  2. In addition, for each verb in a cluster, a confidence score is displayed which is the ratio between the sum of the F-measures of its cluster maXimised features over the sum of the F-measures of the overall cluster maXimised features.
    Page 4, “Clustering Methods, Evaluation Metrics and Experimental Setup”
  3. Verbs whose confidence score is O are considered as orphan data.
    Page 4, “Clustering Methods, Evaluation Metrics and Experimental Setup”
  4. Section 3) highlight strong cluster cohesion with a number of clusters close to the number of gold classes (13 clusters for 11 gold classes); a low number of orphan verbs (i.e., verbs whose confidence score is zero); and a high Cumulated Micro Precision (CMP = 0.3) indicating homogeneous clusters in terms of maximis-ing features.
    Page 5, “Features and Data”

See all papers in Proc. ACL 2012 that mention confidence score.

See all papers in Proc. ACL that mention confidence score.

Back to top.

feature set

Appears in 4 sentences as: feature set (3) feature sets (1)
In Classifying French Verbs Using French and English Lexical Resources
  1. Table 1: Sample output for a cluster produced with the grid-scf-sem feature set and the IGNGF clustering method.
    Page 4, “Clustering Methods, Evaluation Metrics and Experimental Setup”
  2. Table 4(a) includes the evaluation results for all the feature sets when using IGNGF clustering.
    Page 5, “Features and Data”
  3. In terms of features, the best results are obtained using the grid-scf-sem feature set with an F-measure of 0.70.
    Page 5, “Features and Data”
  4. In contrast, the classification obtained using the scf-synt-sem feature set has a higher CMP for the clustering with optimal mPUR (0.57); but a lower F—measure (0.61), a larger number of classes (16)
    Page 5, “Features and Data”

See all papers in Proc. ACL 2012 that mention feature set.

See all papers in Proc. ACL that mention feature set.

Back to top.

clusterings

Appears in 3 sentences as: clusterings (3)
In Classifying French Verbs Using French and English Lexical Resources
  1. We make use of these processes in all our experiments and systematically compute cluster labelling and feature maximisation on the output clusterings .
    Page 3, “Clustering Methods, Evaluation Metrics and Experimental Setup”
  2. As we shall see, this permits distinguishing between clusterings with similar F-measure but lower “linguistic plausibility” (cf.
    Page 3, “Clustering Methods, Evaluation Metrics and Experimental Setup”
  3. Following (Sun et al., 2010), we use modified purity (mPUR); weighted class accuracy (ACC) and F-measure to evaluate the clusterings produced.
    Page 3, “Clustering Methods, Evaluation Metrics and Experimental Setup”

See all papers in Proc. ACL 2012 that mention clusterings.

See all papers in Proc. ACL that mention clusterings.

Back to top.

semantic role

Appears in 3 sentences as: semantic role (2) semantic roles (1)
In Classifying French Verbs Using French and English Lexical Resources
  1. From a practical perspective, they support factorisa—tion and have been shown to be effective in various NLP (Natural language Processing) tasks such as semantic role labelling (Swier and Stevenson, 2005) or word sense disambiguation (Dang, 2004).
    Page 1, “Introduction”
  2. Our aim is to accquire a classification which covers the core verbs of French, could be used to support semantic role labelling and is similar in spirit to the English Verbnet.
    Page 2, “Lexical Resources Used”
  3. In addition we group Verbnet semantic roles as shown in Table 4.
    Page 5, “Features and Data”

See all papers in Proc. ACL 2012 that mention semantic role.

See all papers in Proc. ACL that mention semantic role.

Back to top.

PDF available from www.aclweb.org.
Send us an email