In this paper, we present a robust extension of logistic regression that incorporates the possibility of mislabelling directly into the objective.

This model can be trained through nearly the same means as logistic regression , and retains its efficiency on high-dimensional datasets.

In this work we argue that incorrect examples should be explicitly modelled during training, and present a simple extension of logistic regression that incorporates the possibility of mislabelling directly into the objective.

It has a convex objective, is well-suited to high-dimensional data, and can be efficiently trained with minimal changes to the logistic regression pipeline.

In experiments on a large, noisy NER dataset, we find that this method can provide an improvement over standard logistic regression when annotation errors are present.

Recall that in binary logistic regression , the probability of an example :10, being positive is modeled as

Upon writing the objective in this way, we immediately see that it is convex, just as standard L1-penalized logistic regression is convex.

Bootkrajang and Kaban (2012) present an extension of logistic regression that models annotation errors through flipping probabilities.

In this work we adapt the technique to logistic regression .

To the best of our knowledge, we are the first to experiment with adding ‘shift parameters’ to logistic regression and demonstrate that the model is especially well-suited to the type of high-dimensional, noisy datasets commonly used in NLP.

We can see that the chain based models, Linear Chain Markov Model (LCMM) and Global Chain Model (GCM), outperform the unstructured models, namely Logistic regression (LR) and Decision Trees (J48).

3.2 Logistic Regression (LR)

Our first attempt at solving this problem involved training a logistic regression for the binary prediction task which models P(r|t).

Our logistic regression model uses the following two types of features: Thread only features and Aggregated post features.

The first uses a logistic regression model that primarily incorporates high level information about threads and posts.

With this featurization and training data, we train a binary logistic regression classifier with 61 regularization (Where negative examples are comprised of all character entities in the previous 100 words not labeled as the true antecedent).

Since each multiplicand involves a binary prediction, we can avoid partition functions and use the classic binary logistic regression.7 We have converted the V-way multiclass logistic regression problem of Eq.

7Recall that logistic regression lets PLR(y = 1 I :13, fl) = logit_1(a:Tfl) = 1/(1 —|— exp —:1:Tfl) for binary dependent variable y, independent variables :13, and coefficients fl.

This equates to solving 4V El-regularized logistic regressions (see Eq.

o LRl, our most basic logistic regression baseline, uses only bag of words (BOW) features.

o LR-(W2V) is a logistic regression model trained on the average of the pretrained word embeddings for each sentence (Section 2.2).

The RNN framework, adding phrase-level data, and initializing with word2vec all improve performance over logistic regression baselines.

(2013) use logistic regression ).

In addition, decision-tree classifiers learn nonlinear responses to inputs and often outperform logistic regression (Perlich et al., 2003).9 Our random forest classifier models the probability that the input syntactic relation is metaphorical.

(2013), we use a logistic regression classifier to propagate abstractness and imageability scores from MRC ratings to all words for which we have vector space representations.

9In our experiments, random forests model slightly outperformed logistic regression and SVM classifiers.

In our experiments, we used L2 reg-ularised logistic regression .

The L-BFGS (Liu and Nocedal, 1989) method is used to train the CRF and logistic regression models.

Specifically, in POS tagging, a CRF trained on source domain labeled sentences is applied to target domain test sentences, whereas in sentiment classification, a logistic regression classifier trained using source domain labeled reviews is applied to the target domain test reviews.

To train our system on binarized data, we replaced the £2 -regularized linear regression model with an 62-regularized logistic regression and used Kendall’s 7' rank correlation between the predicted probabilities of the positive class and the binary gold standard labels as the grid search metric (§3.1) instead of Pearson’s 7“.

Therefore, we used the same learning algorithms as for our system (i.e., ridge regression for the ordinal task and logistic regression for the binary task).14

14In preliminary experiments, we observed little difference in performance between logistic regression and the original support vector classifier used by the system from Post (201 l).

To verify the quality of our subjectivity features, we measured their performance as predictors in a logistic regression classifying the 3,250 adjectives labelled as subjective or not in the Wilson et al.

( logistic regression with 10-fold cross-validation), we test whether our lexical representations based on subjectivity and concreteness convey sufficient information to perform the same classification.

We again aim to classify the remaining 211 intersective and 93 sub-sective pairs with a logistic regression .

model (Lin), a generalized linear model which is the logistic regression model (Logit) and non linear regression models such as polynomial regression model (Polyn) of order n. Given an input vector cc E R", where $1,...,:cm represent features, we find a prediction 3) E R" for the co-occurrence count of a couple of words 3/ E R using one of the regression models presented below:

We contrast the simple linear regression model (Lin) with the second and the third order polynomial regressions (Poly2 and P0ly3) and the logistic regression model (Logit).

This suggests that both linear and polynomial regressions are suitable as a preprocessing step of the standard approach, while the logistic regression seems to be inappropriate according to the results shown in Table 6.

This measure of density was used as a factor in logistic regression to predict the first occurrence of a verb in the construction, coded as the binary variable OCCURRENCE, set to 1 for the first period in which the verb is attested in the construction, and to 0 for all preceding periods (later periods were discarded).

Table 1: Summary of logistic regression results for different values of N. Model formula: OCCURRENCE ~ DENSITY + (1 + DENSITY|VERB).

Using multidimensional scaling and logistic regression , it was shown that the occurrence of new items throughout the history of the construction can be predicted by the density of the semantic space in the neighborhood of these items in prior usage.

form logistic regression baseline.

While prior work employed tens of thousands of human labeled examples (Zhang et al., 2012) and only got a 6.5% increase in F-score over a logistic regression baseline, our approach uses much less labeled data (about 1/8) but achieves much higher improvement on performance over stronger baselines.

2All classifiers are implemented using L2-regularized logistic regression with Stanford CoreNLP package.

We use logistic regression (LR) with L2 regularization (Fan et al., 2008) and the SVMWWW (SVM) system (Joachims, 2007) with its default settings as the classifiers.

Many classification algorithms give such scores, e.g., SVM and logistic regression .

On developing effective learning techniques, supervised classification has been the dominant approach, e.g., neural networks (Graham et al., 2005; Zheng et al., 2006), decision tree (Uzuner and Katz, 2005; Zhao and Zobel, 2005), logistic regression (Madigan et al., 2005), SVM (Diederich et al., 2000; Gamon 2004; Li et al., 2006; Kim et al., 2011), etc.