Index of papers in Proc. ACL 2010 that mention
  • development set
Xiong, Deyi and Zhang, Min and Li, Haizhou
Error Detection with a Maximum Entropy Model
To avoid overfitting, we optimize the Gaussian prior on the development set .
Experiments
We find that the LG parser can not fully parse 560 sentences (63.8%) in the training set (MT-02), 731 sentences (67.6%) in the development set (MT-05) and 660 sentences (71.8%) in the test set (MT-03).
Experiments
To compare with previous work using word posterior probabilities for confidence estimation, we carried out experiments using wpp estimated from N -best lists with the classification threshold 7', which was optimized on our development set to minimize CER.
Features
1This does not mean we do not need a development set .
Features
We do validate our feature selection and other experimental settings on the development set .
Features
We optimize the discrete factor on our development set and find the optimal value is 1.
Introduction
3) Divide words into two groups (correct translations and errors) by using a classification threshold optimized on a development set .
Related Work
rectly output prediction results from our dis-criminatively trained classifier without optimizing a classification threshold on a distinct development set beforehand.1 Most previous approaches make decisions based on a pre-tuned classification threshold 7' as follows
SMT System
For minimum error rate tuning (Och, 2003), we use NIST MT-02 as the development set for the translation task.
development set is mentioned in 9 sentences in this paper.
Topics mentioned in this paper:
Berant, Jonathan and Dagan, Ido and Goldberger, Jacob
Experimental Evaluation
The graphs were randomly split into a development set (11 graphs) and a test set (12 graphs)6.
Experimental Evaluation
The left half depicts methods where the development set was needed to tune parameters, and the right half depicts methods that do not require a (manually created) development set at all.
Experimental Evaluation
Results on the left were achieved by optimizing the top-K parameter on the development set , and on the right by optimizing on the training set automatically generated from WordNet.
Learning Entailment Graph Edges
Note that this constant needs to be optimized on a development set .
Learning Entailment Graph Edges
Importantly, while the score-based formulation contains a parameter A that requires optimization, this probabilistic formulation is parameter free and does not utilize a development set at all.
development set is mentioned in 8 sentences in this paper.
Topics mentioned in this paper:
Turian, Joseph and Ratinov, Lev-Arie and Bengio, Yoshua
Supervised evaluation tasks
training partition sentences, and evaluated their F1 on the development set .
Supervised evaluation tasks
After each epoch over the training set, we measured the accuracy of the model on the development set .
Supervised evaluation tasks
Training was stopped after the accuracy on the development set did not improve for 10 epochs, generally about 50—80 epochs total.
development set is mentioned in 7 sentences in this paper.
Topics mentioned in this paper:
Liu, Zhanyi and Wang, Haifeng and Wu, Hua and Li, Sheng
Experiments on Parsing-Based SMT
The feature weights are tuned on the development set using the minimum error
Experiments on Phrase-Based SMT
We used the NIST MT-2002 set as the development set and the NIST MT-2004 test set as the test set.
Experiments on Phrase-Based SMT
And Koehn's implementation of minimum error rate training (Och, 2003) is used to tune the feature weights on the development set .
Experiments on Word Alignment
(11), we also manually labeled a development set including 100 sentence pairs, in the same manner as the test set.
Experiments on Word Alignment
By minimizing the AER on the development set , the interpolation coefficients of the collocation probabilities on CM-l and CM-2 were set to 0.1 and 0.9.
Improving Phrase Table
For the phrase only including one word, we set a fixed collocation probability that is the average of the collocation probabilities of the sentences on a development set .
development set is mentioned in 6 sentences in this paper.
Topics mentioned in this paper:
Chambers, Nathanael and Jurafsky, Daniel
Experiments
We randomly chose 9 documents from the year 2001 for a development set , and 41 documents for testing.
How Frequent is Unseen Data?
We then record every seen (vd, n) pair during training that is seen two or more times3 and then count the number of unseen pairs in the NYT development set (1455 tests).
How Frequent is Unseen Data?
Figure 1: Percentage of NYT development set that is unseen when trained on varying amounts of data.
How Frequent is Unseen Data?
Figure 2: Percentage of subject/object/preposition arguments in the NYT development set that is unseen when trained on varying amounts of NYT data.
development set is mentioned in 5 sentences in this paper.
Topics mentioned in this paper:
Cheung, Jackie Chi Kit and Penn, Gerald
Introduction
There are 216 documents and 4126 original-permutation pairs in the training set, and 24 documents and 465 pairs in the development set .
Introduction
Transition length, salience, and a regularization parameter are tuned on the development set .
Introduction
We only report results using the setting of transition length g 4, and no salience threshold, because they give the best performance on the development set .
development set is mentioned in 4 sentences in this paper.
Topics mentioned in this paper:
Jiang, Wenbin and Liu, Qun
Boosting an MST Parser
The relative weight A is adjusted to maximize the performance on the development set , using an algorithm similar to minimum error-rate training (Och, 2003).
Experiments
Figure 3: Performance curves of the word-pair classification model on the development sets of WSJ and CTB 5.0, with respect to a series of ratio 7“.
Experiments
Figure 4: The performance curve of the word-pair classification model on the development set of CTB 5.0, with respect to a series of threshold (9.
Experiments
Then, on each instance set we train a classifier and test it on the development set of CTB 5.0.
development set is mentioned in 4 sentences in this paper.
Topics mentioned in this paper:
Xiao, Tong and Zhu, Jingbo and Zhu, Muhua and Wang, Huizhen
Background
1 The data set used for weight training is generally called development set or tuning set in the SMT field.
Background
We see, first of all, that all the three systems are improved during iterations on the development set .
Background
iteration number Figure 2: BLEU scores on the development set
development set is mentioned in 4 sentences in this paper.
Topics mentioned in this paper:
Sun, Jun and Zhang, Min and Tan, Chew Lim
Substructure Spaces for BTKs
The coefficient Oi for the composite kernel are tuned with respect to F-measure (F) on the development set of HIT corpus.
Substructure Spaces for BTKs
Those thresholds are also tuned on the development set of HIT corpus with respect to F-measure.
Substructure Spaces for BTKs
We use these sentences with less than 50 characters from the NIST MT-2002 test set as the development set (to speed up tuning for syntax based system) and the NIST MT-2005 test set as our test set.
development set is mentioned in 3 sentences in this paper.
Topics mentioned in this paper: