Stochastic optimization methods have proven to be extremely efficient for the training of models involving computationally expensive objective functions like those encountered with our task (Vishwanathan et al., 2006) and, in fact, the online backpropagation learning used in the neural network parser of Henderson (2004) is a form of stochastic gradient descent.

In our experiments SGD converged to a lower objective function value than L-BFGS, however it required far

Utilization of stochastic optimization routines requires the implementation of a stochastic objective function .

2.2 Computing the Objective Function

By both changing the objective function to include a bias toward sparser models and improving the pruning techniques and efficiency, we achieve significant gains on test data with practical speed.

Given an unlimited amount of time, we would tune the prior to maximize end-to-end performance, using an objective function such as BLEU.

First we change the objective function by incorporating a prior over the phrasal parameters.