In this paper we estimate approximate posterior inference using collapsed Gibbs sampling (Griffiths and Steyvers, 2004).

The Gibbs sampling equation used to update the assignment of a topic I to the word 21) E W at the position n in document d, conditioned on at, flu, is:

We use a subscript d, fin to denote the current token, zdm is ignored in the Gibbs sampling update.

l. Infer T number of topics on D for LDA using collapsed Gibbs sampling .

Update M D using collapsed Gibbs sampling update in Equation 1.

Infer ‘0‘ number of topics on the sprinkled document corpus D using collapsed Gibbs sampling update.

We then update the new LDA model using collapsed Gibbs sampling .

We then infer a set of |C | number of topics on the sprinkled dataset using collapsed Gibbs sampling , where C is the set of class labels of the training documents.

We modify collapsed Gibbs sampling update in Equation 1 to carry class label information while inferring topics.

Most importantly, due to the use of the new form of knowledge, AKL’s inference mechanism ( Gibbs sampler ) is entirely different from that of MC-LDA (Section 5.2), which results in superior performances (Section 6).

In short, our modeling contributions are (1) the capability of handling more expressive knowledge in the form of clusters, (2) a novel Gibbs sampler to deal with inappropriate knowledge.

5.2 The Gibbs Sampler

Therefore, the first-order distribution is not well-defined and we only employ Gibbs sampling for simplicity.

Our first strategy is akin to Gibbs sampling and samples a new head for each word in the sentence, modifying one arc at a time.

iteration of this sampler makes multiple changes to the tree, in contrast to a single-edge change of Gibbs sampler .

3.2.1 Gibbs Sampling

One shortcoming of the Gibbs sampler is that it only changes one variable (arc) at a time.

Note that blocked Gibbs sampling would be exponential in K, and is thus very slow already at K = 4.

We present a block Gibbs sampler for posterior inference and an empirical evaluation on several datasets.

We use a block Gibbs sampler , which from an initial state (190,21), zo) repeats these steps: 1.

The topics of context words are assumed exchangeable, and so we re-sample them using Gibbs sampling (Griffiths and Steyvers, 2004).

Unfortunately, this is prohibitively expensive for the (nonexchangeable) topics of the named mentions c. A Gibbs sampler would have to choose a new value for cc.z with probability proportional to the resulting joint probability of the full sample.

3For Gibbs sampling , we use implementations available in Hu and Boyd-Graber (2012) for tLDA; and Mallet (McCallum, 2002) for LDA and pLDA.

We use a collapsed Gibbs sampler for tree-based topic models to sample the path ydn and topic assignment zdn for word wdn,

For topic .2 and path y, instead of variational updates, we use a Gibbs sampler within a document.

This equation embodies how this is a hybrid algorithm: the first term resembles the Gibbs sampling term encoding how much a document prefers a topic, while the second term encodes the expectation under the variational distribution of how much a path is preferred by this topic,

Here, the topics are extracted from all the documents in the *SEM 2012 shared task using the LDA Gibbs Sampling algorithm (Griffiths, 2002).

where Rel(w,, rm) is the weight of word w in topic rm calculated by the LDA Gibbs Sampling algorithm.

> Topic Modeler: For estimating transition probability Pt(i,m), we employ GibbsLDA++6, an LDA model using Gibbs Sampling technique for parameter estimation and inference.

All experiments are run with 50 iterations of Gibbs sampling to collect samples for the personas p, alternating with maximization steps for 77.

Rather than adopting a fully Bayesian approach (e.g., sampling all variables), we infer these values using stochastic EM, alternating between collapsed Gibbs sampling for each p and maximizing with respect to 77.

8We assume the reader is familiar with collapsed Gibbs sampling as used in latent-variable NLP models.

(2010) used the local best alignment to increase the speed of the Gibbs sampling in training but the impact on accuracy was not explored.

To this end, we model bilingual UWS under a similar framework with monolingual UWS in order to improve efficiency, and replace Gibbs sampling with expectation maximization (EM) in training.

EF/{;}(P(.7-"k/|.7-")) = P(J:k'|f, M) in a similar manner to the marginalization in the Gibbs sampling process which we are replacing;

For constraints with higher-order structures, we use Gibbs Sampling (Geman and Geman, 1984) to approximate the expectations.

For documents where the higher-order constraints apply, we use the same Gibbs sampler as described above to infer the most likely label assignment, otherwise, we use the Viterbi algorithm.

For approximation inference with higher-order constraints, we perform 2000 Gibbs sampling iterations where the first 1000 iterations are bum-in iterations.

We run the Gibbs samplers for 1000 iterations and update all hyper-parameters using slice sampling (Neal, 2003; Wallach, 2008) every 10 iterations.

We also assume symmetric Dirichlet priors on all multinomial distributions and apply collapsed Gibbs sampling .

All probabilities can be computed using collapsed Gibbs sampler for LDA (Griffiths