Variational Inference for Grammar Induction with Prior Knowledge
Cohen, Shay and Smith, Noah A

Article Structure

Abstract

Variational EM has become a popular technique in probabilistic NLP with hidden variables.

Introduction

Learning natural language in an unsupervised way commonly involves the expectation-maximization (EM) algorithm to optimize the parameters of a generative model, often a probabilistic grammar (Pereira and Schabes, 1992).

Variational Mixtures with Constraints

Our EM variant encodes prior knowledge in an approximate posterior by constraining it to be from a mixture family of distributions.

Topics

grammar induction

Appears in 4 sentences as: grammar induction (4)
In Variational Inference for Grammar Induction with Prior Knowledge
  1. For example, Smith and Eisner (2006) have penalized the approximate posterior over dependency structures in a natural language grammar induction task to avoid long range dependencies between words.
    Page 1, “Introduction”
  2. We show that empirically, injecting prior knowledge improves performance on an unsupervised Chinese grammar induction task.
    Page 1, “Introduction”
  3. This is a strict model reminiscent of the successful application of structural bias to grammar induction (Smith and Eisner, 2006).
    Page 4, “Variational Mixtures with Constraints”
  4. We demonstrated the effectiveness of the algorithm on a dependency grammar induction task.
    Page 4, “Variational Mixtures with Constraints”

See all papers in Proc. ACL 2009 that mention grammar induction.

See all papers in Proc. ACL that mention grammar induction.

Back to top.

natural language

Appears in 3 sentences as: natural language (3)
In Variational Inference for Grammar Induction with Prior Knowledge
  1. Learning natural language in an unsupervised way commonly involves the expectation-maximization (EM) algorithm to optimize the parameters of a generative model, often a probabilistic grammar (Pereira and Schabes, 1992).
    Page 1, “Introduction”
  2. Later approaches include variational EM in a Bayesian setting (Beal and Gharamani, 2003), which has been shown to obtain even better results for various natural language tasks over EM (e.g., Cohen et al., 2008).
    Page 1, “Introduction”
  3. For example, Smith and Eisner (2006) have penalized the approximate posterior over dependency structures in a natural language grammar induction task to avoid long range dependencies between words.
    Page 1, “Introduction”

See all papers in Proc. ACL 2009 that mention natural language.

See all papers in Proc. ACL that mention natural language.

Back to top.