Bilingually-Guided Monolingual Dependency Grammar Induction
Article Structure
Abstract
This paper describes a novel strategy for automatic induction of a monolingual dependency grammar under the guidance of bilingually-projected dependency.
Introduction
In past decades supervised methods achieved the state-of-the-art in constituency parsing (Collins, 2003; Charniak and Johnson, 2005; Petrov et al., 2006) and dependency parsing (McDonald et al., 2005a; McDonald et al., 2006; Nivre et al., 2006; Nivre et al., 2007; K00 and Collins, 2010).
Unsupervised Dependency Grammar Induction
In this section, we introduce the unsupervised objective and the unsupervised training algorithm which is used as the framework of our bilingually-guided method.
Bilingual Projection of Dependency Grammar
In this section, we introduce our projection objective and training algorithm which trains the model with arc instances.
Bilingually-Guided Dependency Grammar Induction
This section presents our bilingually-guided grammar induction model, which incorporates unsuperVised framework and bilingual projection model through a joint approach.
Related work
The DMV (Klein and Manning, 2004) is a single-state head automata model (Alshawi, 1996) which is based on POS tags.
Experiments
In this section, we evaluate the performance of the MST dependency parser (McDonald et al., 2005b) which is trained by our bilingually-guided model on 5 languages.
Conclusion and Future Work
This paper presents a bilingually-guided strategy for automatic dependency grammar induction, which adopts an unsupervised skeleton and leverages the bilingually-projected dependency information during optimization.
Topics
treebank
Appears in 20 sentences as: Treebank (2) treebank (19) treebank’s (1)
In Bilingually-Guided Monolingual Dependency Grammar Induction
- A randomly-initialized monolingual treebank evolves in a self-training iterative procedure, and the grammar parameters are tuned to simultaneously maximize both the monolingual likelihood and bilingually-projected likelihood of the evolving treebank .Page 2, “Introduction”
- And the framework of our unsupervised model builds a random treebank on the monolingual corpus firstly for initialization and trains a discriminative parsing model on it.Page 2, “Unsupervised Dependency Grammar Induction”
- Then we use the parser to build an evolved treebank with the l-best result for the next iteration run.Page 2, “Unsupervised Dependency Grammar Induction”
- In this way, the parser and treebank evolve in an iterative way until convergence.Page 2, “Unsupervised Dependency Grammar Induction”
- where E is the monolingual corpus and E E E, DE is the treebank that contains all D E in the corpus, and 15F denotes all other possible dependency arcs which do not exist in the treebank .Page 3, “Unsupervised Dependency Grammar Induction”
- Algorithm 1 outlines the unsupervised training in its entirety, where the treebank DE and unsupervised parsing model with A are updated iteratively.Page 3, “Unsupervised Dependency Grammar Induction”
- In line 1 we build a random treebank DE on the monolingual corpus, and then train the parsing model with it (line 2) through a training procedure train(-, which needs DE and 15F as classification instances.Page 3, “Unsupervised Dependency Grammar Induction”
- Similar to M-step in EM, the algorithm maximizes the whole treebank’s unsupervised objective (Formula 6) through the training procedure (line 6).Page 3, “Unsupervised Dependency Grammar Induction”
- Therefore, we can hardly obtain a treebank with complete trees through direct projection.Page 4, “Bilingual Projection of Dependency Grammar”
- So we extract projected discrete dependency arc instances instead of treebank as training set for the projected grammar induction model.Page 4, “Bilingual Projection of Dependency Grammar”
- Then we incorporate projection model into our iterative unsupervised framework, and jointly optimize unsupervised and projection objectives with evolving treebank and constant projection information respectively.Page 4, “Bilingually-Guided Dependency Grammar Induction”
See all papers in Proc. ACL 2013 that mention treebank.
See all papers in Proc. ACL that mention treebank.
Back to top.
grammar induction
Appears in 13 sentences as: Grammar Induction (1) grammar induction (12)
In Bilingually-Guided Monolingual Dependency Grammar Induction
- Dependency Grammar InductionPage 1, “Introduction”
- In dependency grammar induction , unsupervised methods achieve continuous improvements in recent years (Klein and Manning, 2004; Smith and Eisner, 2005; Bod, 2006; William et al., 2009; Spitkovsky et al., 2010).Page 1, “Introduction”
- In the rest of the paper, we first describe the unsupervised dependency grammar induction framework in section 2 (where the unsupervised optimization objective is given), and introduce the bilingual projection method for dependency parsing in section 3 (where the projected optimization objective is given); Then in section 4 we present the bilingually-guided induction strategy for dependency grammar (where the two objectives above are jointly optimized, as shown in Figure 1).Page 2, “Introduction”
- So we extract projected discrete dependency arc instances instead of treebank as training set for the projected grammar induction model.Page 4, “Bilingual Projection of Dependency Grammar”
- This section presents our bilingually-guided grammar induction model, which incorporates unsuperVised framework and bilingual projection model through a joint approach.Page 4, “Bilingually-Guided Dependency Grammar Induction”
- According to following observation: unsuperVised induction model mines underlying syntactic structure of the monolingual language, however, it is hard to find good grammar induction in the exponential parsing space; bilingual projection obtains relatively reliable syntactic knowledge of thePage 4, “Bilingually-Guided Dependency Grammar Induction”
- Based on the idea, we propose a novel strategy for training monolingual grammar induction model with the guidance of unsupervised and bilingually-projected dependency information.Page 4, “Bilingually-Guided Dependency Grammar Induction”
- Figure 1 outlines our bilingual-guided grammar induction process in its entirety.Page 4, “Bilingually-Guided Dependency Grammar Induction”
- Train the bilingually-guided grammar induction model by multi-objective optimization method with unsupervised objective and projection objective on treebank and projected arc instances respectively.Page 4, “Bilingually-Guided Dependency Grammar Induction”
- The results in Figure 3 prove that our unsupervised framework 04 = 1 can promote the grammar induction if it has a good start (well initialization), and it will be better once we incorporate the information from the projection side (04 = 0.9).Page 7, “Experiments”
- This paper presents a bilingually-guided strategy for automatic dependency grammar induction , which adopts an unsupervised skeleton and leverages the bilingually-projected dependency information during optimization.Page 8, “Conclusion and Future Work”
See all papers in Proc. ACL 2013 that mention grammar induction.
See all papers in Proc. ACL that mention grammar induction.
Back to top.
parsing model
Appears in 6 sentences as: parsing model (6)
In Bilingually-Guided Monolingual Dependency Grammar Induction
- We evaluate the final automatically-induced dependency parsing model on 5 languages.Page 2, “Introduction”
- And the framework of our unsupervised model builds a random treebank on the monolingual corpus firstly for initialization and trains a discriminative parsing model on it.Page 2, “Unsupervised Dependency Grammar Induction”
- Algorithm 1 outlines the unsupervised training in its entirety, where the treebank DE and unsupervised parsing model with A are updated iteratively.Page 3, “Unsupervised Dependency Grammar Induction”
- In line 1 we build a random treebank DE on the monolingual corpus, and then train the parsing model with it (line 2) through a training procedure train(-, which needs DE and 15F as classification instances.Page 3, “Unsupervised Dependency Grammar Induction”
- Use the parsing model to build new treebank on target language for next iteration.Page 4, “Bilingually-Guided Dependency Grammar Induction”
- With this approach, we can optimize the mixed parsing model by maximizing the objective in Formula (9).Page 5, “Bilingually-Guided Dependency Grammar Induction”
See all papers in Proc. ACL 2013 that mention parsing model.
See all papers in Proc. ACL that mention parsing model.
Back to top.
dependency tree
Appears in 5 sentences as: dependency tree (4) dependency trees (2)
In Bilingually-Guided Monolingual Dependency Grammar Induction
- The monolingual likelihood is similar to the optimization objectives of conventional unsupervised models, while the bilingually-projected likelihood is the product of the projected probabilities of dependency trees .Page 2, “Introduction”
- Z(d€ij) : 2 WM: )‘n ° f’n(d€ij 7 (2) y n Given a sentence E, parsing a dependency tree is to find a dependency tree D E with maximum probability PE:Page 3, “Unsupervised Dependency Grammar Induction”
- Figure 2: Projecting a Chinese dependency tree to English side according to DPA.Page 3, “Unsupervised Dependency Grammar Induction”
- From line 4-9, the objective is optimized with a generic optimization step in the subroutine climb(-, -, -, -, For each sentence we parse its dependency tree , and update the tree into the treebank (step 3).Page 5, “Bilingually-Guided Dependency Grammar Induction”
- The source sentences are then parsed by an implementation of 2nd-ordered MST model of McDonald and Pereira (2006), which is trained on dependency trees extracted from Penn Treebank.Page 6, “Experiments”
See all papers in Proc. ACL 2013 that mention dependency tree.
See all papers in Proc. ACL that mention dependency tree.
Back to top.
dependency parsing
Appears in 4 sentences as: dependency parser (1) dependency parsing (3)
In Bilingually-Guided Monolingual Dependency Grammar Induction
- In past decades supervised methods achieved the state-of-the-art in constituency parsing (Collins, 2003; Charniak and Johnson, 2005; Petrov et al., 2006) and dependency parsing (McDonald et al., 2005a; McDonald et al., 2006; Nivre et al., 2006; Nivre et al., 2007; K00 and Collins, 2010).Page 1, “Introduction”
- We evaluate the final automatically-induced dependency parsing model on 5 languages.Page 2, “Introduction”
- In the rest of the paper, we first describe the unsupervised dependency grammar induction framework in section 2 (where the unsupervised optimization objective is given), and introduce the bilingual projection method for dependency parsing in section 3 (where the projected optimization objective is given); Then in section 4 we present the bilingually-guided induction strategy for dependency grammar (where the two objectives above are jointly optimized, as shown in Figure 1).Page 2, “Introduction”
- In this section, we evaluate the performance of the MST dependency parser (McDonald et al., 2005b) which is trained by our bilingually-guided model on 5 languages.Page 6, “Experiments”
See all papers in Proc. ACL 2013 that mention dependency parsing.
See all papers in Proc. ACL that mention dependency parsing.
Back to top.
objective function
Appears in 4 sentences as: objective function (4) objective functions (1)
In Bilingually-Guided Monolingual Dependency Grammar Induction
- We select a simple classifier objective function as the unsupervised objective function which is instinctively in accordance with the parsing objective:Page 3, “Unsupervised Dependency Grammar Induction”
- In that case, we can use a single parameter 04 to control both weights for different objective functions .Page 5, “Bilingually-Guided Dependency Grammar Induction”
- When 04 = 1 it is the unsupervised objective function in Formula (6).Page 5, “Bilingually-Guided Dependency Grammar Induction”
- Contrary, if 04 = 0, it is the projection objective function (Formula (7)) for projected instances.Page 5, “Bilingually-Guided Dependency Grammar Induction”
See all papers in Proc. ACL 2013 that mention objective function.
See all papers in Proc. ACL that mention objective function.
Back to top.
word pair
Appears in 4 sentences as: Word Pair (1) word pair (3)
In Bilingually-Guided Monolingual Dependency Grammar Induction
- denotes the word pair dependency relationship (e;- —> 63-).Page 2, “Unsupervised Dependency Grammar Induction”
- Based on the features around deij, we can calculate the probability Pr(y|deij) that the word pair dew.Page 2, “Unsupervised Dependency Grammar Induction”
- where y is the category of the relationship of dew: y = + means it is the probability that the word pair deij can form a dependency arc and y = —means the contrary.Page 3, “Unsupervised Dependency Grammar Induction”
- The Word Pair Classification (WPC) method (J iang and Liu, 2010) modifies the DPA method and makes it more robust.Page 5, “Related work”
See all papers in Proc. ACL 2013 that mention word pair.
See all papers in Proc. ACL that mention word pair.
Back to top.
dependency relationship
Appears in 3 sentences as: dependency relationship (3) dependency relationships (1)
In Bilingually-Guided Monolingual Dependency Grammar Induction
- denotes the word pair dependency relationship (e;- —> 63-).Page 2, “Unsupervised Dependency Grammar Induction”
- Take Figure 2 as an example, following the Direct Projection Algorithm (DPA) (Hwa et a1., 2005) (Section 5), the dependency relationships between words can be directly projected from the sourcePage 3, “Bilingual Projection of Dependency Grammar”
- The dependency projection method DPA (Hwa et a1., 2005) based on Direct Correspondence Assumption (Hwa et a1., 2002) can be described as: if there is a pair of source words with a dependency relationship, the corresponding aligned words in target sentence can be considered as having the same dependency relationship equivalent-1y (e.g.Page 5, “Related work”
See all papers in Proc. ACL 2013 that mention dependency relationship.
See all papers in Proc. ACL that mention dependency relationship.
Back to top.
