Article Structure
Abstract
The ever growing amount of web images and their associated texts offers new opportunities for integrative models bridging natural language processing and computer vision.
Introduction
The vast number of online images with accompanying text raises hope for drawing synergistic connections between human language technologies and computer vision.
Sentence Generalization as Constraint Optimization
Casting the generalization task as visually-guided sentence compression with lightweight revisions, we formulate a constraint optimization problem that aims to maximize content selection and local linguistic fluency while satisfying constraints driven from dependency parse trees.
Code was provided by Deng et a1. (2012).
Related Work
Several recent studies presented approaches to automatic caption generation for images (e.g., Farhadi et al.
Conclusion
We have introduced the task of image caption generalization as a means to reduce noise in the parallel corpus of images and text.
Topics
parallel corpus
- Evaluation results show the intrinsic quality of the generalized captions and the extrinsic utility of the new image-text parallel corpus with respect to a concrete application of image caption transfer.Page 1, “Abstract”
- Evaluation results show both the intrinsic quality of the generalized captions and the extrinsic utility of the new image-text parallel corpus .Page 2, “Introduction”
- The new parallel corpus will be made publicly available.2Page 2, “Introduction”
- We evaluate the usefulness of our new image-text parallel corpus for automatic generation of image descriptions.Page 4, “Code was provided by Deng et a1. (2012).”
- Therefore, we also report scores based on semantic matching, which gives partial credits to word pairs based on their lexical similarity.5 The best performing approach with semantic matching is VISUAL (with LM = Image corpus), improving BLEU, Precision, F—score substantially over those of ORIG, demonstrating the extrinsic utility of our newly generated image-text parallel corpus in comparison to the original database.Page 5, “Code was provided by Deng et a1. (2012).”
- We have introduced the task of image caption generalization as a means to reduce noise in the parallel corpus of images and text.Page 5, “Conclusion”
See all papers in Proc. ACL 2013 that mention parallel corpus.
See all papers in Proc. ACL that mention parallel corpus.
Back to top.
BLEU
- To compute evaluation measures, we take the average scores of BLEU (1) and F-score (unigram-based with respect to content-words) over k = 5 candidate captions.Page 4, “Code was provided by Deng et a1. (2012).”
- Therefore, we also report scores based on semantic matching, which gives partial credits to word pairs based on their lexical similarity.5 The best performing approach with semantic matching is VISUAL (with LM = Image corpus), improving BLEU , Precision, F—score substantially over those of ORIG, demonstrating the extrinsic utility of our newly generated image-text parallel corpus in comparison to the original database.Page 5, “Code was provided by Deng et a1. (2012).”
- When computing BLEU with semantic matching, we look for the match with the highest similarity score among words that have not been matched before.Page 5, “Related Work”
See all papers in Proc. ACL 2013 that mention BLEU.
See all papers in Proc. ACL that mention BLEU.
Back to top.
sentence compression
- We address this challenge with contributions in two folds: first, we introduce the new task of image caption generalization, formulated as visually-guided sentence compression , and present an efficient algorithm based on dynamic beam search with dependency-based constraints.Page 1, “Abstract”
- Casting the generalization task as visually-guided sentence compression with lightweight revisions, we formulate a constraint optimization problem that aims to maximize content selection and local linguistic fluency while satisfying constraints driven from dependency parse trees.Page 2, “Sentence Generalization as Constraint Optimization”
- In comparison to prior work on sentence compression , our approach falls somewhere between unsupervised to distant-supervised approach (e. g., Turner and Charniak (2005), Filippova and Strube (2008)) in that there is not an in-domain training corpus to learn generalization patterns directly.Page 5, “Related Work”
See all papers in Proc. ACL 2013 that mention sentence compression.
See all papers in Proc. ACL that mention sentence compression.
Back to top.
