Article Structure
Abstract
This paper presents a novel approach to automatic captioning of geo-tagged images by summarizing multiple web-documents that contain information related to an image’s location.
Introduction
The number of images tagged with location information on the web is growing rapidly, facilitated by the availability of GPS (Global Position System) equipped cameras and phones, as well as by the widespread use of online social sites.
Representing conceptual models 2.1 Object type corpora
We derive n-gram language and dependency pattern models using object type corpora made available to us by Aker and Gaizauskas.
Summarizer
We adopted the same overall approach to summarization used by Aker and Gaizauskas (2009) to generate the image descriptions.
Evaluation
To evaluate our approach we used two different assessment methods: ROUGE (Lin, 2004) and manual readability.
Related Work
Our approach has an advantage over related work in automatic image captioning in that it requires only GPS information associated with the image in order to generate captions.
Discussion and Conclusion
We have proposed a method by which dependency patterns extracted from corpora of descriptions of instances of particular obj ect types can be used in a multi-document summarizer to automatically generate image descriptions.
Acknowlegment
The research reported was funded by the TRIPOD project supported by the European Commission under the contract No.
Topics
language models
- Our results show that summaries biased by dependency pattern models lead to significantly higher ROUGE scores than both n-gram language models reported in previous work and also Wikipedia baseline summaries.Page 1, “Abstract”
- They also experimented with representing such conceptual models using n- gram language models derived from corpora consisting of collections of descriptions of instances of specific object types (e.g.Page 1, “Introduction”
- a corpus of descriptions of churches, a corpus of bridge descriptions, and so on) and reported results showing that incorporating such n-gram language models as a feature in a feature-based extractive summarizer improves the quality of automatically generated summaries.Page 1, “Introduction”
- The main weakness of n-gram language models is that they only capture very local information aboutshofitennsequencesandcannotnnxkfllong distance dependencies between terms.Page 1, “Introduction”
- If this information is expressed as in the first line of Table l, n-gram language models are likely toPage 1, “Introduction”
- However, if the type predication occurs with less commonly seen local context, as is the case for the object Rhine in the second row of Table l — most important rivers — n-gram language models may well be unable to identify it.Page 2, “Introduction”
- This intuition suggests that rather than representing object type conceptual models via corpus-derived language models as do Aker and Gaizauskas (2009), we do so instead using corpus-derived dependency patterns.Page 2, “Introduction”
- Since our work aims to extend the work of Aker and Gaizauskas (2009) we reproduce their experiments with n-gram language models in the current setting so as to permit accurate comparison.Page 2, “Introduction”
- 2.2 N-gram language modelsPage 2, “Representing conceptual models 2.1 Object type corpora”
- Aker and Gaizauskas (2009) experimented with uni-gram and bi-gram language models to capture the features commonly used when describing an object type and used these to bias the sentence selection of the summarizer towards the sentences that contain these features.Page 2, “Representing conceptual models 2.1 Object type corpora”
- As in Song and Croft (1999) they used their language models in a gener-Page 2, “Representing conceptual models 2.1 Object type corpora”
See all papers in Proc. ACL 2010 that mention language models.
See all papers in Proc. ACL that mention language models.
Back to top.
n-gram
- Our results show that summaries biased by dependency pattern models lead to significantly higher ROUGE scores than both n-gram language models reported in previous work and also Wikipedia baseline summaries.Page 1, “Abstract”
- a corpus of descriptions of churches, a corpus of bridge descriptions, and so on) and reported results showing that incorporating such n-gram language models as a feature in a feature-based extractive summarizer improves the quality of automatically generated summaries.Page 1, “Introduction”
- The main weakness of n-gram language models is that they only capture very local information aboutshofitennsequencesandcannotnnxkfllong distance dependencies between terms.Page 1, “Introduction”
- If this information is expressed as in the first line of Table l, n-gram language models are likely toPage 1, “Introduction”
- However, if the type predication occurs with less commonly seen local context, as is the case for the object Rhine in the second row of Table l — most important rivers — n-gram language models may well be unable to identify it.Page 2, “Introduction”
- Since our work aims to extend the work of Aker and Gaizauskas (2009) we reproduce their experiments with n-gram language models in the current setting so as to permit accurate comparison.Page 2, “Introduction”
- We derive n-gram language and dependency pattern models using object type corpora made available to us by Aker and Gaizauskas.Page 2, “Representing conceptual models 2.1 Object type corpora”
- 2.2 N-gram language modelsPage 2, “Representing conceptual models 2.1 Object type corpora”
- they calculate the probability that a sentence is generated based on a n-gram language model.Page 3, “Representing conceptual models 2.1 Object type corpora”
- o LMSim3 : The similarity of a sentence S to an n-gram language model LM (the probability that the sentence S is generated by LM).Page 4, “Summarizer”
- Our evaluations show that such an approach yields summaries which score more highly than an approach which uses a simpler representation of an object type model in the form of a n-gram language model.Page 8, “Discussion and Conclusion”
See all papers in Proc. ACL 2010 that mention n-gram.
See all papers in Proc. ACL that mention n-gram.
Back to top.
