Article Structure
Abstract
We propose to directly measure the importance of queries in the source domain to the target domain where no rank labels of documents are available, which is referred to as query weighting.
Introduction
Learning to rank, which aims at ranking documents in terms of their relevance to user’s query, has been widely studied in machine learning and information retrieval communities (Herbrich et al., 2000; Fre-und et al., 2004; Burges et al., 2005; Yue et al., 2007; Cao et al., 2007; Liu, 2009).
Instance Weighting Scheme Review
The basic idea of instance weighting is to put larger weights on source instances which are more similar to target domain.
Query Weighting
In this section, we extend instance weighting to directly estimate query importance for more effective ranking model adaptation.
Ranking Model Adaptation via Query Weighting
To adapt the source ranking model to the target domain, we need to incorporate query weights into existing ranking algorithms.
Evaluation
We evaluated the proposed two query weighting methods on TREC-2003 and TREC-2004 web track datasets, which were released through LETOR3.0 as a benchmark collection for learning to rank by (Qin et al., 2010).
Related Work
Cross-domain knowledge transfer has became an important topic in machine learning and natural language processing (Ben-David et al., 2010; Jiang and Zhai, 2007; Blitzer et al., 2006; Daume III and Marcu, 2006).
Conclusion
We introduced two simple yet effective query weighting methods for ranking model adaptation.
Acknowledgement
P. Cai and A. Zhou are supported by NSFC (No.
Topics
feature vector
- The first compresses the query into a query feature vector, which aggregates all document instances in the same query, and then conducts query weighting based on the query feature vector .Page 1, “Abstract”
- Take Figure 2 as a toy example, where the document instance is represented as a feature vector with four features.Page 2, “Introduction”
- In this work, we present two simple but very effective approaches attempting to resolve the problem from distinct perspectives: (1) we compress each query into a query feature vector by aggregating all of its document instances, and then conduct query weighting on these query feature vectors ; (2) we measure the similarity between the source query and each target query one by one, and then combine these fine- grained similarity values to calculate its importance to the target domain.Page 3, “Introduction”
- The query can be compressed into a query feature vector , where each feature value is obtained by the aggregate of its corresponding features of all documents in the query.Page 3, “Query Weighting”
- We concatenate two types of aggregates to construct the query feature vector : the mean [i = fi ZlqzllPage 3, “Query Weighting”
- is the feature vector of document 2' and |q| denotes the number of documents in q .Page 3, “Query Weighting”
- From step 1 to 9, D; and D; are constructed using query feature vectors from source and target domains.Page 4, “Query Weighting”
- The distance of the query feature vector from H 87; are transformed to the probability P 6 D75) using a sigmoid function (Platt and Platt, 1999).Page 4, “Query Weighting”
- Although the query feature vector in algorithm 1 can approximate a query by aggregating its documents’ features, it potentially fails to capture important feature information due to the averaging effect during the aggregation.Page 4, “Query Weighting”
- For example, the merit of features in some influential documents may be canceled out in the mean-variance calculation, resulting in many distorted feature values in the query feature vector that hurts the accuracy of query classification hyperplane.Page 4, “Query Weighting”
- Specifically, after document feature aggregation, the number of query feature vectors in all adaptation tasks is no more than 150 in source and target domains.Page 7, “Evaluation”
See all papers in Proc. ACL 2011 that mention feature vector.
See all papers in Proc. ACL that mention feature vector.
Back to top.
fine-grained
- The second measures the similarity between the source query and each target query, and then combines these fine-grained similarity values for its importance estimation.Page 1, “Abstract”
- more precise measures of query similarity by utilizing the more fine-grained classification hyperplane for separating the queries of two domains.Page 5, “Query Weighting”
- By contrast, more accurate query weights can be achieved by the more fine-grained similarity measure between the source query and all target queries in algorithm 2.Page 7, “Evaluation”
- fine-grained similarity values.Page 9, “Evaluation”
- The second measures the similarity between a source query and each target query, and then combine the fine-grained similarity values to estimate its importance to target domain.Page 9, “Conclusion”
See all papers in Proc. ACL 2011 that mention fine-grained.
See all papers in Proc. ACL that mention fine-grained.
Back to top.
labeled data
- To alleviate the lack of training data in the target domain, many researchers have proposed to transfer ranking knowledge from the source domain with plenty of labeled data to the target domain where only a few or no labeled data is available, which is known as ranking model adaptation (Chen et al., 2008a; Chen et al., 2010; Chen et al., 2008b; Geng et al., 2009; Gao et al., 2009).Page 1, “Introduction”
- (J iang and Zhai, 2007) used a small number of labeled data from target domain to weight source instances.Page 3, “Instance Weighting Scheme Review”
- All ranking models above were trained only on source domain training data and the labeled data of target domain was just used for testing.Page 6, “Evaluation”
- In (Geng et al., 2009; Chen et al., 2008b), the parameters of ranking model trained on the source domain was adjusted with the small set of labeled data in the target domain.Page 9, “Related Work”
- al., 2008a) weighted source instances by using small amount of labeled data in the target domain.Page 9, “Related Work”
See all papers in Proc. ACL 2011 that mention labeled data.
See all papers in Proc. ACL that mention labeled data.
Back to top.
model trained
- This motivated the popular domain adaptation solution based on instance weighting, which assigns larger weights to those transferable instances so that the model trained on the source domain can adapt more effectively to the target domain (Jiang and Zhai, 2007).Page 1, “Introduction”
- Adaptation takes place when ranking tasks are performed by using the models trained on the domains in which they were originally defined to rank the documents in other domains.Page 6, “Evaluation”
- In (Geng et al., 2009; Chen et al., 2008b), the parameters of ranking model trained on the source domain was adjusted with the small set of labeled data in the target domain.Page 9, “Related Work”
See all papers in Proc. ACL 2011 that mention model trained.
See all papers in Proc. ACL that mention model trained.
Back to top.
significantly outperforms
- Adaptation experiments on LETOR3.0 data set demonstrate that query weighting significantly outperforms document instance weighting methods.Page 1, “Abstract”
- wise approach significantly outperformed pointwise approach, which takes each document instance as independent learning object, as well as pairwise approach, which concentrates learning on the order of a pair of documents (Liu, 2009).Page 2, “Introduction”
- We evaluated our approaches on LETOR3.0 dataset for ranking adaptation and found that: (l) the first method efficiently estimate query weights, and can outperform the document instance weighting but some information is lost during the aggregation; (2) the second method consistently and significantly outperforms document instance weighting.Page 9, “Conclusion”
See all papers in Proc. ACL 2011 that mention significantly outperforms.
See all papers in Proc. ACL that mention significantly outperforms.
Back to top.
