NDLI: Improving quality of training data for learning to rank using click-through data

Please wait, while we are loading the content...

Leveraging temporal dynamics of document content in relevance ranking

Tagging human knowledge

Coupled semi-supervised learning for information extraction

Ranking with query-dependent loss for web search

Large scale query log analysis of re-finding

Learning influence probabilities in social networks

Learning similarity metrics for event identification in social media

A novel click model and its applications to online advertising

Revisiting globally sorted indexes for efficient document retrieval

SBotMiner: large scale search bot detection

Towards recency ranking in web search

Precomputing search features for fast and accurate query classification

Adapting information bottleneck method for automatic construction of domain-oriented sentiment lexicon

IntervalRank: isotonic regression with listwise and pairwise constraints

Anatomy of the long tail: ordinary people with extraordinary tastes

You are who you know: inferring user profiles in online social networks

Early online identification of attention gathering items in social media

Adaptive weighing designs for keyword value computation

Learning URL patterns for webpage de-duplication

Gathering and ranking photos of named entities with high precision, high recall, and diversity

Ranking mechanisms in twitter-like forums

I tag, you tag: translating tags for advanced user models

Data-oriented content query system: searching for data into text on the web

An optimization framework for query recommendation

Inferring search behaviors using partially observable Markov (POM) model

TwitterRank: finding topic-sensitive influential twitterers

Evolution of two-sided markets

Automatic generation of bid phrases for online advertising

On compressing the textual web

Boilerplate detection using shallow text features

Learning concept importance using a weighted dependence model

Pairwise interaction tensor factorization for personalized tag recommendation

Corroborating information from disagreeing views

Improving quality of training data for learning to rank using click-through data

Beyond DCG: user behavior as a predictor of a successful search

Folks in Folksonomies: social link prediction from shared metadata

Personalized click prediction in sponsored search

A sketch-based distance oracle for web-scale graphs

Query reformulation using anchor text

fLDA: matrix factorization through latent dirichlet allocation

A model to estimate intrinsic document relevance from the clickthrough logs of a web search engine

Measuring the reusability of test collections

GeoFolk: latent spatial semantics in web 2.0 social media

Improving ad relevance in sponsored search

Early exit optimizations for additive machine learned ranking systems

Improving quality of training data for learning to rank using click-through data

Content Provider	ACM Digital Library
Author	Chen, Chuanliang Li, Hang Xu, Gu Xu, Jingfang Abib, Elbio Renato Torres
Abstract	In information retrieval, relevance of documents with respect to queries is usually judged by humans, and used in evaluation and/or learning of ranking functions. Previous work has shown that certain level of noise in relevance judgments has little effect on evaluation, especially for comparison purposes. Recently learning to rank has become one of the major means to create ranking models in which the models are automatically learned from the data derived from a large number of relevance judgments. As far as we know, there was no previous work about quality of training data for learning to rank, and this paper tries to study the issue. Specifically, we address three problems. Firstly, we show that the quality of training data labeled by humans has critical impact on the performance of learning to rank algorithms. Secondly, we propose detecting relevance judgment errors using click-through data accumulated at a search engine. Two discriminative models, referred to as sequential dependency model and full dependency model, are proposed to make the detection. Both models consider the conditional dependency of relevance labels and thus are more powerful than the conditionally independent model previously proposed for other tasks. Finally, we verify that using training data in which the errors are detected and corrected by our method, we can improve the performance of learning to rank algorithms.
Starting Page	171
Ending Page	180
Page Count	10
File Format	PDF
ISBN	9781605588896
DOI	10.1145/1718487.1718509
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2010-02-04
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Judgment error correction Relevance label prediction Training data quality
Content Type	Text
Resource Type	Article

Central Library (ISO-9001:2015 Certified)
Indian Institute of Technology Kharagpur
Kharagpur, West Bengal, India | PIN - 721302

See location in the Map
03222 282435
Mail: support@ndl.gov.in

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in