NDLI: A Topical Approach to Retrievability Bias Estimation

Please wait, while we are loading the content...

Advances in Formal Models of Search and Search Behaviour

Exploiting the Bipartite Structure of Entity Grids for Document Coherence and Retrieval

Total Recall: Blue Sky on Mars

The Effect of Document Order and Topic Difficulty on Assessor Agreement

A Simple and Effective Approach to Score Standardisation

Estimating Embedding Vectors for Queries

Learning to Rank User Queries to Detect Search Tasks

On Horizontal and Vertical Separation in Hierarchical Text Classification

Query Anchoring Using Discriminative Query Models

Exploring Urban Lifestyles Using a Nonparametric Temporal Graphical Model

A Utility Maximization Framework for Privacy Preservation of User Generated Content

Utilizing Knowledge Bases in Text-centric Information Retrieval

Efficient and Effective Higher Order Proximity Modeling

Classifying User Search Intents for Query Auto-Completion

A Reproducibility Study of Information Retrieval Models

The Impact of Fixed-Cost Pooling Strategies on Test Collection Bias

Analysis of the Paragraph Vector Model for Information Retrieval

Fast Feature Selection for Learning to Rank

From "More Like This" to "Better Than This"

Rank-at-a-Time Query Processing

EventMiner: Mining Events from Annotated Documents

Joint Estimation of Topics and Hashtag Relevance in Cross-Lingual Tweets

Collaborative Information Retrieval: Frameworks, Theoretical Models, and Emerging Topics

PDF: A Probabilistic Data Fusion Framework for Retrieval and Ranking

An Analysis of the Cost and Benefit of Search Interactions

Power Analysis for Interleaving Experiments by Means of Offline Evaluation

Unbiased Comparative Evaluation of Ranking Functions

End to End Long Short Term Memory Networks for Non-Factoid Question Answering

A Unified Energy-based Framework for Learning to Rank

Understanding the Message of Images with Knowledge Base Traversals

A Study of Document Expansion using Translation Models and Dimensionality Reduction Methods

Who Wants to Join Me?: Companion Recommendation in Location Based Social Networks

Temporal Query Expansion Using a Continuous Hidden Markov Model

Topic Set Size Design and Power Analysis in Practice

Learning to Rank with Labeled Features

Lexical Query Modeling in Session Search

Retrievability in API-Based "Evaluation as a Service"

A Topical Approach to Retrievability Bias Estimation

Embedding-based Query Language Models

Bag-of-Entities Representation for Ranking

Exploiting Entity Linking in Queries for Entity Retrieval

Estimating Retrieval Performance Bound for Single Term Queries

Nearest Neighbour based Transformation Functions for Text Classification: A Case Study with StackOverflow

Optimization Method for Weighting Explicit and Latent Concepts in Clinical Decision Support Queries

Cross-Language Microblog Retrieval using Latent Semantic Modeling

A Topical Approach to Retrievability Bias Estimation

Content Provider	ACM Digital Library
Author	Azzopardi, Leif Wilkie, Colin
Abstract	Retrievability is an independent evaluation measure that offers insights to an aspect of retrieval systems that performance and efficiency measures do not. Retrievability is often used to calculate the retrievability bias, an indication of how accessible a system makes all the documents in a collection. Generally, computing the retrievability bias of a system requires a colossal number of queries to be issued for the system to gain an accurate estimate of the bias. However, it is often the case that the accuracy of the estimate is not of importance, but the relationship between the estimate of bias and performance when tuning a systems parameters. As such, reaching a stable estimation of bias for the system is more important than getting very accurate retrievability scores for individual documents. This work explores the idea of using topical subsets of the collection for query generation and bias estimation to form a local estimate of bias which correlates with the global estimate of retrievability bias. By using topical subsets, it would be possible to reduce the volume of queries required to reach an accurate estimate of retrievability bias, reducing the time and resources required to perform a retrievability analysis. Findings suggest that this is a viable approach to estimating retrievability bias and that the number of queries required can be reduced to less than a quarter of what was previously thought necessary.
Starting Page	119
Ending Page	122
Page Count	4
File Format	PDF
ISBN	9781450344975
DOI	10.1145/2970398.2970437
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2016-09-12
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Evaluation Bias Retrievability
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Retrievability: An Independent Evaluation Measure

Query Length, Retrievability Bias and Performance

A Retrievability Analysis: Exploring the Relationship Between Retrieval Bias and Retrieval Performance

Querylog-based Assessment of Retrievability Bias in a Large Newspaper Corpus

The retrievability of documents

Theory of Retrieval: The Retrievability of Information

Retrievability: an evaluation measure for higher order information access tasks

An Improved Retrievability-Based Cluster-Resampling Approach for Pseudo Relevance Feedback

Search engine predilection towards news media providers

A Topical Approach to Retrievability Bias Estimation

Similar Documents

Retrievability: An Independent Evaluation Measure

Query Length, Retrievability Bias and Performance

A Retrievability Analysis: Exploring the Relationship Between Retrieval Bias and Retrieval Performance

Querylog-based Assessment of Retrievability Bias in a Large Newspaper Corpus

The retrievability of documents

Theory of Retrieval: The Retrievability of Information

Retrievability: an evaluation measure for higher order information access tasks

An Improved Retrievability-Based Cluster-Resampling Approach for Pseudo Relevance Feedback

Search engine predilection towards news media providers

A Topical Approach to Retrievability Bias Estimation