NDLI: Expected reading effort in focused retrieval evaluation

Content Provider	Springer Nature Link
Author	Arvola, Paavo Kekäläinen, Jaana Junkkari, Marko
Copyright Year	2010
Abstract	This study introduces a novel framework for evaluating passage and XML retrieval. The framework focuses on a user’s effort to localize relevant content in a result document. Measuring the effort is based on a system guided reading order of documents. The effort is calculated as the quantity of text the user is expected to browse through. More specifically, this study seeks evaluation metrics for retrieval methods following a specific fetch and browse approach, where in the fetch phase documents are ranked in decreasing order according to their document score, like in document retrieval. In the browse phase, for each retrieved document, a set of non-overlapping passages representing the relevant text within the document is retrieved. In other words, the passages of the document are re-organized, so that the best matching passages are read first in sequential order. We introduce an application scenario motivating the framework, and propose sample metrics based on the framework. These metrics give a basis for the comparison of effectiveness between traditional document retrieval and passage/XML retrieval and illuminate the benefit of passage/XML retrieval.
Starting Page	460
Ending Page	484
Page Count	25
File Format	PDF
ISSN	13864564
Journal	Information Retrieval
Volume Number	13
Issue Number	5
e-ISSN	15737659
Language	English
Publisher	Springer Netherlands
Publisher Date	2010-05-06
Publisher Place	Dordrecht
Access Restriction	Subscribed
Subject Keyword	Passage retrieval XML retrieval Evaluation Metrics Small screen devices Pattern Recognition Data Structures, Cryptology and Information Theory Information Storage and Retrieval Data Mining and Knowledge Discovery Document Preparation and Text Processing
Content Type	Text
Resource Type	Article
Subject	Library and Information Sciences Information Systems

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Specificity aboutness in XML retrieval

Current research in focused retrieval and result aggregation

On information retrieval metrics designed for evaluation with incomplete relevance assessments

Utilizing passage-based language models for ad hoc document retrieval

Focused retrieval and result aggregation with political data

A comparison of extrinsic clustering evaluation metrics based on formal constraints

Information retrieval evaluation using test collections

A comparison of extrinsic clustering evaluation metrics based on formal constraints

Extended structural relevance framework: a framework for evaluating structured document retrieval

Expected reading effort in focused retrieval evaluation

Similar Documents

Specificity aboutness in XML retrieval

Current research in focused retrieval and result aggregation

On information retrieval metrics designed for evaluation with incomplete relevance assessments

Utilizing passage-based language models for ad hoc document retrieval

Focused retrieval and result aggregation with political data

A comparison of extrinsic clustering evaluation metrics based on formal constraints

Information retrieval evaluation using test collections

A comparison of extrinsic clustering evaluation metrics based on formal constraints

Extended structural relevance framework: a framework for evaluating structured document retrieval

Expected reading effort in focused retrieval evaluation