NDLI: Why batch and user evaluations do not give the same results

Please wait, while we are loading the content...

Applying summarization techniques for term selection in relevance feedback

Temporal summaries of new topics

Generic text summarization using relevance measure and latent semantic analysis

A new approach to unsupervised text summarization

Vector-space ranking with effective early termination

Static index pruning for information retrieval systems

Rank-preserving two-level caching for scalable search engines

Using event segmentation to improve indexing of consumer photographs

Ranking retrieval systems without relevance judgments

Evaluation by highly relevant documents

Meta-scoring: automatically evaluating term weighting schemes in IR without precision-recall

Improving cross language retrieval with triangulated translation

Improving query translation for cross-language information retrieval using statistical models

Evaluating a probabilistic model for cross-lingual information retrieval

Document language models, query models, and risk minimization for information retrieval

Relevance based language models

A statistical learning learning model of text classification for support vector machines

A study of thresholding strategies for text categorization

On feature distributional clustering for text categorization

Iterative residual rescaling

Expressive retrieval from XML documents

XIRQL: a query language for information retrieval in XML documents

Empirical investigations on query modification using abductive explanations

Generic summaries for indexing in information retrieval

Automatic generation of concise summaries of spoken dialogues in unrestricted domains

Enhanced topic distillation using text, markup tags, and hyperlinks

Transparent Queries: investigation users' mental models of search engines

Why batch and user evaluations do not give the same results

Evaluating a content based image retrieval system

Evaluating topic-driven web crawlers

Effective site finding using link anchor information

Stable algorithms for link analysis

Modeling score distributions for combining the outputs of search engines

Models for metasearch

The score-distributional threshold optimization for adaptive binary classification tasks

Maximum likelihood estimation for filtering thresholds

A meta-learning approach for text categorization

Unsupervised and supervised clustering for topic tracking

Intelligent information triage

Discovering information flow suing high dimensional conceptual space

A study of smoothing methods for language models applied to Ad Hoc information retrieval

Topic segmentation with an aspect hidden Markov model

Finding topic words for hierarchical summarization

Exploiting redundancy in question answering

High performance question/answering

Searcher performance in question answering

Toward an improved concept-based information retrieval system

Metasearch consistency

Anchor text mining for translation extraction of query terms

Selecting expansion terms in automatic query expansion

An experimental framework for email categorization and management

Analyses of multiple-evidence combinations for retrieval strategies

Flexible pseudo-relevance feedback using optimization tables

Quantifying the utility of parallel corpora

Unitary operators for fast latent semantic indexing (FLSI)

Probabilistic combination of content and links

Structure and content-based segmentation of speech transcripts

Text summarization via hidden Markov models

Reading time, scrolling and interaction: exploring implicit sources of user preferences for relevance feedback

Interactive phrase browsing within compressed text

Query-biased web page summarisation: a task-oriented evaluation

Query expansion based on predictive algorithms for collaborative filtering

Query optimization for vector space problems

Generic topic segmentation of document texts

Towards the use of prosodic information for spoken document retrieval

A homogeneous framework to model relevance feedback

Combining semantic and syntactic document classifiers to improve first story detection

Browsing in a digital library collecting linearly arranged documents

Feature selection for polyphonic music retrieval

Automatic information extraction from web pages

Automatic web search query generation to create minority language corpora

Perpetual consistency improves image retrieval performance

Intelligent object-based image retrieval suing cluster-driven personal preference learning

Construction of a hierarchical classifier schema using a combination of text-based and image-based approaches

A method based on the chi-square test for document classification

Query clustering using content words and user feedback

Modifications of Kleinberg's HITS algorithm using matrix exponentiation and web log records

Cite me, cite my references?: (Scholarly use of the ACM SIGIR proceedings based on two citation indexes)

iFind: a web image search engine

Building interoperable digital library services: MARIAN, open archives, and the NDLTD

AUTINDEX: an automatic multilingual indexing system

Does visualization improve our ability to find and learn from internet based information?

The HySpirit retrieval platform

Distributed resource discovery and structured data searching with Chesire II

Searching the deep web: distributed explorit directed query applications

CROWSE: a system for organizing repositories and web search results

MS read: user modeling in the web environment

Why batch and user evaluations do not give the same results

Content Provider	ACM Digital Library
Author	Hersh, William Turpin, Andrew H.
Abstract	Much system-oriented evaluation of information retrieval systems has used the Cranfield approach based upon queries run against test collections in a batch mode. Some researchers have questioned whether this approach can be applied to the real world, but little data exists for or against that assertion. We have studied this question in the context of the TREC Interactive Track. Previous results demonstrated that improved performance as measured by relevance-based metrics in batch studies did not correspond with the results of outcomes based on real user searching tasks. The experiments in this paper analyzed those results to determine why this occurred. Our assessment showed that while the queries entered by real users into systems yielding better results in batch studies gave comparable gains in ranking of relevant documents for those users, they did not translate into better performance on specific tasks. This was most likely due to users being able to adequately find and utilize relevant documents ranked further down the output list.
Starting Page	225
Ending Page	231
Page Count	7
File Format	PDF
ISBN	1581133316
DOI	10.1145/383952.383992
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2001-09-01
Publisher Place	New York
Access Restriction	Subscribed
Content Type	Text
Resource Type	Article

Central Library (ISO-9001:2015 Certified)
Indian Institute of Technology Kharagpur
Kharagpur, West Bengal, India | PIN - 721302

See location in the Map
03222 282435
Mail: support@ndl.gov.in

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in