NDLI: User performance versus precision measures for simple search tasks

Please wait, while we are loading the content...

Learning user interaction models for predicting web search result preferences

Contextual search and name disambiguation in email using graphs

Spoken document retrieval from call-center conversations

AggregateRank: bringing order to web sites

The role of knowledge in conceptual retrieval: a study in the domain of clinical medicine

On-line spam filter fusion

Using web-graph distance for relevance feedback in web search

Context-sensitive semantic smoothing for the language modeling approach to genomic IR

A study of statistical models for query translation: finding a good unit of translation

Probabilistic model for definitional question answering

Latent semantic analysis for multiple-type interrelated data objects

Evaluation in (XML) information retrieval: expected precision-recall with user modelling (EPRUM)

Finding near-duplicate web pages: a large-scale evaluation of algorithms

Capturing collection size for distributed non-cooperative retrieval

Load balancing for term-distributed parallel retrieval

Mining dependency relations for query expansion in passage retrieval

Document clustering with prior knowledge

Less is more: probabilistic models for retrieving fewer relevant documents

Elicitation of term relevance feedback: an investigation of term source and context

Large scale semi-supervised linear SVMs

Unifying user-based and item-based collaborative filtering approaches by similarity fusion

Evaluating evaluation metrics based on the bootstrap

Learning to advertise

A compositional context sensitive multi-document summarizer: exploring the factors that influence summarization

Clustering of search results using temporal attributes

The TIJAH XML information retrieval system

Quantum haystacks

User performance versus precision measures for simple search tasks

Thread detection in dynamic text message streams

Towards efficient automated singer identification in large music databases

Respect my authority!: HITS without hyperlinks, utilizing cluster-based language models

A parallel derivation of probabilistic information retrieval models

Building bridges for web query classification

Improving the estimation of relevance models using large external corpora

LDA-based document models for ad-hoc retrieval

Combining bidirectional translation and synonymy for cross-language information retrieval

Answering complex questions with random walk models

Identifying comparative sentences in text documents

Minimal test collections for retrieval evaluation

Structure-driven crawler generation by example

Probabilistic latent query analysis for combining multiple retrieval sources

Hybrid index maintenance for growing text collections

What makes a query difficult?

Text clustering with extended user feedback

High accuracy retrieval with multiple nested ranker

Find-similar: similarity browsing as a search tool

Graph-based text classification: learn from your neighbors

Personalized recommendation driven by information flow

Statistical precision of information retrieval evaluation

Getting work done on the web: supporting transactional queries

Information graphics: an untapped resource for digital libraries

A complex document information processing prototype

A location annotation system for personal photos

Social networks, incentives, and search

Improving web search ranking by incorporating user behavior information

Formal models for expert finding in enterprise corpora

Music structure based vector space retrieval

Topical link analysis for web search

Semantic term matching in axiomatic approaches to information retrieval

ProbFuse: a probabilistic approach to data fusion

Regularized estimation of mixture models for robust pseudo-relevance feedback

Adapting ranking SVM to document retrieval

A framework to predict the quality of answers with non-textual features

Tackling concept drift by temporal inductive transfer

Dynamic test collections: measuring search effectiveness on the live web

Building implicit links from content for forum search

User modeling for full-text federated search in peer-to-peer networks

Type less, find more: fast autocompletion search with a succinct index

On ranking the effectiveness of searches

Near-duplicate detection by instance-level constrained clustering

Semantic search via XML fragments: a high-precision approach to IR

Exploring the limits of single-iteration clarification dialogs

Constructing informative prior distributions from domain knowledge in text classification

Analysis of a low-dimensional linear model under recommendation attacks

A statistical method for system evaluation using incomplete judgments

You are what you say: privacy risks of public mentions

News to go: hierarchical text summarization for mobile devices

Inferring document relevance via average precision

Appraisal navigator

Information retrieval at Boeing: plans and successes

Generalizing PageRank: damping functions for link-based ranking algorithms

Distributed query sampling: a quality-conscious approach

Pruned query evaluation using pre-computed impacts

Automatic construction of known-item finding test beds

A platform for Okapi-based contextual information retrieval

Adaptive query-based sampling for distributed IR

Project contexts to situate personal information

PENG: integrated search of distributed news archives

Cheshire3: retrieving from tera-scale grid-based digital libraries

Examining assessor attributes at HARD 2005

DeWild: a tool for searching the web using wild cards

User expectations from XML element retrieval

Searching for expertise using the terrier platform

Theoretical benchmarks of XML retrieval

DiLight: an ontology-based information access system for e-learning environments

Question classification with log-linear models

Supporting semantic visual feature browsing in contentbased video retrieval

Community-based snippet-indexes for pseudo-anonymous personalization in web search

MathFind: a math-aware search engine

Bias and the limits of pooling

Term proximity scoring for ad-hoc retrieval on very large text collections

An exploratory web log study of multitasking

Tensor space model for document analysis

First large-scale information retrieval experiments on turkish texts

Learning a ranking from pairwise preferences

Automated performance assessment in interactive QA

Stylistic text segmentation

On hierarchical web catalog integration with conceptual relationships in thesaurus

Rpref: a generalization of Bpref towards graded relevance judgments

A new web page summarization method

NMF and PLSI: equivalence and a hybrid algorithm

Using historical data to enhance rank aggregation

Enterprise search behaviour of software engineers

Evaluating sources of query expansion terms

Comparing two blind relevance feedback techniques

Information retrieval with commonsense knowledge

Refining hierarchical taxonomy structure via semi-supervised learning

Quantative analysis of the impact of judging inconsistency on the performance of relevance feedback

Swordfish: an unsupervised Ngram based approach to morphological analysis

Authorship attribution with thousands of candidate authors

Simple questions to improve pseudo-relevance feedback results

Is XML retrieval meaningful to users?: searcher preferences for full documents vs. elements

Building a test collection for complex document information processing

Enhancing topic tracking with temporal information

A comparative study of the effect of search feature design on user experience in digital libraries (DLs)

Representing clusters for retrieval

One-sided measures for evaluating ranked retrieval effectiveness with spontaneous conversational speech

Combining fields in known-item email search

Improving QA retrieval using document priors

Content-based video retrieval: does video's semantic visual feature matter?

Action modeling: language models that predict query behavior

A method of rating the credibility of news documents on the web

An analysis of the coupling between training set and neighborhood sizes for the kNN classifier

Fact-focused novelty detection: a feasibility study

Unity: relevance feedback using user query logs

Improving personalized web search using result diversification

Using small XML elements to support relevance

Give me just one highly relevant document: P-measure

Feature diversity in cluster ensembles for robust document clustering

Lightening the load of document smoothing for better language modeling retrieval

The effect of OCR errors on stylistic text classification

History repeats itself: repeat queries in Yahoo's logs

Early precision measures: implications from the downside of blind feedback

An experimental study on automatically labeling hierarchical clusters using statistical features

Strict and vague interpretation of XML-retrieval queries

Why structural hints in queries do not help XML-retrieval

Searching the web using composed pages

A study of real-time query expansion effectiveness

A graph-based framework for relation propagation and its application to multi-label learning

Measuring similarity of semi-structured documents with context weights

Incorporating query difference for learning retrieval functions in information retrieval

Concept-based biomedical text retrieval

User performance versus precision measures for simple search tasks

Content Provider	ACM Digital Library
Author	Scholer, Falk Turpin, Andrew
Abstract	Several recent studies have demonstrated that the type of improvements in information retrieval system effectiveness reported in forums such as SIGIR and TREC do not translate into a benefit for users. Two of the studies used an instance recall task, and a third used a question answering task, so perhaps it is unsurprising that the precision based measures of IR system effectiveness on one-shot query evaluation do not correlate with user performance on these tasks. In this study, we evaluate two different information retrieval tasks on TREC Web-track data: a precision-based user task, measured by the length of time that users need to find a single document that is relevant to a TREC topic; and, a simple recall-based task, represented by the total number of relevant documents that users can identify within five minutes. Users employ search engines with controlled mean average precision (MAP) of between 55% and 95%. Our results show that there is no significant relationship between system effectiveness measured by MAP and the precision-based task. A significant, but weak relationship is present for the precision at one document returned metric. A weak relationship is present between MAP and the simple recall-based task.
Starting Page	11
Ending Page	18
Page Count	8
File Format	PDF
ISBN	1595933697
DOI	10.1145/1148170.1148176
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2006-08-06
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Information retrieval evaluation User study Search engines
Content Type	Text
Resource Type	Article

Central Library (ISO-9001:2015 Certified)
Indian Institute of Technology Kharagpur
Kharagpur, West Bengal, India | PIN - 721302

See location in the Map
03222 282435
Mail: support@ndl.gov.in

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in