NDLI: Using the Crowd to Improve Search Result Ranking and the Search Experience

Content Provider	ACM Digital Library
Author	Teevan, Jaime Collins-thompson, Kevyn Kim, Yubin
Copyright Year	2016
Abstract	Despite technological advances, algorithmic search systems still have difficulty with complex or subtle information needs. For example, scenarios requiring deep semantic interpretation are a challenge for computers. People, on the other hand, are well suited to solving such problems. As a result, there is an opportunity for humans and computers to collaborate during the course of a search in a way that takes advantage of the unique abilities of each. While search tools that rely on human intervention will never be able to respond as quickly as current search engines do, recent research suggests that there are scenarios where a search engine could take more time if it resulted in a much better experience. This article explores how crowdsourcing can be used at query time to augment key stages of the search pipeline. We first explore the use of crowdsourcing to improve search result ranking. When the crowd is used to replace or augment traditional retrieval components such as query expansion and relevance scoring, we find that we can increase robustness against failure for query expansion and improve overall precision for results filtering. However, the gains that we observe are limited and unlikely to make up for the extra cost and time that the crowd requires. We then explore ways to incorporate the crowd into the search process that more drastically alter the overall experience. We find that using crowd workers to support rich query understanding and result processing appears to be a more worthwhile way to make use of the crowd during search. Our results confirm that crowdsourcing can positively impact the search experience but suggest that significant changes to the search process may be required for crowdsourcing to fulfill its potential in search systems.
Starting Page	1
Ending Page	24
Page Count	24
File Format	PDF
ISSN	21576904
e-ISSN	21576912
DOI	10.1145/2897368
Volume Number	7
Issue Number	4
Journal	ACM Transactions on Intelligent Systems and Technology (TIST)
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2016-07-12
Publisher Place	New York
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Slow search Crowdsourcing Information retrieval
Content Type	Text
Resource Type	Article
Subject	Artificial Intelligence Theoretical Computer Science

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

A Crowd-Powered System for Fashion Similarity Search

Crowdsourcing Without a Crowd: Reliable Online Species Identification Using Bayesian Models to Minimize Crowd Size

Parameterized Decay Model for Information Retrieval

Using targeted paraphrasing and monolingual crowdsourcing to improve translation

Aggregated search: A new information retrieval paradigm

Improving recency ranking using twitter data

Multimedia search reranking: A literature survey

Rapid Low-Cost Virtual Human Bootstrapping via the Crowd

Incentives for Effort in Crowdsourcing Using the Peer Truth Serum

Using the Crowd to Improve Search Result Ranking and the Search Experience

Similar Documents

A Crowd-Powered System for Fashion Similarity Search

Crowdsourcing Without a Crowd: Reliable Online Species Identification Using Bayesian Models to Minimize Crowd Size

Parameterized Decay Model for Information Retrieval

Using targeted paraphrasing and monolingual crowdsourcing to improve translation

Aggregated search: A new information retrieval paradigm

Improving recency ranking using twitter data

Multimedia search reranking: A literature survey

Rapid Low-Cost Virtual Human Bootstrapping via the Crowd

Incentives for Effort in Crowdsourcing Using the Peer Truth Serum

Using the Crowd to Improve Search Result Ranking and the Search Experience