NDLI: Using Publications and Domain Knowledge to Build Research Profiles: An Application in Automatic Reviewer Assignment

Content Provider	IEEE Xplore Digital Library
Author	Biswas, H.K. Hasan, M.
Copyright Year	2007
Description	Author affiliation: Shinawatra Univ., Bangkok (Biswas, H.K.; Hasan, M.)
Abstract	Peer-review has been a common practice for quality control in scholarly publications for decades. The ubiquity of the Internet and, subsequently, the availability of easy-to-use Web-based systems (both free and commercial) has made the peer-review process fast, cost-effective and convenient. In a typical scenario, authors upload papers online and manually assign topic-areas; reviewers also sign up by letting the system know about their area of expertise. A rudimentary Paper-Reviewer matching is usually performed by the system and validated by the Program-Chair (for conferences) or by the Editor-in-Chief (for journals). As argued in relevant literature, the peer-review process suffers from several flaws including author's or reviewer's bias in choosing topic-areas and expertise, as well as inter-reviewer agreement, etc. In this research, we explore automatic reviewer assignment for papers by solely considering the content of the papers and the true profile of the reviewers. In this research, we experimented with three approaches to calculate paper-reviewer relevance using the Vector Space Model. We used a set of 10 papers, 30 reviewers and the real paper-reviewer assignment information from a real-conference; and justifed the result of automatic paper-reviewer assignment based on the above three approaches. We noticed that the overlap between real-assignment and automatic-assignment is poor (with only 55-66% of the reviewers being in common). Such a result was not surprising to us, since we are aware that reviewers often express their frustrations claiming that some papers assigned them are not in line with their preferences and expertise. The data-set we used was rather small and suffered from data-sparseness problem and therefore we tried to analyze the automatic-assignment rationales through unbiased human judgment to identify the effect of the above-mentioned approaches in automatic reviewer assignment. We concluded that combining domain-knowledge with automatically extracted keywords (i.e., ontology-driven topic inference using automatically-extracted keywords) could potentially identify the most relevant candidate-reviewers for a paper.
Starting Page	82
Ending Page	86
File Size	1853758
Page Count	5
File Format	PDF
ISBN	9843233948
DOI	10.1109/ICICT.2007.375347
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2007-03-07
Publisher Place	Bangladesh
Access Restriction	Subscribed
Rights Holder	IIC, BUET
Subject Keyword	Ontologies Internet Humans Feature extraction Space technology Communications technology Electronic mail Automatic control Communication system control World Wide Web
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in