NDLI: Automated Assessment of the Quality of Peer Reviews using Natural Language Processing Techniques

Content Provider	Springer Nature Link
Author	Ramachandran, Lakshmi Gehringer, Edward F. Yadav, Ravi K.
Copyright Year	2017
Abstract	A review is textual feedback provided by a reviewer to the author of a submitted version. Peer reviews are used in academic publishing and in education to assess student work. While reviews are important to e-commerce sites like Amazon and e-bay, which use them to assess the quality of products and services, our work focuses on academic reviewing. We seek to help reviewers improve the quality of their reviews. One way to measure review quality is through metareview or review of reviews. We develop an automated metareview software that provides rapid feedback to reviewers on their assessment of authors’ submissions. To measure review quality, we employ metrics such as: review content type, review relevance, review’s coverage of a submission, review tone, review volume and review plagiarism (from the submission or from other reviews). We use natural language processing and machine-learning techniques to calculate these metrics. We summarize results from experiments to evaluate our review quality metrics: review content, relevance and coverage, and a study to analyze user perceptions of importance and usefulness of these metrics. Our approaches were evaluated on data from Expertiza and the Scaffolded Writing and Rewriting in the Discipline (SWoRD) project, which are two collaborative web-based learning applications.
Starting Page	534
Ending Page	581
Page Count	48
File Format	PDF
ISSN	15604292
Journal	International Journal of Artificial Intelligence in Education
Volume Number	27
Issue Number	3
e-ISSN	15604306
Language	English
Publisher	Springer New York
Publisher Date	2017-01-11
Publisher Institution	International AIED Society
Publisher Place	New York
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Educational Technology Intelligent tutoring systems User Interfaces and Human Computer Interaction Artificial Intelligence (incl. Robotics) Computers and Education Peer reviews Collaborative learning
Content Type	Text
Resource Type	Article
Subject	Education Computational Theory and Mathematics E-learning

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Adaptive Intelligent Support to Improve Peer Tutoring in Algebra

Web Delivery of Adaptive and Interactive Language Tutoring: Revisited

Stupid Tutoring Systems, Intelligent Humans

BEETLE II: Deep Natural Language Understanding and Automatic Feedback Generation for Intelligent Tutoring in Basic Electricity and Electronics

AutoTutor and Family: A Review of 17 Years of Natural Language Tutoring

An Educational System for Learning Search Algorithms and Automatically Assessing Student Performance

Evaluation Methods for Intelligent Tutoring Systems Revisited

Reflections on Andes’ Goal-free User Interface

Encouraging Student Reflection and Articulation Using a Learning Companion: A Commentary

Automated Assessment of the Quality of Peer Reviews using Natural Language Processing Techniques

Similar Documents

Adaptive Intelligent Support to Improve Peer Tutoring in Algebra

Web Delivery of Adaptive and Interactive Language Tutoring: Revisited

Stupid Tutoring Systems, Intelligent Humans

BEETLE II: Deep Natural Language Understanding and Automatic Feedback Generation for Intelligent Tutoring in Basic Electricity and Electronics

AutoTutor and Family: A Review of 17 Years of Natural Language Tutoring

An Educational System for Learning Search Algorithms and Automatically Assessing Student Performance

Evaluation Methods for Intelligent Tutoring Systems Revisited

Reflections on Andes’ Goal-free User Interface

Encouraging Student Reflection and Articulation Using a Learning Companion: A Commentary

Automated Assessment of the Quality of Peer Reviews using Natural Language Processing Techniques