NDLI: Scoring method of a Situational Judgment Test: influence on internal consistency reliability, adverse impact and correlation with personality?

Content Provider	Springer Nature Link
Author	De Leng, W. E. Stegers Jager, K. M. Husbands, A. Dowell, J. S. Born, M. Ph. Themmen, A. P. N.
Copyright Year	2016
Abstract	Situational Judgment Tests (SJTs) are increasingly used for medical school selection. Scoring an SJT is more complicated than scoring a knowledge test, because there are no objectively correct answers. The scoring method of an SJT may influence the construct and concurrent validity and the adverse impact with respect to non-traditional students. Previous research has compared only a small number of scoring methods and has not studied the effect of scoring method on internal consistency reliability. This study compared 28 different scoring methods for a rating SJT on internal consistency reliability, adverse impact and correlation with personality. The scoring methods varied on four aspects: the way of controlling for systematic error, and the type of reference group, distance and central tendency statistic. All scoring methods were applied to a previously validated integrity-based SJT, administered to 931 medical school applicants. Internal consistency reliability varied between .33 and .73, which is likely explained by the dependence of coefficient alpha on the total score variance. All scoring methods led to significantly higher scores for the ethnic majority than for the non-Western minorities, with effect sizes ranging from 0.48 to 0.66. Eighteen scoring methods showed a significant small positive correlation with agreeableness. Four scoring methods showed a significant small positive correlation with conscientiousness. The way of controlling for systematic error was the most influential scoring method aspect. These results suggest that the increased use of SJTs for selection into medical school must be accompanied by a thorough examination of the scoring method to be used.
Starting Page	243
Ending Page	265
Page Count	23
File Format	PDF
ISSN	13824996
Journal	Advances in Health Sciences Education
Volume Number	22
Issue Number	2
e-ISSN	15731677
Language	English
Publisher	Springer Netherlands
Publisher Date	2016-10-18
Publisher Place	Dordrecht
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Situational Judgment Test Scoring method Medical school selection Internal consistency reliability Adverse impact Integrity Big Five Medical Education
Content Type	Text
Resource Type	Article
Subject	Education Medicine

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Scoring method of a Situational Judgment Test: influence on internal consistency reliability, adverse impact and correlation with personality?

Evaluation of Situational Judgment Tests in student selection in Indonesia and the impact on diversity issues

Impact of vocational interests, previous academic experience, gender and age on Situational Judgement Test performance

Influence of response instructions and response format on applicant perceptions of a situational judgement test for medical school selection

Impact of vocational interests, previous academic experience, gender and age on Situational Judgement Test performance

Do personality traits assessed on medical school admission predict exit performance? A UK-wide longitudinal cohort study

Future directions in personality, occupational and medical selection: myths, misunderstandings, measurement, and suggestions

Measuring Procedural Knowledge More Simply with a Single-Response Situational Judgment Test

Medical students’ Big Five Personality scores and the effects on the “selection process”

Scoring method of a Situational Judgment Test: influence on internal consistency reliability, adverse impact and correlation with personality?

Similar Documents

Scoring method of a Situational Judgment Test: influence on internal consistency reliability, adverse impact and correlation with personality?

Evaluation of Situational Judgment Tests in student selection in Indonesia and the impact on diversity issues

Impact of vocational interests, previous academic experience, gender and age on Situational Judgement Test performance

Influence of response instructions and response format on applicant perceptions of a situational judgement test for medical school selection

Impact of vocational interests, previous academic experience, gender and age on Situational Judgement Test performance

Do personality traits assessed on medical school admission predict exit performance? A UK-wide longitudinal cohort study

Future directions in personality, occupational and medical selection: myths, misunderstandings, measurement, and suggestions

Measuring Procedural Knowledge More Simply with a Single-Response Situational Judgment Test

Medical students’ Big Five Personality scores and the effects on the “selection process”

Scoring method of a Situational Judgment Test: influence on internal consistency reliability, adverse impact and correlation with personality?