NDLI: Evaluating Item-Sort Task Methods: The Presentation of a New Statistical Significance Formula and Methodological Best Practices

Content Provider	Springer Nature Link
Author	Howard, Matt C. Melloy, Robert C.
Copyright Year	2015
Abstract	An item-sort task is a common method to reduce over-representative item lists during the scale-creation process. The current article delineates the limitations and misapplications of the accepted statistical significance formula for item-sort tasks and proposes a new statistical significance formula with greater utility across a wider range of item-sort tasks.First, a simulation study compares the two formulas in an array of conditions that vary on sample size and number of assignment choices. Second, an empirical study compares the results of three separate item-sort tasks across the two formulas for statistical significance.In the empirical study, the proposed formula produces more correct retention decisions than the existing formula across all three item-sort tasks. In the simulation study, the proposed formula is more appropriate than the existing formula under most conditions. The two formulas function identically in item-sort tasks with only two assignment choices.Researchers could obtain erroneous results when misapplying the existing item-sort task statistical significance formula to cases with more than two assignment choices. The proposed formula corrects this limitation, ultimately providing accurate results more often than the existing formula. Applying the proposed formula could help future research and practice throughout the scale development process.Despite widespread use, few attempts have been made to improve scale-creation pretest methods, particularly item-sort tasks. The current study demonstrates that even conventional statistical methods are susceptible to misuse and misapplication, and future research could benefit from the reexamination of other common methods.
Starting Page	173
Ending Page	186
Page Count	14
File Format	PDF
ISSN	08893268
Journal	Journal of Business and Psychology
Volume Number	31
Issue Number	1
e-ISSN	1573353X
Language	English
Publisher	Springer US
Publisher Date	2015-03-28
Publisher Place	New York
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Item-sort tasks Measurement Scale creation Psychometrics Industrial, Organisational and Economic Psychology Community and Environmental Psychology Personality and Social Psychology Business and Management Social Sciences
Content Type	Text
Resource Type	Article
Subject	Business and International Management Applied Psychology

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Middle Response Scale Options are Inappropriate for Ideal Point Scales

Null Results in Assessing Survey Score Comparability: Illustrating Measurement Invariance Using Item Response Theory

Consequences of Ignoring Ideal Point Items for Applied Decisions and Criterion-Related Validity Estimates

The Effects of Unconsciously Derived Affect on Task Satisfaction and Performance

Unintended Consequences of EEO Enforcement Policies: Being Big is Worse than Being Bad

Demographic Question Placement: Effect on Item Response Rates and Means of a Veterans Health Administration Survey

Conceptual and Statistical Interactions: An Illustration with the AB5C and CWBs

Detecting Insufficient Effort Responding with an Infrequency Scale: Evaluating Validity and Participant Reactions

Proactive Personality at Work: Seeing More to Do and Doing More?

Evaluating Item-Sort Task Methods: The Presentation of a New Statistical Significance Formula and Methodological Best Practices

Similar Documents

Middle Response Scale Options are Inappropriate for Ideal Point Scales

Null Results in Assessing Survey Score Comparability: Illustrating Measurement Invariance Using Item Response Theory

Consequences of Ignoring Ideal Point Items for Applied Decisions and Criterion-Related Validity Estimates

The Effects of Unconsciously Derived Affect on Task Satisfaction and Performance

Unintended Consequences of EEO Enforcement Policies: Being Big is Worse than Being Bad

Demographic Question Placement: Effect on Item Response Rates and Means of a Veterans Health Administration Survey

Conceptual and Statistical Interactions: An Illustration with the AB5C and CWBs

Detecting Insufficient Effort Responding with an Infrequency Scale: Evaluating Validity and Participant Reactions

Proactive Personality at Work: Seeing More to Do and Doing More?

Evaluating Item-Sort Task Methods: The Presentation of a New Statistical Significance Formula and Methodological Best Practices