NDLI: Tests for consistent measurement of external subjective software quality attributes

Content Provider	Springer Nature Link
Author	Moses, John Farrow, Malcolm
Copyright Year	2007
Abstract	One reason that researchers may wish to demonstrate that an external software quality attribute can be measured consistently is so that they can validate a prediction system for the attribute. However, attempts at validating prediction systems for external subjective quality attributes have tended to rely on experts indicating that the values provided by the prediction systems informally agree with the experts’ intuition about the attribute. These attempts are undertaken without a pre-defined scale on which it is known that the attribute can be measured consistently. Consequently, a valid unbiased estimate of the predictive capability of the prediction system cannot be given because the experts’ measurement process is not independent of the prediction system’s values. Usually, no justification is given for not checking to see if the experts can measure the attribute consistently. It seems to be assumed that: subjective measurement isn’t proper measurement or subjective measurement cannot be quantified or no one knows the true values of the attributes anyway and they cannot be estimated. However, even though the classification of software systems’ or software artefacts’ quality attributes is subjective, it is possible to quantify experts’ measurements in terms of conditional probabilities. It is then possible, using a statistical approach, to assess formally whether the experts’ measurements can be considered consistent. If the measurements are consistent, it is also possible to identify estimates of the true values, which are independent of the prediction system. These values can then be used to assess the predictive capability of the prediction system. In this paper we use Bayesian inference, Markov chain Monte Carlo simulation and missing data imputation to develop statistical tests for consistent measurement of subjective ordinal scale attributes.
Starting Page	261
Ending Page	287
Page Count	27
File Format	PDF
ISSN	13823256
Journal	Empirical Software Engineering
Volume Number	13
Issue Number	3
e-ISSN	15737616
Language	English
Publisher	Springer US
Publisher Date	2008-01-30
Publisher Place	Boston
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Subjective software quality attributes Conditional probability Consistent measurement Bayesian inference Multinomial distribution Error rates Distribution principle Minimum rejection principle Data imputation Programming Languages, Compilers, Interpreters Software Engineering/Programming and Operating Systems
Content Type	Text
Resource Type	Article
Subject	Software

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Should we try to measure software quality attributes directly?

Benchmarking quality measurement

Imputation techniques for multivariate missingness in software measurement data

Knowledge Requirements for Software Quality Measurement

Balancing Misclassification Rates in Classification-Tree Models of Software Quality

Predicting object-oriented class reuse-proneness using internal quality attributes

An Investigation of a Method for Identifying a Software Architecture Candidate with Respect to Quality Attributes

Determining the Distribution of Maintenance Categories: Survey versus Measurement

Software defect prediction using Bayesian networks

Tests for consistent measurement of external subjective software quality attributes

Similar Documents

Should we try to measure software quality attributes directly?

Benchmarking quality measurement

Imputation techniques for multivariate missingness in software measurement data

Knowledge Requirements for Software Quality Measurement

Balancing Misclassification Rates in Classification-Tree Models of Software Quality

Predicting object-oriented class reuse-proneness using internal quality attributes

An Investigation of a Method for Identifying a Software Architecture Candidate with Respect to Quality Attributes

Determining the Distribution of Maintenance Categories: Survey versus Measurement

Software defect prediction using Bayesian networks

Tests for consistent measurement of external subjective software quality attributes