NDLI: Reducing Masking Effects in CombinatorialInteraction Testing: A Feedback DrivenAdaptive Approach

Please wait, while we are loading the content...

Reducing Masking Effects in CombinatorialInteraction Testing: A Feedback DrivenAdaptive Approach

Content Provider	IEEE Xplore Digital Library
Author	Yilmaz, C. Dumlu, E. Cohen, M.B. Porter, A.
Copyright Year	1976
Abstract	The configuration spaces of modern software systems are too large to test exhaustively. Combinatorial interaction testing (CIT) approaches, such as covering arrays, systematically sample the configuration space and test only the selected configurations. The basic justification for CIT approaches is that they can cost-effectively exercise all system behaviors caused by the settings of t or fewer options. We conjecture, however, that in practice some of these behaviors are not actually tested because of unanticipated masking effects - test case failures that perturb system execution so as to prevent some behaviors from being exercised. While prior research has identified this problem, most solutions require knowing the masking effects a priori. In practice this is impractical, if not impossible. In this work, we reduce the harmful consequences of masking effects. First we define a novel interaction testing criterion, which aims to ensure that each test case has a fair chance to test all valid t-way combinations of option settings. We then introduce a feedback driven adaptive combinatorial testing process (FDA-CIT) to materialize this criterion in practice. At each iteration of FDA-CIT, we detect potential masking effects, heuristically isolate their likely causes (i.e., fault characterization), and then generate new samples that allow previously masked combinations to be tested in configurations that avoid the likely failure causes. The iterations end when the new interaction testing criterion has been satisfied. This paper compares two different fault characterization approaches - an integral part of the proposed approach, and empirically assesses their effectiveness and efficiency in removing masking effects on two widely used open source software systems. It also compares FDA-CIT against error locating arrays, a state of the art approach for detecting and locating failures. Furthermore, the scalability of the proposed approach is evaluated by comparing it with perfect test scenarios, in which all masking effects are known a priori. Our results suggest that masking effects do exist in practice, and that our approach provides a promising and efficient way to work around them, without requiring that masking effects be known a priori.
Sponsorship	IEEE Computer Society Technical Council on Software Engineering Association for Computing Machinery (ACM) IEEE Computer Society
Starting Page	43
Ending Page	66
Page Count	24
File Size	4122421
File Format	PDF
ISSN	00985589
Volume Number	40
Issue Number	1
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2014-01-01
Publisher Place	U.S.A.
Access Restriction	One Nation One Subscription (ONOS)
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Testing Adaptive arrays Educational institutions Scalability Servers Electronic mail Software systems software quality assurance Combinatorial testing adaptive testing covering arrays
Content Type	Text
Resource Type	Article
Subject	Software

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Test Case-Aware Combinatorial Interaction Testing

Covering arrays for efficient fault characterization in complex configuration spaces

Reducing Feedback Delay of Software Development Tools via Continuous Analysis

Practical Combinatorial Testing: Beyond Pairwise

A Taxonomy of Buffer Overflow Characteristics

A Discrete Particle Swarm Optimization for Covering Array Generation

Moving Forward with Combinatorial Interaction Testing

Developing a Single Model and Test Prioritization Strategies for Event-Driven Software

Tracking your e-mail

Reducing Masking Effects in CombinatorialInteraction Testing: A Feedback DrivenAdaptive Approach

Similar Documents

Test Case-Aware Combinatorial Interaction Testing

Covering arrays for efficient fault characterization in complex configuration spaces

Reducing Feedback Delay of Software Development Tools via Continuous Analysis

Practical Combinatorial Testing: Beyond Pairwise

A Taxonomy of Buffer Overflow Characteristics

A Discrete Particle Swarm Optimization for Covering Array Generation

Moving Forward with Combinatorial Interaction Testing

Developing a Single Model and Test Prioritization Strategies for Event-Driven Software

Tracking your e-mail

Reducing Masking Effects in CombinatorialInteraction Testing: A Feedback DrivenAdaptive Approach