NDLI: Automatic system testing of programs without test oracles

Please wait, while we are loading the content...

Exception analysis and points-to analysis: better together

Advanced code coverage analysis using substring holes

Efficient mutation testing by checking invariant violations

HAMPI: a solver for string constraints

Identifying bug signatures using discriminative graph mining

Incremental covering array failure characterization in large configuration spaces

Loop-extended symbolic execution on binary programs

Testing CPU emulators

Analyzing singularity channel contracts

A formal analysis of requirements-based testing

Automatic mining of functionally equivalent code fragments via random testing

Detecting code clones in binary executables

Fault localization and repair for Java runtime exceptions

Automatic system testing of programs without test oracles

AVA: automated interpretation of dynamically detected anomalies

Specifying the worst case: orthogonal modeling of hardware errors

Identifying semantic differences in AspectJ programs

The influence of size and coverage on test suite effectiveness

A comparative study of programmer-written and automatically inferred contracts

Precise pointer reasoning for dynamic test generation

Memory slicing

Clustering test cases to achieve effective and scalable prioritisation incorporating expert knowledge

Penumbra: automatically identifying failure-relevant inputs using dynamic tainting

Precise interface identification to improve testing and analysis of web applications

Time-aware test-case prioritization using integer linear programming

Automatic system testing of programs without test oracles

Content Provider	ACM Digital Library
Author	Kaiser, Gail Shen, Kuang Murphy, Christian
Abstract	Metamorphic testing has been shown to be a simple yet effective technique in addressing the quality assurance of applications that do not have test oracles, i.e., for which it is difficult or impossible to know what the correct output should be for arbitrary input. In metamorphic testing, existing test case input is modified to produce new test cases in such a manner that, when given the new input, the application should produce an output that can easily be computed based on the original output. That is, if input x produces output f(x), then we create input x' such that we can predict f(x') based on f(x); if the application does not produce the expected output, then a defect must exist, and either f(x), or f(x') (or both) is wrong. In practice, however, metamorphic testing can be a manually intensive technique for all but the simplest cases. The transformation of input data can be laborious for large data sets, or practically impossible for input that is not in human-readable format. Similarly, comparing the outputs can be error-prone for large result sets, especially when slight variations in the results are not actually indicative of errors (i.e., are false positives), for instance when there is non-determinism in the application and multiple outputs can be considered correct. In this paper, we present an approach called Automated Metamorphic System Testing. This involves the automation of metamorphic testing at the system level by checking that the metamorphic properties of the entire application hold after its execution. The tester is able to easily set up and conduct metamorphic tests with little manual intervention, and testing can continue in the field with minimal impact on the user. Additionally, we present an approach called Heuristic Metamorphic Testing which seeks to reduce false positives and address some cases of non-determinism. We also describe an implementation framework called Amsterdam, and present the results of empirical studies in which we demonstrate the effectiveness of the technique on real-world programs without test oracles.
Starting Page	189
Ending Page	200
Page Count	12
File Format	PDF
ISBN	9781605583389
DOI	10.1145/1572272.1572295
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2009-07-19
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Software testing Metamorphic testing Oracle problem
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Techniques for testing scientific programs without an Oracle

Metamorphic Testing: A Simple Method for Alleviating the Test Oracle Problem

Techniques for testing scientific programs without an oracle

Metamorphic testing: a simple method for alleviating the test oracle problem

Using JML Runtime Assertion Checking to Automate Metamorphic Testing in Applications without Test Oracles

Backward-Slice-Based Statistical Fault Localization without Test Oracles

Metamorphic relations for programs without test oracles.

Using machine learning techniques to detect metamorphic relations for programs without test oracles

Automatic system testing of programs without test oracles.

Automatic system testing of programs without test oracles

Similar Documents

Techniques for testing scientific programs without an Oracle

Metamorphic Testing: A Simple Method for Alleviating the Test Oracle Problem

Techniques for testing scientific programs without an oracle

Metamorphic testing: a simple method for alleviating the test oracle problem

Using JML Runtime Assertion Checking to Automate Metamorphic Testing in Applications without Test Oracles

Backward-Slice-Based Statistical Fault Localization without Test Oracles

Metamorphic relations for programs without test oracles.

Using machine learning techniques to detect metamorphic relations for programs without test oracles

Automatic system testing of programs without test oracles.

Automatic system testing of programs without test oracles