NDLI: Towards Using Code Coverage Metrics for Performance Comparison on the Implementation Level

Please wait, while we are loading the content...

Case Studies from the Real World: The Importance of Measurement and Analysis in Building Better Systems

Maximum Likelihood Estimation of Closed Queueing Network Demands from Queue Length Data

A Cost/Benefit Approach to Performance Analysis

Automatic Performance Modelling from Application Performance Management (APM) Data: An Experience Report

Asking "What"?, Automating the "How"?: The Vision of Declarative Performance Engineering

Empirical Analysis of Performance Problems on Code Level

PROST: Predicting Resource Usages with Spatial and Temporal Dependencies

Microservices for Scalability: Keynote Talk Abstract

Performance Modeling of Maximal Sharing

Variations in CPU Power Consumption

A Resource Contention Analysis Framework for Diagnosis of Application Performance Anomalies in Consolidated Cloud Environments

Optimized eeeBond: Energy Efficiency with non-Proportional Router Network Interfaces

Communication Characterization and Optimization of Applications Using Topology-Aware Task Mapping on Large Supercomputers

Cloudy, Foggy and Misty Internet of Things

Efficient Tracing and Versatile Analysis of Lock Contention in Java Applications on the Virtual Machine Level

Optimizing the Performance-Related Configurations of Object-Relational Mapping Frameworks Using a Multi-Objective Genetic Algorithm

Building Custom, Efficient, and Accurate Memory Monitoring Tools for Java Applications

Automated Extraction of Network Traffic Models Suitable for Performance Simulation

Analyzing the Efficiency of a Green University Data Center

Predicting the System Performance by Combining Calibrated Performance Models of its Components: A Preliminary Study

Sustaining Runtime Performance while Incrementally Modernizing Transactional Monolithic Software towards Microservices

SPEC Research Group's Cloud Working Group: RG Cloud Group

End-to-End Java Security Performance Enhancements for Oracle SPARC Servers

Beyond Energy-Efficiency: Evaluating Green Datacenter Applications for Energy-Agility

Automatically Detecting "Excessive Dynamic Memory Allocations" Software Performance Anti-Pattern

Analysis of Overhead in Dynamic Java Performance Monitoring

Automated Parameterization of Performance Models from Measurements

Learning from Source Code History to Identify Performance Failures

Interconnect Emulator for Aiding Performance Analysis of Distributed Memory Applications

Towards Using Code Coverage Metrics for Performance Comparison on the Implementation Level

Integrating Faban with Docker for Performance Benchmarking

Accelerating The Optimal Trade-Off Circular Harmonic Function Filter Design on Multicore Systems

Tackling Latency via Replication in Distributed Systems

Efficient and Viable Handling of Large Object Traces

The Value of Variance

Incorporating Software Performance Engineering Methods and Practices into the Software Development Life Cycle

Resource and Performance Distribution Prediction for Large Scale Analytics Queries

Parallel Graph Processing: Prejudice and State of the Art

Performance Extrapolation of IO Intensive Workloads: Work in Progress

Which Cloud Auto-Scaler Should I Use for my Application?: Benchmarking Auto-Scaling Algorithms

Enhancing Rules For Cloud Resource Provisioning Via Learned Software Performance Models

Experimental Performance Evaluation of different Data Models for a Reflection Software Architecture over NoSQL Persistence Layers

BFT-Bench: A Framework to Evaluate BFT Protocols

Towards Performance and Scalability Analysis of Distributed Memory Programs on Large-Scale Clusters

Towards Using Code Coverage Metrics for Performance Comparison on the Implementation Level

Content Provider	ACM Digital Library
Author	Menninghaus, Mathias Pulvermüller, Elke
Abstract	The development process for new algorithms or data structures often begins with the analysis of benchmark results to identify the drawbacks of already existing implementations. Furthermore it ends with the comparison of old and new implementations by using one or more well established benchmark. But how relevant, reproducible, fair, verifiable and usable those benchmarks may be, they have certain drawbacks. On the one hand a new implementation may be biased to provide good results for a specific benchmark. On the other hand benchmarks are very general and often fail to identify the worst and best cases of a specific implementation. In this paper we present a new approach for the comparison of algorithms and data structures on the implementation level using code coverage. Our approach uses model checking and multi-objective evolutionary algorithms to create test cases with a high code coverage. It then executes each of the given implementations with each of the test cases in order to calculate a cross coverage. Using this it calculates a combined coverage and weighted performance where implementations, which are not fully covered by the test cases of the other implementations, are punished. These metrics can be used to compare the performance of several implementations on a much deeper level than traditional benchmarks and they incorporate worst, best and average cases in an equal manner. We demonstrate this approach by two example sets of algorithms and outline the next research steps required in this context along with the greatest risks and challenges.
Starting Page	101
Ending Page	104
Page Count	4
File Format	PDF
ISBN	9781450340809
DOI	10.1145/2851553.2858663
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2016-03-12
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Performance comparison Performance tests Algorithm engineering Test case generation
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

A software-based methodology for the generation of peripheral test sets based on high-level descriptions

On automatic generation of RTL validation test benches using circuit testing techniques

A genetic algorithm for regression test case prioritization using code coverage.

Towards a test automation framework for alloy

Test Suite generation using Memetic algorithm on adaptive local search

Comparative Performance Analysis of Additional Code Coverage Test Case Prioritization Using Genetic Algorithm

On the correlation between static measures and code coverage using evolutionary test case generation.

Towards a parallel approach for test data generation for branch coverage with genetic algorithm using the extended path prefix strategy

On Use of Coverage Metrics in Assessing Effectiveness of Combinatorial Test Designs

Towards Using Code Coverage Metrics for Performance Comparison on the Implementation Level

Similar Documents

A software-based methodology for the generation of peripheral test sets based on high-level descriptions

On automatic generation of RTL validation test benches using circuit testing techniques

A genetic algorithm for regression test case prioritization using code coverage.

Towards a test automation framework for alloy

Test Suite generation using Memetic algorithm on adaptive local search

Comparative Performance Analysis of Additional Code Coverage Test Case Prioritization Using Genetic Algorithm

On the correlation between static measures and code coverage using evolutionary test case generation.

Towards a parallel approach for test data generation for branch coverage with genetic algorithm using the extended path prefix strategy

On Use of Coverage Metrics in Assessing Effectiveness of Combinatorial Test Designs

Towards Using Code Coverage Metrics for Performance Comparison on the Implementation Level