Loading...
Please wait, while we are loading the content...
Capturing the essence of benchmarks: a new approach to benchmark synthesis.
| Content Provider | CiteSeerX |
|---|---|
| Author | Bell, Robert H. John, Lizy K. |
| Abstract | In the nineteen-eighties, synthetic workloads such as Whetstone and Dhrystone fell out of favor as benchmarks of computer performance as they became less and less representative of the performance of continuouslyevolving applications and language features. Hand-coded synthetic benchmarks are difficult to develop and maintain, and they are subject to compiler optimizations that often eliminate code meant to make a significant contribution to performance. We present a new automatic benchmark synthesis methodology to address these problems. By focusing on major workload characteristics, we are able to create (synthesize) representative programs with significantly reduced runtimes. Given any application, we are able to characterize it and generate a synthetic version of the application with approximately the same instruction stream characteristics. We synthesize representative versions of the SPEC95, SPEC2000, and TPC-C benchmarks, compile and execute them and obtain an average IPC within 3.9%, 2.4 % and 6.4%, respectively, of the average IPC of the original benchmarks with remarkably similar instruction mix and cache access characteristics, dispatch window occupancies, and dependency characteristics in most cases. Also, the changes in IPC due to design changes are found to be proportional to the changes in IPC in the original applications. At the same time, the synthetic benchmarks execute more than three orders of magnitude faster than the original applications. |
| File Format | |
| Access Restriction | Open |
| Subject Keyword | Representative Program Major Workload Characteristic Synthetic Workload New Automatic Benchmark Synthesis Methodology Tpc-c Benchmark Computer Performance Benchmark Synthesis Hand-coded Synthetic Benchmark Average Ipc Instruction Stream Characteristic Dispatch Window Occupancy Synthetic Version New Approach Original Benchmark Synthetic Benchmark Original Application Language Feature Significant Contribution Similar Instruction Mix Cache Access Characteristic Dependency Characteristic Representative Version |
| Content Type | Text |
