NDLI: Brittle metrics in operating systems research

Content Provider	IEEE Xplore Digital Library
Author	Mogul, J.C.
Copyright Year	1999
Description	Author affiliation: Western Res. Lab., Compaq Comput. Corp., Palo Alto, CA, USA (Mogul, J.C.)
Abstract	Most operating systems research publications make claims about performance. We expect these performance claims to be both repeatable and relevant to important applications. We also expect them to be comparable to similar claims made in other papers. This implies the need for realistic and widely-used benchmarks. Often, however, no such benchmark exists. The problem is especially acute in application areas with significant external latencies (such as Internet servers and file systems). Sometimes this leads researchers to measure only what is easily measured. Sometimes it leads to the naive use of unrealistic benchmarks, causing research to be diverted from solving actual problems. A serious project to put operating systems research on a sound quantitative basis requires us to make an explicit effort to develop repeatable, comparable and realistic metrics for evaluating operating systems research. We must also develop reliable techniques for using benchmark results to predict real-world performance.
Starting Page	90
Ending Page	95
File Size	29556
Page Count	6
File Format	PDF
ISBN	0769502377
DOI	10.1109/HOTOS.1999.798383
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	1999-03-30
Publisher Place	USA
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Operating systems Benchmark testing Computer architecture Laboratories Application software Delay Internet File servers Web server File systems
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in