NDLI: Tales of the Tail: Hardware, OS, and Application-level Sources of Tail Latency

Please wait, while we are loading the content...

Automating Distributed Partial Aggregation

Reservation-based Scheduling: If You're Late Don't Blame Us!

Transparent, Live Migration of a Software-Defined Network

GentleRain: Cheap and Scalable Causal Consistency with Physical Clocks

Exploiting iterative-ness for parallel ML computations

Tachyon: Reliable, Memory Speed Storage for Cluster Computing Frameworks

What Bugs Live in the Cloud? A Study of 3000+ Issues in Cloud Systems

PerfScope: Practical Online Server Performance Bug Inference in Production Cloud Computing Infrastructures

Tales of the Tail: Hardware, OS, and Application-level Sources of Tail Latency

On the Control Plane of a Self-service Cloud Platform

DIMMer: A case for turning off DIMMs in clouds

Hybris: Robust Hybrid Cloud Storage

The Case for Drill-Ready Cloud Computing

Merlin: Application- and Platform-aware Resource Allocation in Consolidated Server Systems

DaRPC: Data Center RPC

Adaptive Stream Processing using Dynamic Batch Sizing

ATAC: Ambient Temperature-Aware Capping for Power Efficient Datacenters

Connected Components in MapReduce and Beyond

Vanguard: Increasing Server Efficiency via Workload Isolation in the Storage I/O Path

Long-term SLOs for reclaimed cloud computing resources

Vagabond: Dynamic Network Endpoint Reconfiguration in Virtualized Environments

Paxos Quorum Leases: Fast Reads Without Sacrificing Writes

C-Hint: An Effective and Reliable Cache Management for RDMA-Accelerated Key-Value Stores

Distributed Autonomous Virtual Resource Management in Datacenters Using Finite-Markov Decision Process

ReproLite: A Lightweight Tool to Quickly Reproduce Hard System Bugs

Wrangler: Predictable and Faster Jobs using Fewer Resources

vPipe: Piped I/O Offloading for Efficient Data Movement in Virtualized Clouds

Dynamic Performance Profiling of Cloud Caches

PriorityMeister: Tail Latency QoS for Shared Networked Storage

Tales of the Tail: Hardware, OS, and Application-level Sources of Tail Latency

Content Provider	ACM Digital Library
Author	Ports, Dan R. K. Gribble, Steven D. Sharma, Naveen Kr. Li, Jialin
Abstract	Interactive services often have large-scale parallel implementations. To deliver fast responses, the median and tail latencies of a service's components must be low. In this paper, we explore the hardware, OS, and application-level sources of poor tail latency in high throughput servers executing on multi-core machines. We model these network services as a queuing system in order to establish the best-achievable latency distribution. Using fine-grained measurements of three different servers (a null RPC service, Memcached, and Nginx) on Linux, we then explore why these servers exhibit significantly worse tail latencies than queuing models alone predict. The underlying causes include interference from background processes, request re-ordering caused by poor scheduling or constrained concurrency models, suboptimal interrupt routing, CPU power saving mechanisms, and NUMA effects. We systematically eliminate these factors and show that Memcached can achieve a median latency of 11 μs and a 99.9th percentile latency of 32 μs at 80% utilization on a four-core system. In comparison, a naïve deployment of Memcached at the same utilization on a single-core system has a median latency of 100 μs and a 99.9th percentile latency of 5 ms. Finally, we demonstrate that tradeoffs exist between throughput, energy, and tail latency.
Starting Page	1
Ending Page	14
Page Count	14
File Format	PDF
ISBN	9781450332521
DOI	10.1145/2670979.2670988
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2014-11-03
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Tail latency Predictable latency
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Tales of the Tail: Hardware, OS, and Application-level Sources of Tail Latency

Tales of the Tail: Hardware, OS, and Application-level Sources of Tail Latency

Chronos: predictable low latency for data center applications.

Chronos: predictable low latency for data center applications

C3: Cutting Tail Latency in Cloud Data Stores via Adaptive Replica Selection

Treadmill: attributing the source of tail latency through precise load testing and statistical inference

Where does the time go? characterizing tail latency in memcached

PCS: Predictive Component-Level Scheduling for Reducing Tail Latency in Cloud Online Services

Silo: Predictable Message Latency in the Cloud

Tales of the Tail: Hardware, OS, and Application-level Sources of Tail Latency

Similar Documents

Tales of the Tail: Hardware, OS, and Application-level Sources of Tail Latency

Tales of the Tail: Hardware, OS, and Application-level Sources of Tail Latency

Chronos: predictable low latency for data center applications.

Chronos: predictable low latency for data center applications

C3: Cutting Tail Latency in Cloud Data Stores via Adaptive Replica Selection

Treadmill: attributing the source of tail latency through precise load testing and statistical inference

Where does the time go? characterizing tail latency in memcached

PCS: Predictive Component-Level Scheduling for Reducing Tail Latency in Cloud Online Services

Silo: Predictable Message Latency in the Cloud

Tales of the Tail: Hardware, OS, and Application-level Sources of Tail Latency