NDLI: Where does the time go? characterizing tail latency in memcached

Content Provider	IEEE Xplore Digital Library
Author	Blake, G. Saidi, A.G.
Copyright Year	2015
Abstract	To function correctly Online, Data-Intensive (OLDI) services require low and consistent service times. Maintaining predictable service times entails requiring $99^{th}$ or higher percentile latency targets across hundreds to thousands of servers in the data-center. However, to maintain the $99^{th}$ percentile targets servers are routinely run well below full utilization. The main difficulty in optimizing a server to run closer to peak utilization and maintain predictable $99^{th}$ percentile response latencies is identifying and mitigating the causes of a request missing the target service time. In practice this analysis is challenging as requests and responses overlap their execution with respect to one another and traverse multiple layers of software, user/kernel protection boundaries, and the hardware/software divide. Traditional profiling methods that record the time being spent in each function usually yield few clues as to where a bottleneck may be present due to the many layers of software each consuming only a small fraction of time each. In this work we analyze the end-to-end sources of latency in a Memcached server from the wire through the kernel into the application and back again. To do so, we develop a tool that utilizes the Linux SystemTap infrastructure to measure latency throughout the many software layers that make up the complete request and response path for Memcached. While memory copies and the Linux networking stack are often suggested as major contributors to latency, we find that the main cause of missing response latency guarantees is the formation of standing queues and the application's inability to detect and remedy this situation.
Sponsorship	IEEE Comput. Soc.
Starting Page	21
Ending Page	31
File Size	365499
Page Count	11
File Format	PDF
ISBN	9781479919574
DOI	10.1109/ISPASS.2015.7095781
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2015-03-29
Publisher Place	USA
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Servers Kernel Linux Probes IP networks Hardware
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Low latency packet processing in software routers

A study of network stack latency for game servers

Off-path round trip time measurement via TCP/IP side channels

Lightweight built-in network monitor in Linux kernel for self-adaptive IoT devices

Minimizing Latency of Real-Time Container Cloud for Software Radio Access Networks

How to Reduce Network Interrupt Latency for Real-Time Processing

Anytime Available Single IP Address Cluster

Research on IP Address Replacement Technology Based on Iptables

Tethered Linux CPE for IP service delivery

Where does the time go? characterizing tail latency in memcached

Similar Documents

Low latency packet processing in software routers

A study of network stack latency for game servers

Off-path round trip time measurement via TCP/IP side channels

Lightweight built-in network monitor in Linux kernel for self-adaptive IoT devices

Minimizing Latency of Real-Time Container Cloud for Software Radio Access Networks

How to Reduce Network Interrupt Latency for Real-Time Processing

Anytime Available Single IP Address Cluster

Research on IP Address Replacement Technology Based on Iptables

Tethered Linux CPE for IP service delivery

Where does the time go? characterizing tail latency in memcached