NDLI: Untangling cluster management with Helix

Please wait, while we are loading the content...

Logic and lattices for distributed programming

vBalance: using interrupt load balance to improve I/O performance for SMP virtual machines

Improving large graph processing on partitioned graphs in the cloud

Sailfish: a framework for large scale data processing

OS-Sommelier: memory-only operating system fingerprinting in the cloud

How consistent is your cloud application?

Heterogeneity and dynamicity of clouds at scale: Google trace analysis

Using vector interfaces to deliver millions of IOPS from a networked key-value storage server

Chronos: predictable low latency for data center applications

Bridging the tenant-provider gap in cloud services

Using batteries to reduce the power costs of internet-scale distributed networks

Zeta: scheduling interactive services with partial execution

Themis: an I/O-efficient MapReduce

Cake: enabling high-level SLOs on shared storage systems

Generalized resource allocation for the cloud

Balancing reducer skew in MapReduce workloads using progressive sampling

Probabilistic deduplication for cluster-based storage systems

All aboard the Databus!: Linkedin's scalable consistent change data capture platform

Untangling cluster management with Helix

More for your money: exploiting performance heterogeneity in public clouds

Romano: autonomous storage management using performance prediction in multi-tenant datacenters

The potential dangers of causal consistency and an explicit solution

A case for dual stack virtualization: consolidating HPC and commodity applications in the cloud

True elasticity in multi-tenant data-intensive compute clusters

alsched: algebraic scheduling of mixed workloads in heterogeneous clouds

Designing good algorithms for MapReduce and beyond

Distributed programming and consistency: principles and practice

Open source cloud technologies

Untangling cluster management with Helix

Content Provider	ACM Digital Library
Author	Silberstein, Adam Gopalakrishna, Kishore Zhang, Zhen Schulman, Bob Lu, Shi Subramonian, Ramesh Surlaker, Kapil
Abstract	Distributed data systems systems are used in a variety of settings like online serving, offline analytics, data transport, and search, among other use cases. They let organizations scale out their workloads using cost-effective commodity hardware, while retaining key properties like fault tolerance and scalability. At LinkedIn we have built a number of such systems. A key pattern we observe is that even though they may serve different purposes, they tend to have a lot of common functionality, and tend to use common building blocks in their architectures. One such building block that is just beginning to receive attention is cluster management, which addresses the complexity of handling a dynamic, large-scale system with many servers. Such systems must handle software and hardware failures, setup tasks such as bootstrapping data, and operational issues such as data placement, load balancing, planned upgrades, and cluster expansion. All of this shared complexity, which we see in all of our systems, motivates us to build a cluster management framework, Helix, to solve these problems once in a general way. Helix provides an abstraction for a system developer to separate coordination and management tasks from component functional tasks of a distributed system. The developer defines the system behavior via a state model that enumerates the possible states of each component, the transitions between those states, and constraints that govern the system's valid settings. Helix does the heavy lifting of ensuring the system satisfies that state model in the distributed setting, while also meeting the system's goals on load balancing and throttling state changes. We detail several Helix-managed production distributed systems at LinkedIn and how Helix has helped them avoid building custom management components. We describe the Helix design and implementation and present an experimental study that demonstrates its performance and functionality.
Starting Page	1
Ending Page	13
Page Count	13
File Format	PDF
ISBN	9781450317610
DOI	10.1145/2391229.2391248
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2012-10-14
Publisher Place	New York
Access Restriction	Subscribed
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Pandora's Box Exposed: Untangling the Web of the Double Helix in Light of Insurance and Managed Care

Pandora's Box Exposed: Untangling the Web of the Double Helix in Light of Insurance and Managed Care

Untangling African indigenous management

Untangling mixed information to calibrate resource utilization in virtual machines

Untangling cloth

Helix TWT with dielectric corrective of the slowing

Cluster Initiative in Fine Chemicals as a Case of Practical Implementation of Triple Helix Collaboration for Regional Economic Growth and Innovation-Driven Development

Features of the triple helix model in cross-border clusters.

Untangling participation in art and design: a dialogical exhibition by traders

Untangling cluster management with Helix

Similar Documents

Pandora's Box Exposed: Untangling the Web of the Double Helix in Light of Insurance and Managed Care

Pandora's Box Exposed: Untangling the Web of the Double Helix in Light of Insurance and Managed Care

Untangling African indigenous management

Untangling mixed information to calibrate resource utilization in virtual machines

Untangling cloth

Helix TWT with dielectric corrective of the slowing

Cluster Initiative in Fine Chemicals as a Case of Practical Implementation of Triple Helix Collaboration for Regional Economic Growth and Innovation-Driven Development

Features of the triple helix model in cross-border clusters.

Untangling participation in art and design: a dialogical exhibition by traders

Untangling cluster management with Helix