NDLI: Modeling and synthesizing task placement constraints in Google compute clusters

Please wait, while we are loading the content...

An efficient multi-tier tablet server storage architecture

Small-world datacenters

Modeling and synthesizing task placement constraints in Google compute clusters

DOT: a matrix model for analyzing, optimizing and deploying software for big data analytics in distributed systems

CloudScale: elastic resource scaling for multi-tenant cloud systems

ALIAS: scalable, decentralized label assignment for data centers

Incoop: MapReduce for incremental computations

CloudNaaS: a cloud networking platform for enterprise applications

YCSB++: benchmarking and performance debugging advanced features in scalable table stores

Silverline: toward data confidentiality in storage-intensive cloud applications

CoScan: cooperative scan sharing in the cloud

Query optimization for massively parallel data processing

PrIter: a distributed framework for prioritized iterative computations

Policy expressivity in the Anzere personal cloud

ActiveSLA: a profit-oriented admission control framework for database-as-a-service providers

Orleans: cloud computing for everyone

PipeCloud: using causality to overcome speed-of-light delays in cloud-based disaster recovery

No one (cluster) size fits all: automatic cluster sizing for data-intensive analytics

Pesto: online storage performance management in virtualized datacenters

Making time-stepped applications tick in the cloud

Trojan data layouts: right shoes for a running elephant

Cuanta: quantifying effects of shared on-chip resource interference for consolidated virtual machines

Small cache, big effect: provable load balancing for randomly partitioned cluster services

Opportunistic flooding to improve TCP transmit performance in virtualized clouds

Improving per-node efficiency in the datacenter with new OS abstractions

Declarative automated cloud resource orchestration

Automatic management of partitioned, replicated search services

Scaling the mobile millennium system in the cloud

To cloud or not to cloud?: musings on costs and viability

Switching the optical divide: fundamental challenges for hybrid electrical/optical datacenter networks

Modeling and synthesizing task placement constraints in Google compute clusters

Content Provider	ACM Digital Library
Author	Chudnovsky, Victor Sharma, Bikash Das, Chita R. Rifaat, Rasekh Hellerstein, Joseph L.
Abstract	Evaluating the performance of large compute clusters requires benchmarks with representative workloads. At Google, performance benchmarks are used to obtain performance metrics such as task scheduling delays and machine resource utilizations to assess changes in application codes, machine configurations, and scheduling algorithms. Existing approaches to workload characterization for high performance computing and grids focus on task resource requirements for CPU, memory, disk, I/O, network, etc. Such resource requirements address how much resource is consumed by a task. However, in addition to resource requirements, Google workloads commonly include task placement constraints that determine which machine resources are consumed by tasks. Task placement constraints arise because of task dependencies such as those related to hardware architecture and kernel version. This paper develops methodologies for incorporating task placement constraints and machine properties into performance benchmarks of large compute clusters. Our studies of Google compute clusters show that constraints increase average task scheduling delays by a factor of 2 to 6, which often results in tens of minutes of additional task wait time. To understand why, we extend the concept of resource utilization to include constraints by introducing a new metric, the Utilization Multiplier (UM). UM is the ratio of the resource utilization seen by tasks with a constraint to the average utilization of the resource. UM provides a simple model of the performance impact of constraints in that task scheduling delays increase with UM. Last, we describe how to synthesize representative task constraints and machine properties, and how to incorporate this synthesis into existing performance benchmarks. Using synthetic task constraints and machine properties generated by our methodology, we accurately reproduce performance metrics for benchmarks of Google compute clusters with a discrepancy of only 13% in task scheduling delay and 5% in resource utilization.
Starting Page	1
Ending Page	14
Page Count	14
File Format	PDF
ISBN	9781450309769
DOI	10.1145/2038916.2038919
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2011-10-26
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Performance evaluation Metrics Workload characterization Benchmarks Benchmarking Modeling
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Modeling and synthesizing task placement constraints in google compute clusters (2011)

Modeling and synthesizing task placement constraints in google compute clusters.

Modeling and synthesizing task placement constraints in google compute clusters (2011)

Synthesizing benchmarks for predictive modeling

Characterizing Task Usage Shapes in Google Compute Clusters

NERSC-6 Workload Analysis and Benchmark Selection Process

Characterizing task usage shapes in Google compute clusters (2011)

Language directed computer performance evaluation

Guidelines for benchmarking automated software traceability techniques

Modeling and synthesizing task placement constraints in Google compute clusters

Similar Documents

Modeling and synthesizing task placement constraints in google compute clusters (2011)

Modeling and synthesizing task placement constraints in google compute clusters.

Modeling and synthesizing task placement constraints in google compute clusters (2011)

Synthesizing benchmarks for predictive modeling

Characterizing Task Usage Shapes in Google Compute Clusters

NERSC-6 Workload Analysis and Benchmark Selection Process

Characterizing task usage shapes in Google compute clusters (2011)

Language directed computer performance evaluation

Guidelines for benchmarking automated software traceability techniques

Modeling and synthesizing task placement constraints in Google compute clusters