NDLI: Enterprise Resource Management in Mesos Clusters

Please wait, while we are loading the content...

The DragonBeam Framework: Hardware-Protected Security Modules for In-Place Intrusion Detection

Health-Binning: Maximizing the Performance and the Endurance of Consumer-Level NAND Flash

Enabling Space Elasticity in Storage Systems

Black box replication: Breaking the latency limits

Exploiting Parallelism of Distributed Nested Transactions

Proper Timed I/O: High-Accuracy Real-Time Control for Conventional Operating Systems

A study on the errors and uncertainties of file system trace capture methods

Enterprise Resource Management in Mesos Clusters

SeMiNAS: A Secure Middleware for Wide-Area Network-Attached Storage

Elastic Queue: A Universal SSD Lifetime Extension Plug-in for Cache Replacement Algorithms

SSD Failures in Datacenters: What? When? and Why?

Versioned Programming: A Simple Technique for Implementing Efficient, Lock-Free, and Composable Data Structures

Reducing Journaling Harm on Virtualized I/O Systems

Utilizing Optical Circuits in Hybrid Packet/Circuit Data-Center Networks

Helping Protect Software Distribution with PSWD

S-RAC: SSD Friendly Caching for Data Center Workloads

Supporting data-driven I/O on GPUs using GPUfs

File System Usage in Android Mobile Phones

Coded Network Switches for Improved Throughput

A Robotic Mobile Hot Spot Relay (MHSR) for Disaster Areas

Using Storage Class Memory Efficiently for an In-memory Database

Software-Defined Emulation Infrastructure for High Speed Storage

IO Core Manager for Virtual Environments

Cross-ISA Container Migration

Memory-Aware Management for Multi-Level Main Memory Complex using an Optimization of the Aging Paging Algorithm

Flexible Download Time Analysis of Coded Storage Systems

Fast and Scalable Security Support for Directory-Based Distributed Shared Memory

Optics in Data Centers: Adapting to Diverse Modern Workloads

Semi-Open Trace Based Simulation for Reliable Evaluation of Job Throughput and User Productivity

Enterprise Resource Management in Mesos Clusters

Content Provider	ACM Digital Library
Author	Glikson, Alex Ahmed, Khalid Breitgand, David Abu-Dbai, Abed Gershinsky, Gidon
Abstract	Enterprise data centers increasingly adopt a cloud-like architecture that enables the execution of multiple workloads on a shared pool of resources, reduces the data center footprint and drives down the costs. A number of cluster resource managers have appeared over the last few years, aimed at providing a uniform technology-neutral resource representation and management substrate. Examples include Apache YARN, Google Borg and Omega, Apache Mesos, and IBM Platform EGO. The Apache Mesos project [2] is emerging as a leading open source resource management technology for server clusters. Mesos offers simple yet powerful and flexible APIs, highly available and fault tolerant architecture, scalability to large clusters, isolation between tasks using Linux containers, multi-dimensional resource scheduling, ability to allocate shares of the cluster to roles representing users or user groups, and a clear separation of concerns between the applications (termed frameworks) and the "cluster kernel", which is Mesos. The resource scheduler of Mesos supports a generalization of max-min fairness, termed Dominant Resource Fairness (DRF) [1] scheduling discipline, which allows to harmonize execution of heterogeneous workloads (in terms of resource demand) by maximizing the share of any resource allocated to a specific framework. However, the default Mesos allocation mechanism lacks a number of policy and tenancy capabilities, important in enterprise deployments. We have investigated integration of Mesos with the IBM EGO (enterprise grid orchestrator) technology [3] which underpins various high performance computing, analytics and big data clusters in a variety of industry verticals including financial services, life sciences, manufacturing and electronics. We have designed and implemented an experimental integration prototype, and have tested it with SparkBench workloads. We demonstrate how Mesos can be enriched with new resource policy capabilities, required for managing enterprise data centers, such as • Capturing of the hierarchical structure of an enterprise (organisations, departments, groups, teams, users) by defining the corresponding resource consumer tree; • A fine grained resource plan allowing to define resource share ratio, ownership and lending/borrowing policies for each resource consumer; • A rich set of resource management policies making use of the hierarchical resource consumer model and providing fairness and isolation to the members of hierarchy including an important ability to dynamically change the allocations (time-based policy); • A Web-based GUI providing a centralized console through which the whole cluster is observed and managed. In particular, the cluster-wide resource management policies are applied through this GUI.
Starting Page	1
Ending Page	1
Page Count	1
File Format	PDF
ISBN	9781450343817
DOI	10.1145/2928275.2933272
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2016-06-06
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Resource pool Cluster Resource management
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Megos : Enterprise Resource Management in Mesos Clusters

Mesos -- Flexible resource sharing . . . (2011)

Mesos: A platform for fine-grained resource sharing in the data center (2010)

Mesos: A platform for fine-grained resource sharing in the data center (2010)

Mesos: A platform for fine-grained resource sharing in the data center (2010)

Autonomic energy management in clusters

Satisfying Service Level Objectices in a Self-Managing Resource Pool

A managed distributed processing pipeline with Storm and Mesos

Resource Pool Oriented Trust Management for Cloud Infrastructure

Enterprise Resource Management in Mesos Clusters

Similar Documents

Megos : Enterprise Resource Management in Mesos Clusters

Mesos -- Flexible resource sharing . . . (2011)

Mesos: A platform for fine-grained resource sharing in the data center (2010)

Mesos: A platform for fine-grained resource sharing in the data center (2010)

Mesos: A platform for fine-grained resource sharing in the data center (2010)

Autonomic energy management in clusters

Satisfying Service Level Objectices in a Self-Managing Resource Pool

A managed distributed processing pipeline with Storm and Mesos

Resource Pool Oriented Trust Management for Cloud Infrastructure

Enterprise Resource Management in Mesos Clusters