NDLI: User Managed Virtual Clusters in Comet

Please wait, while we are loading the content...

Time Propagation of Partial Differential Equations Using the Short Iterative Lanczos Method and Finite-Element Discrete Variable Representation: An Experiment Using the Intel Phi Coprocessors: Extended Abstract

Containers in Research: Initial Experiences with Lightweight Infrastructure

Optimization of 3D Fusion Devices

Orion: Discovery Environment for HPC Research and Bridging XSEDE Resources

Optimization and parallel load balancing of the MPAS Atmosphere Weather and Climate Code

An Overview of the Implementation of CILogon at NICS

CloudBridge: a Simple Cross-Cloud Python Library

XSEDE Training: Achieving Access and Inclusion

Optimization of non-linear image registration in AFNI

Globus: Recent Enhancements and Future Plans

Introducing a New Client/Server Framework for Big Data Analytics with the R Language

Rescuing Lost History: Using Big Data to Recover Black Women's Lived Experiences

A Parallel Evolutionary Algorithm for Subset Selection in Causal Inference Models

BIC-LSU: Big Data Research Integration with Cyberinfrastructure for LSU

TopoLens: Building a CyberGIS Community Data Service for Enhancing the Usability of High-resolution National Topographic Datasets

The Advanced Cyberinfrastructure Research and Education Facilitators Virtual Residency: Toward a National Cyberinfrastructure Workforce

Three Dimensional Simulations of Fluid Flow and Heat Transfer with Spectral Element Method

Developing Applications with Networking Capabilities via End-to-End SDN (DANCES)

Anatomy of the SEAGrid Science Gateway

Performance and Scalability Analysis for Parallel Reservoir Simulations on Three Supercomputer Architectures

Library Function Tracking with XALT

The XSEDE BLAST Gateway: Leveraging Campus Development for the XSEDE Community

Implementation of Simple XSEDE-Like Clusters: Science Enabled and Lessons Learned

A web interface for XALT log data analysis

Extended Batch Sessions and Three-Phase Debugging: Using DMTCP to Enhance the Batch Environment

A Scalable High-performance Topographic Flow Direction Algorithm for Hydrological Information Analysis

A Quantitative Analysis of Node Sharing on HPC Clusters Using XDMoD Application Kernels

Open OnDemand: Transforming Computational Science Through Omnidisciplinary Software Cyberinfrastructure

Using High Performance Computing to Model Cellular Embryogenesis

From OpenACC to OpenMP 4: Toward Automatic Translation

Delayed Update Algorithms for Quantum Monte Carlo Simulation on GPU: Extended Abstract

Understanding the Evolving Cyberinfrastructure Needs of the Neuroscience Community

Efficient Primitives for Standard Tensor Linear Algebra

User behavior and usage patterns for a highly accessed science gateway

Tools for studying populations and timeseries of neuroanatomy enabled through GPU acceleration in the Computational Anatomy Gateway

Case Study: Microservice Evolution and Software Lifecycle of the XSEDE User Portal API

Minimization of Xeon Phi Core Use with Negligible Execution Time Impact

SeedMe: A scientific data sharing and collaboration platform

Computational Considerations in Transcriptome Assemblies and Their Evaluation, using High Quality Human RNA-Seq data

Practical Monitoring of Resource Utilization for HPC Applications

Estimating the Accuracy of User Surveys for Assessing the Impact of HPC Systems

The 10 Attributes that Drive Adoption and Diffusion of Computational Tools in E-Science

Cross-Accelerator Performance Profiling

Improving the Scalability of a Charge Detection Mass Spectrometry Workflow

Scaling GIS analysis tasks from the desktop to the cloud utilizing contemporary distributed computing and data management approaches: A case study of project-based learning and cyberinfrastructure concepts

User Managed Virtual Clusters in Comet

Content Provider	ACM Digital Library
Author	Cooper, Trevor Wang, Fugang von Laszewski, Gregor Wagner, Rick Papadopoulos, Philip Mishin, Dmitry Tatineti, Mahidhar Fox, Geoffrey C.
Abstract	Hardware virtualization has been gaining a significant share of computing time in the last years. Using virtual machines (VMs) for parallel computing is an attractive option for many users. A VM gives users a freedom of choosing an operating system, software stack and security policies, leaving the physical hardware, OS management, and billing to physical cluster administrators. The well-known solutions for cloud computing, both commercial (Amazon Cloud, Google Cloud, Yahoo Cloud, etc.) and open-source (OpenStack, Eucalyptus) provide platforms for running a single VM or a group of VMs. With all the benefits, there are also some drawbacks, which include reduced performance when running code inside of a VM, increased complexity of cluster management, as well as the need to learn new tools and protocols to manage the clusters. At SDSC, we have created a novel framework and infrastructure by providing virtual HPC clusters to projects using the NSF sponsored Comet supercomputer. Managing virtual clusters on Comet is similar to managing a bare-metal cluster in terms of processes and tools that are employed. This is beneficial because such processes and tools are familiar to cluster administrators. Unlike platforms like AWS, Comet's virtualization capability supports installing VMs from ISOs (i.e., a CD-ROM or DVD image) or via an isolated management VLAN (PXE). At the same time, we're helping projects take advantage of VMs by providing an enhanced client tool for interaction with our management system called Cloudmesh client. Cloudmesh client can also be used to manage virtual machines on OpenStack, AWS, and Azure. The article describes our design and approach to running virtual clusters, the tools we developed, and initial user experience.
Starting Page	1
Ending Page	8
Page Count	8
File Format	PDF
ISBN	9781450347556
DOI	10.1145/2949550.2949555
Language	English
Publisher	Association for Computing Machinery (ACM)
Publisher Date	2016-07-17
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Supercomputer Virtual machine
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Running User-Provided Virtual Machines in Batch-Oriented Computing Clusters

Efficient virtual machine caching in dynamic virtual clusters

Virtual Organization Clusters

Efficient Deterministic Replay of Multithreaded Executions in a Managed Language Virtual Machine

Managed Learning in 3D Multi User Virtual Environments

Usher: An Extensible Framework for Managing Clusters of Virtual Machines (2007)

VMKit: a substrate for managed runtime environments

Minimal-overhead virtualization of a large scale supercomputer

Virtcloud: virtual network for user-controlled virtual clusters.

User Managed Virtual Clusters in Comet

Similar Documents

Running User-Provided Virtual Machines in Batch-Oriented Computing Clusters

Efficient virtual machine caching in dynamic virtual clusters

Virtual Organization Clusters

Efficient Deterministic Replay of Multithreaded Executions in a Managed Language Virtual Machine

Managed Learning in 3D Multi User Virtual Environments

Usher: An Extensible Framework for Managing Clusters of Virtual Machines (2007)

VMKit: a substrate for managed runtime environments

Minimal-overhead virtualization of a large scale supercomputer

Virtcloud: virtual network for user-controlled virtual clusters.

User Managed Virtual Clusters in Comet