NDLI: HConfig: Resource adaptive fast bulk loading in HBase

Content Provider	IEEE Xplore Digital Library
Author	Xianqiang Bao Liu, Ling Nong Xiao Fang Liu Qi Zhang Tao Zhu
Copyright Year	2014
Description	Author affiliation: State Key Laboratory of High Performance Computing, National University of Defense Technology, Changsha, Hunan 410073, China (Xianqiang Bao; Nong Xiao; Fang Liu) \|\| College of Computing, Georgia Institute of Technology, Atlanta, 30332-0765, USA (Liu, Ling; Qi Zhang; Tao Zhu)
Abstract	NoSQL (Not only SQL) data stores become a vital component in many big data computing platforms due to its inherent horizontal scalability. HBase is an open-source distributed NoSQL store that is widely used by many Internet enterprises to handle their big data computing applications (e.g. Facebook handles millions of messages each day with HBase). Optimizations that can enhance the performance of HBase are of paramount interests for big data applications that use HBase or Big Table like key-value stores. In this paper we study the problems inherent in misconfiguration of HBase clusters, including scenarios where the HBase default configurations can lead to poor performance. We develop HConfig, a semi-automated configuration manager for optimizing HBase system performance from multiple dimensions. Due to the space constraint, this paper will focus on how to improve the performance of HBase data loader using HConfig. Through this case study we will highlight the importance of resource adaptive and workload aware auto-configuration management and the design principles of HConfig. Our experiments show that the HConfig enhanced bulk loading can significantly improve the performance of HBase bulk loading jobs compared to the HBase default configuration, and achieve 2∼3.7x speedup in throughput under different client threads while maintaining linear horizontal scalability.
Starting Page	215
Ending Page	224
File Size	873180
Page Count	10
File Format	PDF
ISBN	9781631900433
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2014-10-22
Publisher Place	USA
Access Restriction	Subscribed
Rights Holder	ICST
Subject Keyword	Loading Random access memory Big Data Throughput Bulk Loading Compaction Servers Resource management Optimization HBase
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

HConfig: Resource Adaptive Fast Bulk Loading in HBase

Multi-resource schedulable unit for adaptive application-driven unified resource management in data centers

A Compatible LZMA ORC-Based Optimization for High Performance Big Data Load

Fast memory and storage architectures for the big data era

Effectiveness Assessment of Solid-State Drive Used in Big Data Services

TMemCanal: A VM-oblivious Dynamic Memory Optimization Scheme for Virtual Machines in Cloud Computing

Towards High Performance Spatio-temporal Data Management Systems

Efficient data management on 3D stacked memory for big data applications

Task data Optimization allocation with Data Migration

HConfig: Resource adaptive fast bulk loading in HBase

Similar Documents

HConfig: Resource Adaptive Fast Bulk Loading in HBase

Multi-resource schedulable unit for adaptive application-driven unified resource management in data centers

A Compatible LZMA ORC-Based Optimization for High Performance Big Data Load

Fast memory and storage architectures for the big data era

Effectiveness Assessment of Solid-State Drive Used in Big Data Services

TMemCanal: A VM-oblivious Dynamic Memory Optimization Scheme for Virtual Machines in Cloud Computing

Towards High Performance Spatio-temporal Data Management Systems

Efficient data management on 3D stacked memory for big data applications

Task data Optimization allocation with Data Migration

HConfig: Resource adaptive fast bulk loading in HBase