NDLI: Tajo: A distributed data warehouse system on large clusters

Content Provider	IEEE Xplore Digital Library
Author	Hyunsik Choi Jihoon Son Haemi Yang Hyoseok Ryu Byungnam Lim Soohyung Kim Yon Dohn Chung
Copyright Year	2013
Description	Author affiliation: Dept. of Comput. Sci. & Eng., Korea Univ., Seoul, South Korea (Hyunsik Choi; Jihoon Son; Haemi Yang; Hyoseok Ryu; Byungnam Lim; Soohyung Kim; Yon Dohn Chung)
Abstract	The increasing volumes of relational data let us find an alternative to cope with them. Recently, several hybrid approaches (e.g., HadoopDB and Hive) between parallel databases and Hadoop have been introduced to the database community. Although these hybrid approaches have gained wide popularity, they cannot avoid the choice of suboptimal execution strategies. We believe that this problem is caused by the inherent limits of their architectures. In this demo, we present Tajo, a relational, distributed data warehouse system on shared-nothing clusters. It uses Hadoop Distributed File System (HDFS) as the storage layer and has its own query execution engine that we have developed instead of the MapReduce framework. A Tajo cluster consists of one master node and a number of workers across cluster nodes. The master is mainly responsible for query planning and the coordinator for workers. The master divides a query into small tasks and disseminates them to workers. Each worker has a local query engine that executes a directed acyclic graph of physical operators. A DAG of operators can take two or more input sources and be pipelined within the local query engine. In addition, Tajo can control distributed data flow more flexible than that of MapReduce and supports indexing techniques. By combining these features, Tajo can employ more optimized and efficient query processing, including the existing methods that have been studied in the traditional database research areas. To give a deep understanding of the Tajo architecture and behavior during query processing, the demonstration will allow users to submit TPC-H queries to 32 Tajo cluster nodes. The web-based user interface will show (1) how the submitted queries are planned, (2) how the query are distributed across nodes, (3) the cluster and node status, and (4) the detail of relations and their physical information. Also, we provide the performance evaluation of Tajo compared with Hive.
Sponsorship	IEEE Comput. Soc.
Starting Page	1320
Ending Page	1323
File Size	1328233
Page Count	4
File Format	PDF
ISBN	9781467349093
ISSN	10636382
e-ISBN	9781467349109
e-ISBN	9781467349086
DOI	10.1109/ICDE.2013.6544934
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2013-04-08
Publisher Place	Australia
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Engines Query processing Planning Fault tolerance Fault tolerant systems Catalogs
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Fault-tolerance in distributed query processing

A fully automated fault-tolerant system for distributed video processing and off-site replication

Fault tolerant distributed database system via data inference

Greft: Arbitrary Fault-Tolerant Distributed Graph Processing

Building a fault tolerant distributed processing environment

Mapping a Fault-Tolerant Distributed Algorithm to Systems on Chip

Distributed fault-tolerant control for a large-scale power generator network

Simulator for fault tolerance in large scale distributed systems

A fault tolerant MPI-IO implementation using the Expand parallel file system

Tajo: A distributed data warehouse system on large clusters

Similar Documents

Fault-tolerance in distributed query processing

A fully automated fault-tolerant system for distributed video processing and off-site replication

Fault tolerant distributed database system via data inference

Greft: Arbitrary Fault-Tolerant Distributed Graph Processing

Building a fault tolerant distributed processing environment

Mapping a Fault-Tolerant Distributed Algorithm to Systems on Chip

Distributed fault-tolerant control for a large-scale power generator network

Simulator for fault tolerance in large scale distributed systems

A fault tolerant MPI-IO implementation using the Expand parallel file system

Tajo: A distributed data warehouse system on large clusters