NDLI: Hash semi cascade join for joining multi-way map reduce

Content Provider	IEEE Xplore Digital Library
Author	Mohamed, M.H. Khafagy, M.H.
Copyright Year	2015
Description	Author affiliation: Comput. Sci. Dept., Fayoum Univ., Cairo, Egypt (Khafagy, M.H.) \|\| Inf. Syst. Dept., Technol. & Maritime Transp., Cairo, Egypt (Mohamed, M.H.)
Abstract	Map-reduce is a programming model popularized by Google since 2004. It's used with large-scale datasets and processing data on a shared-nothing cluster. Map-Reduce accomplish high performance by partitioning the processes into small units of work that can run in parallel across thousands of nodes in the cluster. Rapidly, increasing in data size has risen importance to uncover hidden pattern to acquire new knowledge and get valuable information. But, map-reduce doesn't directly support join operation. This paper discusses some types of two-way algorithms, list some advantage and disadvantage of every algorithms. We propose a new multi - way join algorithm hash semi cascade join used to join more than two data sets. Using hash tables in the first phase, deleting unused records for joint operation as early as possible to reduce network bottleneck and increase performance. We compare this new algorithm with some types of multi-way join like map side join, reduce side one shot join and reduce side cascade join. Our experimental results show that the map side join has more time for sorting data and do join result with small data sets with high performance but, time increase while data are increased. Reduce side one shot join has join result near map side join. Reduce side cascade join get more time to get the final result. Hash semi cascade join gain high performance using hash tables. According to, reduce shuffling records as in reduce side one shot and reduce side cascade join it can do join for any data set size. As well, using a hash table doesn't effect in memory size.
Starting Page	355
Ending Page	361
File Size	328097
Page Count	7
File Format	PDF
e-ISBN	9781467376068
DOI	10.1109/IntelliSys.2015.7361167
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2015-11-10
Publisher Place	UK
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Algorithm design and analysis Two-way Google Hadoop Hash tables Sorting Multi-way Distributed databases Clustering algorithms Big data Intelligent systems .semi-join Map-reduce
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

An efficient improved join algorithm using map reduce in hadoop

In-Map/In-Reduce: Concurrent Job Execution in MapReduce

Distributed FP-ARMH algorithm in Hadoop map reduce framework

Strategies for Big Data Clustering

A novel algorithm for distributed data mining in HDFS

Big data: Mathematical topology video data analytics using superimposed learning

Big data analysis using Apache Hadoop

A 2-Tier Clustering Algorithm with Map-Reduce

Market basket analysis algorithm on map/reduce in aws ec2.

Hash semi cascade join for joining multi-way map reduce

Similar Documents

An efficient improved join algorithm using map reduce in hadoop

In-Map/In-Reduce: Concurrent Job Execution in MapReduce

Distributed FP-ARMH algorithm in Hadoop map reduce framework

Strategies for Big Data Clustering

A novel algorithm for distributed data mining in HDFS

Big data: Mathematical topology video data analytics using superimposed learning

Big data analysis using Apache Hadoop

A 2-Tier Clustering Algorithm with Map-Reduce

Market basket analysis algorithm on map/reduce in aws ec2.

Hash semi cascade join for joining multi-way map reduce