NDLI: Strategic and suave processing for performing similarity joins using MapReduce

Content Provider	Springer Nature Link
Author	Lakshminarayanan, Mahalakshmi Acosta, William F. Green, Robert C. Devabhaktuni, Vijay
Copyright Year	2014
Abstract	An efficient MapReduce Algorithm for performing Similarity Joins between multisets is proposed. Filtering techniques for similarity joins minimize the number of pairs of entities joined and hence improve the efficiency of the algorithm. Multisets represent real-world data better by considering the frequency of its elements. Prior serial algorithms incorporate filtering techniques only for sets, but not multisets, while prior MapReduce algorithms do not incorporate any filtering technique or inefficiently and unscalably incorporate prefix filtering. This work extends the filtering techniques, namely the prefix, size and positional to multisets, and also achieves the challenging task of efficiently incorporating them in the shared-nothing MapReduce model, thereby minimizing the pairs generated and joined, resulting in I/O, network and computational efficiency. A technique to enhance the scalability of the algorithm is also presented as a contingency need. Algorithms are developed using Hadoop and tested using real-world Twitter data. Experimental results demonstrate unprecedented performance gain.
Starting Page	930
Ending Page	954
Page Count	25
File Format	PDF
ISSN	09208542
Journal	The Journal of Supercomputing
Volume Number	69
Issue Number	2
e-ISSN	15730484
Language	English
Publisher	Springer US
Publisher Date	2014-05-05
Publisher Place	Boston
Access Restriction	One Nation One Subscription (ONOS)
Subject Keyword	Similarity Joins Multisets MapReduce Programming Languages, Compilers, Interpreters Processor Architectures Computer Science
Content Type	Text
Resource Type	Article
Subject	Theoretical Computer Science Information Systems Hardware and Architecture Software

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

An efficient MapReduce algorithm for similarity join in metric spaces

Join processing with threshold-based filtering in MapReduce

Detecting subgraph isomorphism with MapReduce

Optimized big data K-means clustering using MapReduce

Big data pre-processing methods with vehicle driving data using MapReduce techniques

Peacock: a customizable MapReduce for multicore platform

Software Architectures and Coordination Models

Advances in multicore systems architectures

Green networks

Strategic and suave processing for performing similarity joins using MapReduce

Similar Documents

An efficient MapReduce algorithm for similarity join in metric spaces

Join processing with threshold-based filtering in MapReduce

Detecting subgraph isomorphism with MapReduce

Optimized big data K-means clustering using MapReduce

Big data pre-processing methods with vehicle driving data using MapReduce techniques

Peacock: a customizable MapReduce for multicore platform

Software Architectures and Coordination Models

Advances in multicore systems architectures

Green networks

Strategic and suave processing for performing similarity joins using MapReduce