NDLI: Groupwise analytics via adaptive MapReduce

Content Provider	IEEE Xplore Digital Library
Author	Liping Peng Kai Zeng Balmin, A. Ercegovac, V. Haas, P.J. Sismanis, Y.
Copyright Year	2015
Description	Author affiliation: Comput. Sci. Dept., Univ. of California, Berkeley, Berkeley, CA, USA (Kai Zeng) \|\| Sch. of Comput. Sci., Univ. of Massachusetts, Amherst, MA, USA (Liping Peng)
Abstract	Shared-nothing systems such as Hadoop vastly simplify parallel programming when processing disk-resident data whose size exceeds aggregate cluster memory. Such systems incur a significant performance penalty, however, on the important class of “groupwise set-valued analytics” (GSVA) queries in which the data is dynamically partitioned into groups and then a set-valued synopsis is computed for some or all of the groups. Key examples of synopses include top-k sets, bottom-k sets, and uniform random samples. Applications of GSVA queries include micro-marketing, root-cause analysis for problem diagnosis, and fraud detection. A naive approach to executing GSVA queries first reshuffles all of the data so that all records in a group are at the same node and then computes the synopsis for the group. This approach can be extremely inefficient when, as is typical, only a very small fraction of the records in each group actually contribute to the final groupwise synopsis, so that most of the shuffling effort is wasted. We show how to significantly speed up GSVA queries by slightly modifying the shared-nothing environment to allow tasks to occasionally access a small, common data structure; we focus on the Hadoop setting and use the “Adaptive MapReduce” infrastructure of Vernica et al. to implement the data structure. Our approach retains most of the advantages of a system such as Hadoop while significantly improving GSVA query performance, and also allows for incremental updating of query results. Experiments show speedups of up to 5x. Importantly, our new technique can potentially be applied to other shared-nothing systems with disk-resident data.
Starting Page	1059
Ending Page	1070
File Size	440360
Page Count	12
File Format	PDF
ISBN	9781479979646
DOI	10.1109/ICDE.2015.7113356
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2015-04-13
Publisher Place	South Korea
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Standards Electronic mail Aggregates Data structures Approximation methods Optimization Generators
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Multicriteria optimization of multiparameter adaptive information transfer system

Uniform approximation of continuous functions of bounded variation by aggregates of summatory type

A cost aware adaptive multiple table join evaluation in MapReduce

Breaking the MapReduce Stage Barrier

Rewriting complex SPARQL analytical queries for efficient cloud-based processing

Accessing linked open data

Adaptive data driven controller for nonlinear systems

Two-dimensional joint process adaptive filtering via principal component support region

Dependent Randomized Rounding via Exchange Properties of Combinatorial Structures

Groupwise analytics via adaptive MapReduce

Similar Documents

Multicriteria optimization of multiparameter adaptive information transfer system

Uniform approximation of continuous functions of bounded variation by aggregates of summatory type

A cost aware adaptive multiple table join evaluation in MapReduce

Breaking the MapReduce Stage Barrier

Rewriting complex SPARQL analytical queries for efficient cloud-based processing

Accessing linked open data

Adaptive data driven controller for nonlinear systems

Two-dimensional joint process adaptive filtering via principal component support region

Dependent Randomized Rounding via Exchange Properties of Combinatorial Structures

Groupwise analytics via adaptive MapReduce