NDLI: Impact of data distribution, level of parallelism, and communication frequency on parallel data cube construction

Content Provider	IEEE Xplore Digital Library
Author	Ge Yang Ruoming Jin Agrawal, G.
Copyright Year	2003
Description	Author affiliation: Dept. of Comput. & Inf. Sci., Ohio State Univ., Columbus, OH, USA (Ge Yang; Ruoming Jin; Agrawal, G.)
Abstract	Data cube construction is a commonly used operation in data warehouses. Because of the volume of data that is stored and analyzed in a data warehouse and the amount of computation involved in data cube construction, it is natural to consider parallel machines for this operation. We have developed a set of parallel algorithms for data cube construction using a new data structure called aggregation tree. Our experience has shown that a number of performance trade-offs arise in developing a parallel data cube implementation. We focus on three important issues, which are: (1) data distribution, i.e., how the original array is distributed among the processors; (2) level of parallelism, i.e., what parts of the computation are parallelized and sequentialized; and (3) frequency of communication, i.e., does the implementation require frequent interprocessor communication (and less memory) or less frequent communication (and more memory). We present a detailed experimental study evaluating the above trade-offs. We consider parallel data cube construction with different cube sizes and sparsity levels. Our experimental results show the following: (1) In all cases, reducing the frequency of communication and using higher memory gave better performance, though the difference was relatively small. (2) Choosing data distribution to minimize communication volume made a substantial difference in the performance in most of the cases. (3) Finally, using parallelism at all levels gave better performance, even though it increases the total communication volume.
Sponsorship	IEEE Comput. Soc Tech. Committee on Parallel Process. IEEE Comput. Soc. Tech. Committee on Comput. Archit. IEEE Comput. Soc. Tech. Committee on Distrib. Process. ACM SIGARCH
File Size	383501
File Format	PDF
ISBN	0769519261
ISSN	15302075
DOI	10.1109/IPDPS.2003.1213162
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2003-04-22
Publisher Place	France
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Frequency Concurrent computing Parallel processing Data warehouses Parallel algorithms Data analysis Distributed computing Companies Performance analysis Aggregates
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Communication and memory optimal parallel data cube construction

Using tiling to scale parallel data cube construction

Communication and memory optimal parallel data cube construction

Parallel data cube construction for high performance on-line analytical processing

Impact of data distribution, level of parallelism, and communication frequency on parallel data cube construction.

Parallel processing of multi-join expansion-aggregate data cube query in high performance database systems

ParaCube: A Scalable OLAP Model Based on Distributed Aggregate Computing with Sibling Cubes

Exploiting Data- and Thread-Level Parallelism for Image Correlation

Modeling parallel and distributed computing resources using structured metadata

Impact of data distribution, level of parallelism, and communication frequency on parallel data cube construction

Similar Documents

Communication and memory optimal parallel data cube construction

Using tiling to scale parallel data cube construction

Communication and memory optimal parallel data cube construction

Parallel data cube construction for high performance on-line analytical processing

Impact of data distribution, level of parallelism, and communication frequency on parallel data cube construction.

Parallel processing of multi-join expansion-aggregate data cube query in high performance database systems

ParaCube: A Scalable OLAP Model Based on Distributed Aggregate Computing with Sibling Cubes

Exploiting Data- and Thread-Level Parallelism for Image Correlation

Modeling parallel and distributed computing resources using structured metadata

Impact of data distribution, level of parallelism, and communication frequency on parallel data cube construction