Loading...
Please wait, while we are loading the content...
Similar Documents
A Comprehensive Spark-Based Layer for Converting Relational Databases to NoSQL
| Content Provider | MDPI |
|---|---|
| Author | Abdel-Fattah, Manal A. Mohamed, Wael Abdelgaber, Sayed |
| Copyright Year | 2022 |
| Description | Currently, the continuous massive growth in the size, variety, and velocity of data is defined as big data. Relational databases have a limited ability to work with big data. Consequently, not only structured query language (NoSQL) databases were utilized to handle big data because NoSQL represents data in diverse models and uses a variety of query languages, unlike traditional relational databases. Therefore, using NoSQL has become essential, and many studies have attempted to propose different layers to convert relational databases to NoSQL; however, most of them targeted only one or two models of NoSQL, and evaluated their layers on a single node, not in a distributed environment. This study proposes a Spark-based layer for mapping relational databases to NoSQL models, focusing on the document, column, and key–value databases of NoSQL models. The proposed Spark-based layer comprises of two parts. The first part is concerned with converting relational databases to document, column, and key–value databases, and encompasses two phases: a metadata analyzer of relational databases and Spark-based transformation and migration. The second part focuses on executing a structured query language (SQL) on the NoSQL. The suggested layer was applied and compared with Unity, as it has similar components and features and supports sub-queries and join operations in a single-node environment. The experimental results show that the proposed layer outperformed Unity in terms of the query execution time by a factor of three. In addition, the proposed layer was applied to multi-node clusters using different scenarios, and the results show that the integration between the Spark cluster and NoSQL databases on multi-node clusters provided better performance in reading and writing while increasing the dataset size than using a single node. |
| Starting Page | 71 |
| e-ISSN | 25042289 |
| DOI | 10.3390/bdcc6030071 |
| Journal | Big Data and Cognitive Computing |
| Issue Number | 3 |
| Volume Number | 6 |
| Language | English |
| Publisher | MDPI |
| Publisher Date | 2022-06-27 |
| Access Restriction | Open |
| Subject Keyword | Big Data and Cognitive Computing Medical Informatics Big Data Nosql Spark Relational Database Migration Transformation |
| Content Type | Text |
| Resource Type | Article |
