Scalable Data Partitioning Strategies for Efficient Query Optimization in Cloud Data Warehouses

Authors:
Venkata Sai Abhishek Anala, Sahithi Chintapalli

Addresses:
Department of Computer Science, University of Central Missouri, Atlanta, Georgia, United States of America. Department of Computer Science, VISA, Atlanta, Georgia, United States of America. Department of Computer Science, University of Central Missouri, Atlanta, Georgia, United States of America. Department of Computer Science, Home Depot Management LLC, Atlanta, Georgia, United States of America.

Abstract:

Huge structured and unstructured data will be stored in cloud-based data warehousing. However, the data warehouses get huge at times; in that case, it will frequently cause query performance to be the bottleneck of execution. The data partitioning with the target to disperse and organize data for efficient resource management and query execution time has now surfaced as a most important technique. Here are the scalable data partitioning strategies surveyed for efficient query optimization in cloud data warehouses. Some include techniques such as horizontal and vertical partitioning hybrid and indexes. That grouping would improve the efficiency and the scalability of techniques due to this aspect of techniques. This paper discusses state-of-the-art data partitioning, propounds a new hybrid partitioning technique that dynamically adapts to workloads, and evaluates improvements across query types and warehouse scales. The authors run a series of experiments on synthetic datasets as well as on real-world datasets. This final section of the paper outlines the present limitations of the partitioning techniques and hypothesizes some areas of research that would eventually enhance query execution in the cloud-based setup.

Keywords: Cloud Data Warehouses; Query Optimization; Data Partitioning; Scalability and Hybrid Partitioning; Storage and Management; Cloud Computing; Data Partitioning; Cloud Environment.

Received on: 02/05/2024, Revised on: 29/06/2024, Accepted on: 27/08/2024, Published on: 03/12/2024

DOI: 10.69888/FTSCL.2024.000279

FMDB Transactions on Sustainable Computer Letters, 2024 Vol. 2 No. 4, Pages: 195-206

  • Views : 91
  • Downloads : 9
Download PDF