GamaBox-One: A Proposed Architecture for Cloud-based Big Data Management Platform for Multipurpose Computation using Hybrid Architecture
Article Sidebar
Main Article Content
Abstract
The architecture of the data center is the main key for producing a highly functional demand big data management platform for multipurpose uses. Nowadays various technologies have come to provide construction of that purpose, providing several use cases for big data analytics and processing. In this paper, we want to explore possibilities of architecture that had to be built in answer to the multipurpose data center, such as analytical research, scientific simulation, machine learning, deep data learning, and data orchestration. We discover how Hadoop and its element supporter can be used alongside cloud orchestrators such as Terraform or Occopus and container orchestrators such as Kubernetes or Docker Swarm. We also provide possible supporting components that can handle the different jobs in High-Performance Computing and how the system can be secured. Our proposed approach in this research has developed the architecture for cloud-based big data management for multipurpose computation.
Article Details
This work is licensed under a Creative Commons Attribution 4.0 International License.
References
A. Mosa, T. Kiss, G. Pierantoni, J. DesLauriers, D. Kagialis, and G. Terstyanszky, Towards a Cloud Native Big Data Platform using MiCADO, in 2020 19th International Symposium on Parallel and Distributed Computing (ISPDC), Jul. 2020, pp. 118–125. doi: 10.1109/ISPDC51135.2020.00025.
C. G. N. TAMA, Sistem Operasi untuk Pemrosesan Big Data dengan berbasis Centos 7, Universitas Gadjah Mada, 2017.
M. M. Abror, Implementasi Container dan Kubernetes Orchestration dalam Peluncuran, Pengawasan dan Pengaturan Ekosistem Big Data, Universitas Gadjah Mada, 2019.
T. Kiss, Scalable multi-cloud platform to support industry and scientific applications, in 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), May 2018, pp. 0150–0154. doi: 10.23919/MIPRO.2018.8400029.
What is cloud orchestration (cloud orchestrator)? - Definition from WhatIs.co’, SearchITOperations. https://searchitoperations.techtarget.com/definition/cloud-orchestrator (accessed Dec. 17, 2021).
What is container orchestration? https://www.redhat.com/en/topics/containers/what-is-container-orchestration (accessed Dec. 17, 2021).
B. Soewito and C. E. Andhika, Next Generation Firewall for Improving Security in Company and IoT Network, in 2019 International Seminar on Intelligent Technology and Its Applications (ISITIA), Aug. 2019, pp. 205–209. doi: 10.1109/ISITIA.2019.8937145
Apache Hadoop, https://hadoop.apache.org/ (accessed November 10, 2022).
Yarn, https://yarnpkg.com/ (accessed November 10, 2022).
Map Reduce Tutorial, https://hadoop.apache.org/docs/r1.2.1/mapred_tutorial.html (accessed November 10, 2022).
Apache Ambari, https://ambari.apache.org/ (accessed November 10, 2022).
Avro, https://binx.io/2018/12/09/apache-avro/ (accessed November 10, 2022).
Apache Cassandra, https://cassandra.apache.org/_/index.html (accessed November 10. 2022).
Apache Chukwa, https://chukwa.apache.org/ (accessed November 2022).
Apache HBase, https://hbase.apache.org/ (accessed November 2022).
Apache Hive, https://hive.apache.org/ (accessed November 2022).
Apache Mahout, https://mahout.apache.org// (accessed November 2022).
Apache Hadoop Ozone, https://hadoop.apache.org/ozone/ (accessed November 2022).
Apache Pig, https://pig.apache.org/ (accessed November 11, 2022).
Apache Spark, https://spark.apache.org/ (accessed November 11, 2022).
Apache Submarine, https://submarine.apache.org/ (accessed November 11, 2022).
Apache Tez, https://tez.apache.org/ (accessed November 11, 2022).
Apache Zookeeper, https://zookeeper.apache.org/ (accessed November 11, 2022).
Belov, V., Nikulchev, E. Analysis of Big Data Storage Tools for Data Lakes based on Apache Hadoop Platform. International Journal of Advanced Computer Science and Applications(IJACSA), Volume 12 Issue 8, 2021, DOI: 10.14569/IJACSA.2021.0120864
Voit, A., Stankus, A., Megomedov, S, Ivanoca, I., Big Data Processing for Full-Text Search and Visualization with Elasticsearch. International Journal of Advanced Computer Science and Applications(IJACSA), Volume 8 Issue 12, 2017. DOI: 10.14569/IJACSA.2017.081211
Acharjya, D.P., Kauser, A.P. A Survey on Big Data Analytics: Challenges, Open Research Issues and Tools. International Journal of Advanced Computer Science and Applications(IJACSA), Volume 7 Issue 2, 2016. DOI: 10.14569/IJACSA.2016.070267
Shah, H., Din, A.u., Abizar, Khan, A., Din, S.u. Enhancing the Quality of Service of Cloud Computing in Big Data Using Virtual Private Network and Firewall in Dense Mode. International Journal of Advanced Computer Science and Applications(IJACSA), Volume 11 Issue 3, 2020. DOI : 10.14569/IJACSA.2020.0110351
Shabbir, A., Abu Bakar, K., Radzi, R.Z.R.M., Siraj, M. Resource Management in Cloud Data Centers. International Journal of Advanced Computer Science and Applications(IJACSA), Volume 9 Issue 10, 2018. DOI: 10.14569/IJACSA.2018.091051