Hybrid Data Management Architecture with Microservices Integrating SQL and NoSQL Databases for Asset Management Companies
Abstract
In today's financial technology ecosystem, asset management companies require the simultaneous processing of large volumes of structured and unstructured data. Traditional database systems alone are insufficient to address these complex requirements. This study presents a novel hybrid data management framework built on a microservices architecture, integrating SQL and NoSQL databases.
The proposed architecture incorporates an innovative adapter layer that combines the flexible document-based structure of MongoDB with the robust relational capabilities of PostgreSQL. This adapter enables real-time data synchronization between the two database paradigms, leveraging the strengths of both systems. The architecture is designed to support complex data models required for managing non-performing loans in the asset management sector while offering high scalability.
The paper details the key components of the hybrid architecture, data transformation strategies, and inter-service communication protocols. The proposed solution is built on a three-layered microservices architecture—data access layer, synchronization layer, and application layer—ensuring high performance and flexibility while maintaining data consistency. The design adheres to CAP theorem and ACID principles, guaranteeing data integrity and system consistency.
This study introduces a new approach to integrating different database paradigms, addressing the complex data management needs of modern financial systems. The proposed architecture is adaptable not only for the asset management sector but also for other industries with similar hybrid data management requirements.
References
- 1.L. N. Nalla and V. M. Reddy, “SQL vs. NoSQL: Choosing the Right Database for Your Ecommerce Platform,” International Journal of Advanced Engineering Technologies and Innovations, vol. 1, no. 2, pp. 54–69, 2022.
- 2.R. Cattell, “Scalable SQL and NoSQL data stores,” Acm Sigmod Record, vol. 39, no. 4, pp. 12–27, 2011.
- 3.W. Khan, T. Kumar, Z. Cheng, K. Raj, A. Roy, and B. Luo, “SQL and NoSQL Databases Software architectures performance analysis and assessments—A Systematic Literature review. arXiv 2022,” arXiv preprint arXiv:2209.06977.
- 4.E. Brewer, “A certain freedom: thoughts on the CAP theorem,” in Proceedings of the 29th ACM SIGACT-SIGOPS symposium on Principles of distributed computing, 2010, p. 335.
- 5.E. A. Lee, S. Bateni, S. Lin, M. Lohstroh, and C. Menard, “Quantifying and generalizing the CAP theorem,” arXiv preprint arXiv:2109.07771, 2021.
- 6.L. Frank, R. U. Pedersen, C. H. Frank, and N. J. Larsson, “The cap theorem versus databases with relaxed acid properties,” in Proceedings of the 8th International Conference on Ubiquitous Information Management and Communication, 2014, pp. 1–7.
- 7.A. Davoudian, L. Chen, and M. Liu, “A survey on NoSQL stores,” ACM Computing Surveys (CSUR), vol. 51, no. 2, pp. 1–43, 2018.
- 8.C. Strauch, U.-L. S. Sites, and W. Kriha, “NoSQL databases,” Lecture Notes, Stuttgart Media University, vol. 20, no. 24, p. 79, 2011.
- 9.M. T. Özsu and P. Valduriez, Principles of distributed database systems, vol. 2. Springer, 1999.
- 10.J. D. Ullman, A first course in database systems. Pearson Education India, 2007.
- 11.R. Elmasri, Fundamentals of database systems. Pearson Education India, 2008.
- 12.P. Beynon-Davies, Database systems. Springer, 2004.
- 13.C. Zaniolo, Advanced database systems. Morgan Kaufmann, 1997.
- 14.W. Kim, Modern database systems: the object model, interoperability, and beyond. ACM Press/Addison-Wesley Publishing Co., 1995.
Goze, I. M., Erginer, G., Dilber, C., Pozut, B., Ozpinar, A. (2024). Hybrid Data Management Architecture with Microservices Integrating SQL and NoSQL Databases for Asset Management Companies. *Orclever Proceedings of Research and Development*, 5(1), 485-497. https://doi.org/10.56038/oprd.v5i1.586
Bibliographic Info