Case Study

Building a Data Lake

Ekspertki ds. chmury wdrażające architekturę chmury hybrydowej

Industry
  • Banking & Finance (a major French commercial bank)

 

Challenge
  • Building a data lake for an instant payment project

Solution

  • Design and implementation of the data lake architecture;
  • Implementation of data pipelines using Hadoop ecosystem technologies;
  • Development of the data storage layer using Kassandra;
  • Setting up the data visualisation and analysis platform with ELK;
  • Technical environment: the project was developed using the Hadoop, Java, Kassandra and ELK ecosystem. It was implemented in a hybrid cloud environment.

Technologies

  • Java

  • Springboot

  • ELK

  • HDFS

  • Kafka

  • Spark

  • Kassandra


Keys to success

  1. Mastery of the Hadoop ecosystem: our expertise in data processing and storage technologies, particularly Hadoop, enabled us to design and implement a reliable and scalable system for the bank's data lake;
  2. Use of hybrid cloud: we implemented a hybrid cloud architecture to enable the bank to deploy large-scale data processing and analysis services quickly and easily. This approach optimised costs while guaranteeing data security and compliance;
  3. Close collaboration with business teams: we worked closely with the bank's business teams to understand their data processing needs and goals. This collaboration resulted in the creation of a data lake tailored to the bank's needs in terms of performance, capacity and functionality, and facilitated the adoption of the solution by end users.