As one of the largest holdings in the Europe, Wepa required implementing a full data pipeline so that they would gather data from different offices and factories, store and process them respectively, and make them accessible via APIs.
I designed the entire data pipeline for Wepa. Raw data were gathered from MSSQL sources via Kafka and then transferred into a Flink instance. Processed data was then stored on a CosmosDB instance. We implemented APIs to make these data available for further usage.
I implemented an interactive dashboard in which users from various departments can register and request to gain access to the mentioned APIs.
Like this project
0
Posted Aug 12, 2024
For Wepa, Sajad designed a data pipeline using Kafka, Flink, and CosmosDB, creating APIs and an interactive dashboard for data access across departments.