As one of the largest holdings in the Europe, Wepa required implementing a full data pipeline so that they would gather data from different offices and factories, store and process them respectively, and make them accessible via APIs.
I designed the entire data pipeline for Wepa. Raw data were gathered from MSSQL sources via Kafka and then transferred into a Flink instance. Processed data was then stored on a CosmosDB instance. We implemented APIs to make these data available for further usage.
I implemented an interactive dashboard in which users from various departments can register and request to gain access to the mentioned APIs.
Like this project
Posted Aug 12, 2024
For Wepa, Sajad designed a data pipeline using Kafka, Flink, and CosmosDB, creating APIs and an interactive dashboard for data access across departments.