Built data pipelines on AWS cloud
I enabled Full and Incremental data load for TB's data from legacy MySQL database to AWS RDS postgres instance I independently worked for developing, modifying scala code for enabling near real time data pipeline. It helped stakeholders to migrate data from on-prem db to cloud datalake. I came up with these solutions after successful proof of concepts.
Tip: Try to keep this description brief and under 150 words by highlighting the best parts of this project— recruiters don’t spend much time reading detailed descriptions! 💪
Like this project
0
Posted Nov 13, 2023
Built scala spark framework to enable full and incremental data load to redshift. Enabled near real time pipelines using lambda functions.
Data scrapped from 100+ APIs
Implementing Real-time streaming pipeline