Data Engineer, Infosys Ltd
Akhil Prasad
Utilized Scoop for ingesting data from various RDBMS systems into Data Lake.
Developed Spark applications utilizing Spark-SQL to for transforming and loading data into Hive.
Used optimizations like Partitioning, Bucketing and Broadcast Joins to improve performance ranging from 40% to 350%
Used Autosys for workflow management and orchestrating spark jobs.
Created technical design, data model and documentation of the solution.
Involved in overall SDLC, post-production support and maintenance of the application.
Like this project
Posted Apr 10, 2024
Built high-performance data pipelines to bring data from RDBMS systems into data lake using Sqoop & Spark and Improved efficiency (up to 350%)
Likes
0
Views
2
Clients
Infosys
