Data Engineering (Multiple)

Created base framework for live search on Job Portal containing job positions and candidate profiles. The framework extracts data from MongoDB, performs multi-level enrichments to form a data map to be pushed to Elastic Search repository which provides live search data to web. Project Link.

Designed and developed framework to build data ETL pipelines

Achieved high speed indexing performance (4M docs/minute)

Designed the system to work for both Day0 data load and live sync

Designed and developed a platform for an adapter module which performs calculations on employee’s leave records, perform business validations, transform the data to ADP readable form and supply the extracts to ADP. The extracts supplied to ADP reflects on the employee’s salary, hence accuracy and timeliness were the key things in this project.

Developed an analogous system using cutting edge technologies that provide fresh user experience and enlightening performance

Analyzed existing data in Oracle, extract critical information from it and design data models for the new system

Developed a base framework which can migrate the data from Oracle to MongoDB