Commission Calculation System
Hariharan Sambath
Data Engineer
Data Visualizer
Database Specialist
Microsoft Power BI
Microsoft SQL Server
Python
Project Description:
The main goal of this project is to provide data to the application which is used for calculating commission.
Project Tasks:
- Create an ETL pipeline that reads the data from CSV and REST API and transforms the data per the requirement and loads the data into the target tables in MS SQL Server.
- Schedule a job in Databricks to run this pipeline on a daily basis.
- Create an email notification system that emails the unprocessed records as a CSV attachment.
- Create Ad-hoc reports for visualizing the data.
- Create Paginated reports to view Commission statements.
Project Stack:
- Python
- SQL
- Power BI
- Azure
- Databricks
- MS SQL Server
- SendGrid