Project Description:
The main goal of this project is to provide data to the application which is used for calculating commission.
Project Tasks:
Create an ETL pipeline that reads the data from CSV and REST API and transforms the data per the requirement and loads the data into the target tables in MS SQL Server.
Schedule a job in Databricks to run this pipeline on a daily basis.
Create an email notification system that emails the unprocessed records as a CSV attachment.
Create Ad-hoc reports for visualizing the data.
Create Paginated reports to view Commission statements.
Project Stack:
Python
SQL
Power BI
Azure
Databricks
MS SQL Server
SendGrid
✨Unlock your data's potential. ETL services available now!✨
Like this project
0
Posted Feb 24, 2023
Design an etl pipeline that reads the data from multiple sources & visualizes the data by creating Ad-hoc reports & create Paginated reports to view statement.
Payer Data Load
Migration of Power BI Reports from SQL Server to Databricks
PDF Report Using Python