Extraction of data from various sources such as relational databases, cloud storage services, REST APIs, and file formats (CSV, JSON, PARQUET) using Python programming language.
Transformation of extracted data into a consistent and structured format suitable for further analysis and processing.
Loading of transformed data into MS SQL Server database or Azure cloud storage for secure and reliable storage.
Development of ETL pipelines to automate the data extraction, transformation, and loading process using Azure Databricks.
What's included
ETL Deliverables
Code that implements the ETL pipeline, including the extraction, transformation, and loading of data.