Azure Data Factory & Databricks Lakehouse Implementation
Pranay K
Azure Data Factory & Databricks Lakehouse
Your role
Cloud Data Engineer
Project description
Implemented a lakehouse ingesting 2 TB/day. Built ADF pipelines for raw-to-Delta ingestion, automated schema evolution, and Delta Optimizer jobs in Databricks. Added row-level ACLs, cost-based autoscaling and Terraform IaC, lowering compute spend 30 % and shrinking batch latency from 2 h to 25 min.
Skills and deliverables
Azure Data Factory, Databricks, Delta Lake, Terraform, SQL
Like this project
Posted May 25, 2025
Implemented a lakehouse ingesting 2 TB/day with Azure Data Factory and Databricks.
Likes
0
Views
0
