Campaigns Performance Evaluator

Project Overview

As a Tech Leading Data Engineer, I spearheaded the Campaigns Performance Evaluator project for a prominent Silicon Valley startup. This initiative was crucial for enhancing the effectiveness of marketing campaigns across diverse customer groups. By transitioning ETL processes to a more efficient Data Lake environment utilizing Databricks, I facilitated significant cost savings and elevated the client’s capability to analyze campaign data more precisely and effectively.

Technologies Utilized:

• Languages & Tools: Python, SQL, Apache Airflow, Apache Spark, Data Warehouse, Apache Kafka, Snowflake, ETL

Project Details

The project primarily focused on developing a comprehensive metrics system for marketing campaigns. Key responsibilities included optimizing ETL processes, transitioning jobs from Snowflake to a Data Lake environment, and implementing Databricks for enhanced ETL processes. These changes led to an approximate annual cost saving of several hundreds of thousands dollars, improved budget management, and enhanced data analysis capabilities.

Accomplishments

• Leadership: Led the Campaigns Performance Evaluator project, driving the overall vision and execution.
• Optimization: Directed the transition of ETL processes, achieving significant cost reductions.
• Enhancement: Improved data analysis capabilities, providing more nuanced and effective campaign performance metrics.
• Efficiency: Demonstrated effective budget management through operational cost reduction while enhancing efficiency.