Data Engineer, Horizon Media

Execute OAuth and REST API calls using Postman API Platform to acquire client data in JSON format.

Develop Python scripts for data extraction and categorization into S3 buckets for incremental and full load using Airflow, AWS infrastructure and Big Data (Hive, Presto, Spark).

Manage codebase on GitHub, ensuring proper version control and documentation in Confluence.

Establish Fivetran connections to facilitate ETL processes from S3 to Snowflake Data Warehouse.

Collaborate on data transformations within Snowflake, optimizing query performance for analytics.

Utilize DBT for efficient data modeling and transformations in both Dev and Prod environments.

Tools: SQL Server, Airflow, DBT, AWS (S3, Lambda, Glue, Step Functions, Redshift), Fivetran, Snowflake, GitHub, Confluence, Postman, Python (JSON, PyTest, PyCharm, Pandas), Jira