Data Engineer, Horizon Media

Ankit B

0

Data Modelling Analyst

Data Scientist

Data Engineer

Execute OAuth and REST API calls using Postman API Platform to acquire client data in JSON format.
Develop Python scripts for data extraction and categorization into S3 buckets for incremental and full load using Airflow, AWS infrastructure and Big Data (Hive, Presto, Spark).
Manage codebase on GitHub, ensuring proper version control and documentation in Confluence.
Establish Fivetran connections to facilitate ETL processes from S3 to Snowflake Data Warehouse.
Collaborate on data transformations within Snowflake, optimizing query performance for analytics.
Utilize DBT for efficient data modeling and transformations in both Dev and Prod environments.
Tools: SQL Server, Airflow, DBT, AWS (S3, Lambda, Glue, Step Functions, Redshift), Fivetran, Snowflake, GitHub, Confluence, Postman, Python (JSON, PyTest, PyCharm, Pandas), Jira
Like this project
0

Posted Feb 12, 2024

Develop data pipelines to ingest / automate WebAPI data from USA based retail clients and populate data warehouses, data marts for various business teams.

Likes

0

Views

15

Clients

Horizon Media Group

Tags

Data Modelling Analyst

Data Scientist

Data Engineer

Data Engineer, Meta Platforms Inc (Facebook)
Data Engineer, Meta Platforms Inc (Facebook)
Data Engineer, Expedia Group Inc
Data Engineer, Expedia Group Inc
Optimizing Data Warehousing for Fortune 500 Companies
Optimizing Data Warehousing for Fortune 500 Companies
PowerBI | Tableau | Looker | ThoughtSpot
PowerBI | Tableau | Looker | ThoughtSpot