Worked with Enterprise Data Pipeline

serhan sari

0

Automation Engineer

Backend Engineer

Data Engineer

AWS Lambda

Python

Snowflake

● Worked in AWS environment (S3 / EC2 / ECS / Lambda / RDS)
● Used PySpark to process batch data pipelines
● Wrote unittest and integration test with pytest and moto
● Fetched all the snowflake-config- repos in GitHub into a base repo with gitmodules and Automate the CICD pipeline with Jenkins API, Git, and Python
● Worked with Snowflake to clear old entitlements from snowflake via python and pandas
Like this project
0

Posted Nov 25, 2023

Utilized PySpark for batch data, and automated CICD pipeline with Jenkins, Git, and Python. Managed Snowflake configurations and entitlements.

Likes

0

Views

4

Clients

Progressive Insurance

Tags

Automation Engineer

Backend Engineer

Data Engineer

AWS Lambda

Python

Snowflake

Personal Drive on AWS S3
Personal Drive on AWS S3
Investing Platform Conversion
Investing Platform Conversion
Volkswagen website (https://www.vw.com)
Volkswagen website (https://www.vw.com)