Worked with Enterprise Data Pipeline
serhan sari
● Worked in AWS environment (S3 / EC2 / ECS / Lambda / RDS)
● Used PySpark to process batch data pipelines
● Wrote unittest and integration test with pytest and moto
● Fetched all the snowflake-config- repos in GitHub into a base repo with gitmodules and Automate the CICD pipeline with Jenkins API, Git, and Python
● Worked with Snowflake to clear old entitlements from snowflake via python and pandas
Like this project
Posted Nov 25, 2023
Utilized PySpark for batch data, and automated CICD pipeline with Jenkins, Git, and Python. Managed Snowflake configurations and entitlements.
Likes
0
Views
8
Clients
Progressive Insurance
