Worked with Enterprise Data Pipeline
0
Automation Engineer
Backend Engineer
Data Engineer
AWS Lambda
Python
Snowflake
● Worked in AWS environment (S3 / EC2 / ECS / Lambda / RDS)
● Used PySpark to process batch data pipelines
● Wrote unittest and integration test with pytest and moto
● Fetched all the snowflake-config- repos in GitHub into a base repo with gitmodules and Automate the CICD pipeline with Jenkins API, Git, and Python
● Worked with Snowflake to clear old entitlements from snowflake via python and pandas
Like this project
0
Posted Nov 25, 2023
Utilized PySpark for batch data, and automated CICD pipeline with Jenkins, Git, and Python. Managed Snowflake configurations and entitlements.
Likes
0
Views
4
Clients
Progressive Insurance
Tags
Automation Engineer
Backend Engineer
Data Engineer
AWS Lambda
Python
Snowflake
Personal Drive on AWS S3
Investing Platform Conversion
Volkswagen website (https://www.vw.com)