Kubernetes Data Pipeline Deployment
Archit Jhingan
0
Data Modelling Analyst
Data Science Specialist
Data Engineer
Google BigQuery
Kubernetes
Python
took a lead role in designing and implementing an efficient data pipeline using Kubernetes. The primary objective was to automate and streamline data movement processes, ensuring scalability, reliability, and maintainability.
Kubernetes YAML Template Creation:
Dynamic Configuration with Config Table:
Code Implementation for Data Movement:
Scalability and Parallel Processing:
Error Handling and Logging:
Documentation and Knowledge Transfer:
DevOps:
Like this project
0
Posted Feb 3, 2024
My work on this project involved creating a highly scalable and configurable data pipeline using Kubernetes.
Likes
0
Views
7
Tags
Data Modelling Analyst
Data Science Specialist
Data Engineer
Google BigQuery
Kubernetes
Python
Archit Jhingan
Data Engineer | Kubernetes, Python, Airflow, GCP
Enhancing Data Alerting through a dynamic system
Marketing Pipelines using Adverity