Kubernetes Data Pipeline Deployment
Archit Jhingan
Data Modelling Analyst
Data Science Specialist
Data Engineer
Google BigQuery
Kubernetes
Python
took a lead role in designing and implementing an efficient data pipeline using Kubernetes. The primary objective was to automate and streamline data movement processes, ensuring scalability, reliability, and maintainability.
Kubernetes YAML Template Creation:
Dynamic Configuration with Config Table:
Code Implementation for Data Movement:
Scalability and Parallel Processing:
Error Handling and Logging:
Documentation and Knowledge Transfer:
DevOps:
