Data Engineering for a Sustainability-Focused Startup
Will Danckwerts
As a Data Engineer for a sustainability-focused startup, I developed data validation pipelines and ETL processes using Python. This work included implementing unit tests to automate data verification and augmentation, ensuring the integrity of geospatial business data for a map-based application.
To optimise development workflows, I iteratively improved the data infrastructure and DevOps architecture, leveraging AWS services such as S3, Lambda, PostgreSQL, and ECR. I utilised Docker for containerisation and GitHub Actions for CI/CD, ensuring secure, efficient, and scalable deployments. I also contributed to enhancing the backend architecture with a Django API, enabling seamless integration of data services.
This role demonstrated my ability to work at a professional level on impactful, technically challenging projects while maintaining security and coding standards in a collaborative, mission-driven environment.
Like this project
Posted Dec 4, 2024
Developed data pipelines, ETL processes, and DevOps infrastructure in Python and AWS for a sustainability-focused startup, ensuring secure, scalable, workflows.
