From Airflow DAGs to a lightweight dashboard-only mode.
I developed an automated ETL pipeline for London weather data. The pipeline handles Extract (REST API polling), Transform (data cleaning & normalization), and Load (SQLite with timestamps). The dashboard shows Current Temp 23.3°C, PM2.5 4.30 µg/m³, historical trends, and raw data table.
Key decisions: switched from full Airflow setup to a lightweight Streamlit app for faster deployment and lower resource usage, while keeping the code clean and reproducible.
Repo structure: dags / src / dashboard / data / scripts - ready to be used as a template for any weather/IoT data project.
From Airflow DAGs to a lightweight dashboard-only mode.
I developed an automated ETL pipeline for London weather data. The pipeline handles Extract (REST API...