Projects using Apache Airflow in KarachiProjects using Apache Airflow in KarachiProblem:
Financial data from stocks and crypto APIs was scattered, refreshed manually, and not ready for analytics or ML use.
Solution:
Built an Apache Airflow pipeline to collect, transform, validate, and load real-time financial data from multiple APIs into PostgreSQL, MongoDB, AWS RDS, and Qdrant.
Tools:
Apache Airflow, Python, PostgreSQL, MongoDB, AWS RDS, Qdrant, APIs
Result:
Automated sub-hourly data refresh, processed thousands of records daily, and delivered clean data for dashboards, analytics, and vector search.
