Problem: Financial data from stocks and

Problem: Financial data from stocks and crypto APIs was scattered, refreshed manually, and not ready for analytics or ML use.

Solution: Built an Apache Airflow pipeline to collect, transform, validate, and load real-time financial data from multiple APIs into PostgreSQL, MongoDB, AWS RDS, and Qdrant.

Tools: Apache Airflow, Python, PostgreSQL, MongoDB, AWS RDS, Qdrant, APIs

Result: Automated sub-hourly data refresh, processed thousands of records daily, and delivered clean data for dashboards, analytics, and vector search.