Muhammad Arsalan
Data Engineer | Web Scraping | SQL | BI Dashboards
Ready for work
Muhammad is ready for their next project!
Problem:
Financial data from stocks and crypto APIs was scattered, refreshed manually, and not ready for analytics or ML use.
Solution:
Built an Apache Airflow pipeline to collect, transform, validate, and load real-time financial data from multiple APIs into PostgreSQL, MongoDB, AWS RDS, and Qdrant.
Tools:
Apache Airflow, Python, PostgreSQL, MongoDB, AWS RDS, Qdrant, APIs
Result:
Automated sub-hourly data refresh, processed thousands of records daily, and delivered clean data for dashboards, analytics, and vector search.
0
63
Problem:
Batch data processing was not suitable for real-time analytics and scalable cloud-based data ingestion.
Solution:
Created a real-time streaming pipeline using Kafka on AWS EC2, stored processed data in S3, cataloged it with AWS Glue, and queried it with Amazon Athena.
Tools:
Python, Apache Kafka, AWS EC2, Amazon S3, AWS Glue, Amazon Athena, Pandas
Result:
Built an end-to-end cloud data streaming workflow that supports real-time ingestion, storage, cataloging, and SQL-based analytics.
0
61
Problem:
Finance data was difficult to analyze across sales, profit, orders, discounts, countries, and customer segments.
Solution:
Created an interactive Power BI dashboard with KPI cards, sales trends, profit analysis, country performance, and segment-level insights.
Tools:
Power BI, Power Query, DAX, Excel, Data Modeling
Result:
Delivered a clear executive dashboard for tracking financial performance, identifying trends, and making faster business decisions.
0
51
Problem:
Users needed a faster way to find the most relevant counselor based on specialization and available data.
Solution:
Built a recommendation workflow using PySpark for data processing and Redis for fast lookup and recommendation serving.
Tools:
Python, PySpark, Redis, Docker, Docker Compose
Result:
Created a containerized recommendation system that processes counselor data and returns relevant matches efficiently.
0
47