Ashutosh Gavali
Python Data Engineer | AI Automation | Web Scraping Expert
New to Contra
Ashutosh is ready for their next project!
AI Job Application Automation with Real-Time Email Tracking
0
6
Business Intelligence Dashboard | React + FastAPI
0
8
Building efficient data pipelines is not only about handling large volumes of data—it's about doing it with minimal resources and low latency.
In this project, I built a lightweight market data pipeline capable of processing 10+ million market ticks per day while running on only 1GB RAM.
Exchange WebSocket Streams
↓
Concurrent Go Parser
↓
Data Validation & Normalization
↓
Apache Parquet (Columnar Compressed Storage)
↓
DuckDB Analytics
↓
FastAPI APIs & Live Dashboard
Tech Stack
Python • Go • FastAPI • DuckDB • Apache Parquet • Docker • WebSockets
This project strengthened my expertise in high-performance data engineering, concurrent programming, distributed systems, and real-time analytics.
Currently building scalable Data Engineering, Backend Engineering, and Low-Latency Trading Infrastructure solutions.
#Python #DataEngineering #Go #DuckDB #ApacheParquet #FastAPI #Docker #WebSockets #RealTimeData #Backend #SoftwareEngineering
0
14
Built a high performance low-latency crypto arbitrage engine for real-time market monitoring. The system uses Rust and Go for ultra fast processing, Python for orchestration, WebSockets for live market feeds, mempool monitoring, flash loan support, and a real-time dashboard for tracking arbitrage opportunities across decentralized exchanges.
1
23