Triton Server
Abdul Hadi Bharara
Cloud Infrastructure Architect
ML Engineer
Security Manager
Docker
Flask
Python
A Docker-based system that combines Triton servers to serve machine learning models and Flask-based API to manage requests on those models. I was the architect of this project and created a system where each model can have multiple iterations across multiple GPUs and VMs. At the same time, the load-balancing algorithm maintained a balance between them all. The system served over 500,000 requests a day, primarily for high-quality images.
