Performance-Engineered AWS LLaVA Deployment and Cold-Start ReductionPerformance-Engineered AWS LLaVA Deployment and Cold-Start Reduction
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started
Cold start: 17 min → under 3 min. Shard loading: 16 min → 50 sec. Serving cost: ~1/6 of a managed SageMaker equivalent.
Performance-engineered a LLaVA vision-language deployment on AWS as part of a specialised strike team, with the full write-up published on LinkedIn (linked under Completed Work).
Reduced end-to-end cold start from ~17 minutes to under 3 (66%) via custom Deep Learning AMIs (−32% immediately), ephemeral-storage model caching with parallel aria2c downloads (checkpoint loading 16 min → 50 sec), and 4-bit NF4 quantization (shard loading 41s → 18s). Along the way, diagnosed a storage-path bug where checkpoint shards kept re-downloading despite being baked into the AMI.
Post image
Post image
Back to feed
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started