Cold start: 17 min → under 3 min. Shard loading: 16 min → 50 sec. Serving cost: ~1/6 of a managed SageMaker equivalent.
Performance-engineered a LLaVA vision-language deployment on AWS as part of a specialised strike team, with the full write-up published on LinkedIn (linked under Completed Work).
Reduced end-to-end cold start from ~17 minutes to under 3 (66%) via custom Deep Learning AMIs (−32% immediately), ephemeral-storage model caching with parallel aria2c downloads (checkpoint loading 16 min → 50 sec), and 4-bit NF4 quantization (shard loading 41s → 18s). Along the way, diagnosed a storage-path bug where checkpoint shards kept re-downloading despite being baked into the AMI.
0 likes · 0 views · Oct 18, 2026
Cold start: 17 min → under 3 min. Shard loading: 16 min → 50 sec. Serving cost: ~1/6 of a managed SageMaker equivalent. Performance-engineered a LLaVA vision...
