Blog: Serving LLMs on EKS after Kubernetes 1.37 by Arjun KanojiaBlog: Serving LLMs on EKS after Kubernetes 1.37 by Arjun Kanojia

Blog: Serving LLMs on EKS after Kubernetes 1.37

Portfolio project
Arjun Kanojia
A technical guide I wrote on running GPU inference on Amazon EKS with Kubernetes 1.37.

What it covers

Requesting GPUs with DRA ResourceClaims and Karpenter
MIG versus time-slicing for sharing GPUs
Inference-aware routing with the Gateway API Inference Extension
HPA scale-to-zero and cold starts
Device taints, failure modes and a rollout checklist
More posts on running AWS and Kubernetes platforms in production: arjunkanojia.com/blog

1 like · 0 views · Nov 14, 2026

Technical guide on GPU inference on EKS: DRA with Karpenter, MIG vs time-slicing, inference-aware routing, scale-to-zero and a rollout checklist.

Tags