The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started
Back to feedPost
๐ง ๐ง๐ต๐ถ๐ป๐ธ๐ถ๐ป๐ด ๐ผ๐ณ ๐ฟ๐๐ป๐ป๐ถ๐ป๐ด ๐ฎ๐ป ๐๐๐ ๐ผ๐ป ๐๐๐ฏ๐ฒ๐ฟ๐ป๐ฒ๐๐ฒ๐?
Hereโs a rough outline of what it takes
๐๐ผ๐ป๐๐ฎ๐ถ๐ป๐ฒ๐ฟ๐ถ๐๐ฒ ๐๐ต๐ฒ ๐บ๐ผ๐ฑ๐ฒ๐น โ Start with something like a quantized Llama2, Mistral, or a custom fine-tuned model.
Use a lightweight serving framework (like text-generation-inference, vLLM, or TGI) and wrap it in a Docker container.
๐๐ฃ๐จ ๐๐ฐ๐ต๐ฒ๐ฑ๐๐น๐ถ๐ป๐ด โ Use node selectors or taints/tolerations to schedule pods on GPU-enabled nodes
๐๐๐๐ผ๐๐ฐ๐ฎ๐น๐น๐ถ๐ป๐ด โ Use KEDA or HPA to scale pods based on requests per second or GPU utilization. LLM workloads are spiky, so dynamic scaling saves $$.
๐๐ฃ๐ ๐๐ฎ๐๐ฒ๐๐ฎ๐ / ๐๐ผ๐ฎ๐ฑ ๐๐ฎ๐น๐ฎ๐ป๐ฐ๐ฒ๐ฟ โ Expose your model via a gateway (like Istio, NGINX, or even API Gateway in hybrid setups).
#LLM #Kubernetes #DevOps #MLOps #CloudNative #K8s #OpenSource
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started
Trending
maxearnings
The next frontier of payments is live on Contra. How are you maximizing revenue?
freelancerlife
Freelancer life is wins, pivots, and everything in between. Whatโs yours right now?
aidesignflow
AI tools are redefining how designer work. What does your workflow look like?
micrographics
Micrographics started as utility - barcodes, packaging, instruction labels. How would you use them?
aivideo
AI video tools are moving at warp speed. What tools are you using?