An LLM chatbot that can installed on a local machine and accessed via the command line or network. https://github.com/aldian/llm-chatbot
0 likes · 35 views · Aug 22, 2024
High-throughput local LLM inference API supporting quantized weights (Mistral/LLaMA), streaming tokens, and OpenAI-compatible endpoints.