The challenge
A Belgian federal public institution wanted its staff to benefit from generative AI, but its data could not be sent to external AI providers. Public cloud chatbots were not an option.
What I built
Self-hosted LLM infrastructure on the organisation’s own GPU servers (vLLM, LiteLLM, Ollama)
An internal chatbot that answers from SharePoint documents through RAG, with the sources behind each answer
A built-in translation model for multilingual use
An on-premise vs cloud comparison (Azure AI Foundry, AWS Bedrock) to guide infrastructure decisions
Results
Staff use generative AI on internal documents with no data leaving the infrastructure
One shared LLM gateway that other internal AI projects can reuse
A clear cost and risk picture for on-premise vs cloud AI
Stack
vLLM, LiteLLM, Ollama, GPU servers, RAG, SharePoint, open-source LLMs.
0 likes · 0 views · Nov 10, 2026
Internal AI assistant running on self-hosted LLMs, with RAG over SharePoint and translation, so no data ever leaves the organisation.
