rag-agents: self-hosted RAG with citations and an eval harness by Maxim Popelnitskiyrag-agents: self-hosted RAG with citations and an eval harness by Maxim Popelnitskiy

rag-agents: self-hosted RAG with citations and an eval harness

Portfolio project
Maxim Popelnitskiy
Self-hosted RAG (retrieval-augmented generation) system, open source: github.com/Gegirhasut/rag-agents. Create an agent, upload documents (txt, fb2, epub, pdf, docx up to 100 MB) and ask questions. Answers come only from those documents and are streamed with [n] citations to the exact section. If the answer is not in the documents, the agent says so instead of making something up.
Retrieval: Ollama bge-m3 embeddings, Qdrant with per-agent isolation (another user's resource returns 404, tested on every endpoint), DeepSeek for generation. Ingestion runs on Celery + RabbitMQ: idempotent, with retries, a dead-letter queue, a sweeper and structure-aware chunking. The RAG core is written by hand, without LangChain.
Evaluation harness: a 59-question golden set, hit@k / recall@k / MRR and an LLM judge. Baseline: hit@8 0.83, recall@8 0.79, MRR@10 0.57, faithfulness 0.95, refusal precision 0.93. Observability: Langfuse traces, Prometheus metrics and cost tracking (about $0.001 per answer).
Stack: Python, FastAPI, HTMX + SSE, Qdrant, Ollama, DeepSeek, Celery, RabbitMQ, PostgreSQL, Redis, Docker Compose.

0 likes · 0 views · Nov 11, 2026

Open-source RAG over your own documents: streamed answers with citations, refusal when the docs lack the answer, isolated agents, and a measured eval harness.

Tags