groundcheck — a $0 eval judge that matches the frontier
Daniel Enache
I replaced a frontier eval judge with a 1.5B model I fine-tuned on my laptop. It agreed with the frontier judge 100% of the time — for $0.
Every serious AI system needs a groundedness judge: something that reads a (question, retrieved context, answer) triple and decides whether every claim is actually supported. Usually that judge is a frontier-model API call — so every CI run costs money, leaks data off-box, and gets rate-limited.
So I trained a small one to do exactly that job, locally.
The build (open source):
A dataset with labels correct by construction — no human labelling, no LLM labelling.
A QLoRA fine-tune of a 1.5B model, 4-bit, ~4.5 minutes on an 8 GB laptop GPU.
The rule: the fine-tune ships only if the eval suite says it beats the baseline. Training is cheap; judgment is the product.
The result, on held-out cases the model never saw:
base model, zero-shot: F1 0.083 (it hedges; handles refusals 0% of the time)
fine-tuned: F1 1.000
Then the question that actually matters — does it match a frontier judge? On 60 held-out cases, my local judge and a frontier teacher both scored 1.000 accuracy, with 100% agreement. Cost: $0.026 / 1,000 calls for the frontier judge, $0.00 for mine.
Honest scope: the corpus is synthetic (labels correct by construction) — this proves the pipeline end to end. Next: real RAG traces.
The loss curve doesn't decide what ships. The eval suite does.
→ Repo + results: https://github.com/enached134-ctrl/groundcheck
Like this project
Posted Jul 4, 2026
A groundedness-judge model fine-tuned (QLoRA, 4-bit) on a laptop GPU into a local eval judge that agrees with a frontier judge 100% of the time — at $0 per call. Base F1 0.083 → fine-tuned 1.000. Open source, gated by its own evals.
