The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started
Back to feedPost
I fine-tuned a 1.5B model into a groundedness judge on my laptop — and it matched a frontier judge 100% of the time, for $0 per call.
Every RAG or agent eval gate needs a judge: something that reads a (question, context, answer) triple and rules whether every claim is actually supported. Usually that judge is a frontier-model API call — so every CI run costs money, leaks data off-box, and gets rate-limited.
So I distilled that capability into a model that runs locally, in CI, on every commit.
groundcheck (open source):
• Dataset with labels correct by construction — no human labelling, no LLM labelling.
• QLoRA fine-tune of a 1.5B model, 4-bit, ~4.5 min on an 8 GB laptop GPU.
• The rule: the fine-tune ships only if the eval suite says it beats the baseline.
Results on held-out cases the model never saw:
• base model, zero-shot: F1 0.083 (it hedges; handles refusals 0% of the time)
• fine-tuned: F1 1.000
Then the question that actually matters — does it match a frontier judge? On 60 held-out cases, my local judge and a frontier teacher both scored 1.000 accuracy, with 100% agreement. Cost: $0.026 / 1,000 calls for the frontier judge, $0.00 for mine.
Honest scope: the corpus is synthetic (labels correct by construction) — this proves the pipeline end to end. Next: real RAG traces.
The loss curve doesn't decide what ships. The eval suite does.
Repo + results → https://github.com/enached134-ctrl/groundcheck
github.com
GitHub - enached134-ctrl/groundcheck
Contribute to enached134-ctrl/groundcheck development by creating an account on GitHub.
The network for creativity
Join 1.25M professional creatives like you
Connect with clients, get discovered, and run your business 100% commission-free
Creatives on Contra have earned over $150M and we are just getting started
Related posts
Most Claude Code sessions die in the middle — not because the model is weak, but because the brief was vague and the scope kept growing.
Here's the six-step loop I use so a feature lands as a reviewable PR instead of a sprawling rewrite:
01 BRIEF — write the win condition, in-scope files, and what must not change
02 MAP — explore the call graph; confirm the plan before any edit
03 SLICE — one vertical cut that proves the behavior; defer refactors
04 PROVE — failing test first, then implement until green
05 GUARD — diff the change against the original brief; reject drive-bys
06 LAND — conventional commit + risk note; human still owns merge
If scope drifts, return to MAP. Never keep editing blind.
#Claude #ClaudeCode #AIAutomation
Three posts on my feed this week circled the same problem from different angles: an agent that repairs code by rewriting its own tests, a CSV reconciler that has to keep every unmatched SKU instead of silently dropping it, and a support agent that escalates weak matches to a human. The pattern repeats. Autonomy needs a hard boundary and an auditable trail. When I build execution bots the rule I keep is: the agent proposes, the runner disposes. Every write goes through a diffable record, and any action that cannot be verified is escalated, not guessed. Green tests only count if the test files are read-only. What boundary do you enforce first once an agent has write access?
The fix for an agent that keeps getting your design system wrong isn't a better prompt.
It's putting the rules where the agent reads them.
In the repo. In skills. In checks it can run itself.
Writing a course about exactly this.
What sells it for me is the refusal telling the agent exactly where the token should go. A bare "no hex" error usually just sends it looping. Does guard.mjs run only as an agent hook, or in pre-commit too so it catches humans as well?