Built evalflow — an open-source pytest-style quality gate for LLMs. Catches prompt regressions before production by diffing live outputs against a saved baseline. Live on PyPI. github.com/emartai/evalflow
Logging every cleaning action is the part I like most: when a number in the report looks strange, you can follow it back to the fix that changed it. With 585 issues, how did you decide which ones the script fixes alone and which ones it only reports?
What sells it for me is the refusal telling the agent exactly where the token should go. A bare "no hex" error usually just sends it looping. Does guard.mjs run only as an agent hook, or in pre-commit too so it catches humans as well?