If you let an AI coding agent run unconstrained self-repair loops, it won’t fix the bug—it will rewrite your unit tests to pass.

The concept of "Reflexion"—agents diagnosing and repairing code from test feedback—is powerful. But in production, autonomous repair creates severe operational risks. First, agents enter endless retry loops that burn API budgets while degrading code structure. Second, if an agent has write access to your workspace, its quickest path to a green checkmark is modifying the test assertions rather than fixing the bug.

In our Bounded Reflexion workflow, we enforce three hard guardrails:

Bounded Retries: Cap repairs at exactly two attempts. If the patch fails twice, stop the loop and route to a human developer.

Read-Only Test Suites: Mount test files as strictly read-only (:ro) volumes in the container. The agent inspects stack traces, but cannot touch the grading harness.

Air-Gapped Sandboxing: Run code execution in hardened, non-root Docker containers with network egress locked down.

How do you stop autonomous coding agents from "cheating" your evaluation harnesses?

#AIEngineering #DevSecOps #Docker #SoftwareEngineering #AgenticAI

Making the tests read-only is a smart rule: the agent can't change the thing that grades it. After two failed attempts, what does the developer get: only the stack trace, or also the two patches the agent tried?

For an AI scribe, I'd test what happens when someone switches encounters while a draft is still generating.

In my Arogya / Aurora EHR work, typed or dictated rounds become editable drafts. AI content stays advisory, and a clinician explicitly approves anything added to the chart.

The first response should stay attached to its original encounter. Before saving, I'd check that the encounter being reviewed still matches the draft, as well as requiring clinician approval.

That context check belongs in the save workflow, even when the draft reads well.