Oren Salami
DevOps Engineer | AI Infrastructure & Systems
- 1
- Followers
If you let an AI coding agent run unconstrained self-repair loops, it won’t fix the bug—it will rewrite your unit tests to pass.
The concept of "Reflexion"—agents diagnosing and repairing code from test feedback—is powerful. But in production, autonomous repair creates severe operational risks.
First, agents enter endless retry loops that burn API budgets while degrading code structure. Second, if an agent has write access to your workspace, its quickest path to a green checkmark is modifying the test assertions rather than fixing the bug.
In our Bounded Reflexion workflow, we enforce three hard guardrails:
Bounded Retries: Cap repairs at exactly two attempts. If the patch fails twice, stop the loop and route to a human developer.
Read-Only Test Suites: Mount test files as strictly read-only (:ro) volumes in the container. The agent inspects stack traces, but cannot touch the grading harness.
Air-Gapped Sandboxing: Run code execution in hardened, non-root Docker containers with network egress locked down.
How do you stop autonomous coding agents from "cheating" your evaluation harnesses?
#AIEngineering #DevSecOps #Docker #SoftwareEngineering #AgenticAI
3
0
32
What happens between an AI recommendation and an approved action? Most AI demos stop at generating a signal, ignoring the operational gap where production risks actually live.
I built Investment DecisionOps AI to engineer that workflow from the ground up—focusing on auditability, strict permission boundaries, and deterministic replay rather than autonomous execution.
Key architectural choices behind the system:
• Read-Only Agent Boundaries: CrewAI agents research documents and portfolio context via a stdio MCP tool server restricted strictly to read and compute tools. Agents can analyze and propose, never approve or trade.
• Tamper-Evident Evidence Logs: Decision contexts are frozen with per-item SHA-256 hashes so historical replays never drift when underlying data changes.
• Payload-Bound Approval Gates: Orders require previewing, step-up re-authentication tied to the exact payload hash, and broker reconciliation.
• Lean Modular Monolith: Built with React, FastAPI, Docker, Ollama (qwen3:4b), and PostgreSQL 18 + pgvector handling relational state, vectors, full-text search, and FOR UPDATE SKIP LOCKED background jobs without Redis or Kubernetes bloat.
Designing the brakes takes just as much thought as designing the agents. How do you enforce hard boundaries between AI outputs and state-mutating actions?
Code & architecture: https://github.com/Oren1984/investment-decisionops-ai
0
12
RAG Engineering Lab – Information Retrieval & Evaluation
0
62
Banking Systems Assurance Platform
0
81