Auditable AI Support Operations — FastAPI, Next.js & RAG by Yusuke MiyajimaAuditable AI Support Operations — FastAPI, Next.js & RAG by Yusuke Miyajima

Auditable AI Support Operations — FastAPI, Next.js & RAG

Portfolio project
Yusuke Miyajima
60-second walkthrough of the real operations dashboard, persisted trace, human-review queue, and verified failure behavior.
60-second walkthrough of the real operations dashboard, persisted trace, human-review queue, and verified failure behavior.
Auditable AI support triage with policy-gated actions Business problem A fluent AI answer is not enough for a support operation. Operators need to know which policy supported it, why a write action was approved or blocked, whether provider failure was mistaken for missing evidence, and whether one customer's data can ever reach another customer's run. Delivery • FastAPI workflow with explicit classification, retrieval, citation, policy, and action stages • Credential-derived tenant isolation before retrieval or tool access • Versioned knowledge, inspectable retrieval, and exact-quote citation validation • Separate retry, repair, deadline, call-count, and cost ceilings around model calls • Human review for risky writes, conflicting evidence, low confidence, tool failure, and injection • Content-free metrics and persisted traces for operational review • TypeScript/Next.js operations console for metrics, run traces, evidence, knowledge versions, and review decisions, with the tenant credential kept behind the server boundary Verified evidence • 49 backend tests, 2 frontend contract tests, and 3 Chromium E2E tests pass across contracts, failure boundaries, tenant isolation, injection filtering, pagination, and reviewer workflows • A scripted failure demonstration passes 5/5 scenarios • The clean Docker Compose path passes 3/3 public API checks with PostgreSQL, and all four operations-console routes return HTTP 200 • The deterministic end-to-end evaluation passes 8/8 scenarios • Four seeded defects are detected, including fabricated citations and an extra retry • The measured optional hybrid retrieval path reaches Recall@4 1.00, MRR 0.95, and unsupported-query empty-result accuracy 1.00 on the checked-in evaluation set Client fit The deliverable is not merely a prompt or an API. It gives an operator one inspectable path from metrics to a run's evidence and tool trace, then through a human decision without exposing the tenant credential to the browser. The same design applies to ticketing, CRM, onboarding, and other AI-assisted operations. This is a portfolio demonstration; it does not claim paid-provider answer quality or production traffic. Completed work https://github.com/yuten0901/ai-support-triage Need a production-ready AI support system? My Reliable RAG & AI Support Automation with Evaluation service covers grounded retrieval, approval gates, evaluation, observability, and deployment handoff. https://contra.com/s/26UqocOO-reliable-rag-and-ai-support-automation-with-evaluation

0 likes · 2 views · Aug 26, 2026

Built an auditable FastAPI + Next.js AI support console with tenant isolation, grounded traces, human approval, PostgreSQL, Docker, and browser E2E tests.

Tags