Sunesh Agarwal
AI Engineer shipping LLM & GenAI solutions that scale
- 1
- Followers
AI agents read web pages, tickets and files, and any of them can hide instructions. That's prompt injection, the #1 risk in the OWASP Top 10 for LLM Applications.
I built a policy layer that sits between an agent and its tools and stops those instructions from turning into actions. Five deterministic rules, no prompt tricks: tool allowlists, read scope, write confinement, egress pinning and sink provenance.
Then I attacked it. On an agent that obeyed 43% of injection attacks, the rules cut successful attacks to 0 out of 60, across all five attack goals: stealing a secret, phoning home, poisoning files, planting a payment IBAN and moving customer data. They held on Claude Haiku 5.5 too, with zero false blocks, and on the AgentDojo benchmark they cut worst-case attack success from 98% to 3.1%.
Every number comes from logged runs with 95% confidence intervals.
Want the same for your agent? I map its tools, attack it with your own workflows, write the policy and hand back a before/after attack rate.
#AIAgents #AgentSecurity #AISecurity #PromptInjection #LLMSecurity #GenAI #LLM #AIEngineer #MCP #ModelContextProtocol #Claude #Anthropic #RedTeaming #CyberSecurity #Python #AIAutomation
0
64
Teams running LLM features in production watch the bill climb and can't say why. Which feature is driving it? Which model? How much goes to retries? Most optimisation advice is a percentage nobody actually measured.
llm-audit is a command-line toolkit I built to answer those questions with evidence.
HOW IT WORKS
→ Ingest: OpenTelemetry GenAI traces, Langfuse data, or usage exports from OpenAI and Anthropic, loaded into one DuckDB table.
→ Analyze: 16 deterministic checks, 8 for cost (unused prompt cache, model overkill, duplicate calls, context bloat, retry waste, off-peak eligible work and more) and 8 for reliability (error rates, latency, rate limiting, single-provider risk and more).
→ Backtest: replays the workload with each fix switched on alone, then compares predicted and measured savings.
→ Report: one HTML file with ranked findings. Each finding has its evidence query, a fix and a confidence level.
WHAT MAKES IT DIFFERENT
• Content-free. It works from prompt hashes and token counts only, so prompt text never has to leave the client.
• No unmeasured claims. A saving is either measured by the backtest or labelled as an estimate. A fix that lowers quality is marked rejected, even when it saves money.
• Honest gaps. Checks that can't run on the data are listed with the reason, not quietly dropped.
THE PART I'M PROUDEST OF
My first paid backtest failed its own checks: 75% recall and 103% savings error. Instead of tuning the numbers, I debugged the harness. The test runs were sharing the provider's prompt cache, so they weren't independent. A fresh marker per run fixed it, and the re-run confirmed the isolation (0.0 cache hits where there had been up to 1.0).
FIELD-TESTED ON MY OWN APP
I audited AI Lab, my public Q&A app over the EU AI Act and GDPR. Finding zero: the app kept no per-call record at all. My merged PR added a content-free, per-attempt trace. The audit then showed that the evaluation judges took about two thirds of the spend, so scoring answers cost twice as much as producing them.
BUILT LIKE THE AUDIT IT SELLS
915 tests passing · dependencies pinned with hashes · $0.97 total LLM spend for the whole build, under a $15 cap enforced in code.
The demo workload is synthetic and the field run covers one app, so no percentage here applies to your bill. An audit of your own data is the only way to find out. A good starting point is a Bill Snapshot: send your usage exports and get ranked findings back in 48 hours.
#LLMOps #FinOps #AIEngineering #GenAI #LLM #AICostOptimization #Observability #OpenTelemetry #Python #DuckDB #AIAudit #PrivacyByDesign
0
108
A compliance RAG copilot for the EU AI Act and GDPR
Most AI tools answer every question. In compliance, a confident wrong answer is worse than none, because someone might act on it.
So I built a system designed to say "I don't know" when the law doesn't support an answer.
Ask a question in plain English and you get one of two things:
✅ An answer where every claim links to the exact Article or Recital it came from
🟠 An honest abstention that explains why the sources can't support an answer
How it works
• A six-step LangGraph agent: plan → retrieve → rerank → generate → critique → human check
• The critic can send a weak draft back for better evidence, with hard caps on retries and token spend
• Citations are checked against the sources the model actually saw. Invented ones are removed, not trusted
• 1,437 passages from the AI Act and GDPR, with embeddings computed locally on CPU
Measured, not claimed (37-question golden set)
• 36/37 correct abstention decisions, 0 false answers
• 0.99 faithfulness to sources
• 0.97 recall@5
Runs on $0/month: free-tier hosting end to end (Cloud Run, Vercel, Neon, Upstash), with provider fallback on every model call.
Stack: Python · FastAPI · LangGraph · Postgres + pgvector · Next.js · AI SDK
🔗 Try it live: ai-lab-web-ten.vercel.app (http://ai-lab-web-ten.vercel.app)
0
145
Every missed call is a lost customer. I build AI voice agents that answer, qualify and book, 24/7. 📞
Ava is a voice receptionist that handles inbound calls like your best front-desk person:
✅ Answers instantly, day or night, with no hold music and no voicemail
✅ Qualifies the caller with your rules (need, budget, timing, location, fit), one question at a time and never repeating one
✅ Books the appointment in your real calendar (offers open slots, reads the time back, confirms)
✅ Updates your CRM with no duplicate contacts
✅ Follows up automatically: hot-lead alert to your team, confirmation email to the caller, every call logged
Works for any business that runs on calls and appointments:
🏠 Home services · 🦷 Clinics & dental · 🏡 Real estate · ⚖️ Law firms · 💇 Salons & spas · 🚗 Auto repair · 💼 Agencies & B2B demos
You change the questions, rules and integrations; the engine stays the same.
Built-in guardrails: never quotes prices it shouldn't, routes emergencies correctly, resists prompt injection, and hands off to a human callback when needed.
Tested like software, not vibes
• 18 real-world call scenarios, 18/18 passing
• My own checks caught bugs the vendor grader missed
• ~1.8 s voice response · ~$0.19 per call
Stack: Retell AI · GPT-4.1 mini · Cal.com (http://Cal.com) · HubSpot · n8n
Demo built for a fictional home-services company, Northwind Home Services.
👉 Want an AI receptionist, appointment setter or lead-qualification agent for your business? Message me.
#VoiceAI #AIAgents #AIReceptionist #AIVoiceAgent #ConversationalAI #AppointmentSetting #LeadQualification #LeadGeneration #CustomerSupport #Automation #AIAutomation #WorkflowAutomation #n8n #RetellAI #HubSpot #CRM #SmallBusiness #AIEngineer
0
77