This project focused on building and testing a practical AI security assessment lab for evaluating LLM defenses against prompt injection and jailbreak attacks.

I integrated Spikee by Reversec with a locally hosted cybersecurity model running through LM Studio, then added NVIDIA NeMo Guardrails to compare model behavior under three conditions: no guardrails, input filtering, and combined input/output protection.

The work included configuring the local model environment, building a custom FastAPI gateway, integrating NeMo Guardrails, troubleshooting model latency and timeout issues, creating a reusable Spikee target, and analyzing attack results using Spikee’s built-in reporting tools.

The project also explored different adversarial testing approaches, including prompt injection datasets, obfuscation, encoded attacks, Best-of-N testing, synthetic canary leakage tests, and structured benchmark comparisons.

The objective was to measure how much the guardrails reduced successful attacks while keeping the model, dataset, and testing conditions consistent.

Tools used: Spikee, NVIDIA NeMo Guardrails, LM Studio, Python, FastAPI, PowerShell, Parrot OS, local LLMs, JSONL datasets, and custom security testing scripts.

This project demonstrates a hands-on approach to LLM red teaming, AI safety testing, prompt-injection assessment, and guardrail validation for organizations deploying generative AI systems.

pretty cool tbh. testing the same attacks with different guardrail setups makes the results way more interesting.

The question I ask before automating anything: "What does this look like on a Tuesday in month four?"

Not the demo. Not launch day. Month four, when the person who championed it has moved on, the data has drifted, and the model has quietly started doing something slightly different.

The automations that survive month four have three things: a human somewhere in the loop, a log a non-engineer can read, and a kill switch that doesn't need me. The PO intake agent I posted last week is built that way on purpose. It registers and notifies, people decide, and every email in and out is logged.

If yours has all three, you're fine. If it has none, I'd love to hear how it's going. I collect these stories.

the month four test is so spot on zahid. everyone designs for the launch demo, but silent prompt drift kills more workflows than actual broken APIS.