Prompt Firewall
Mangesh Bhattacharya
Prompt Firewall 🛡️
A lightweight, dependency-light library and service for detecting prompt injection attempts, the #1 risk on OWASP's LLM Top 10 two years running, before they reach an LLM, or before an LLM's output reaches a downstream tool/action.
Runs fully offline: no embedding model download, no external API call. Analysis takes single-digit milliseconds, so it's cheap enough to sit inline on every request rather than as an occasional audit.
Ships with an interactive web playground (screenshot below) and a one-command Docker setup, clone it, run docker compose up, and you have a live demo on localhost:8000 with zero manual configuration, on Linux, macOS, or Windows.
Contents
Why This Exists
Who This Is For
How It Works
Web UI
Install
Docker
Usage
Industry Examples
Evaluation
Testing
Limitations
Roadmap
Security
License
Why This Exists
83% of organizations are deploying AI in production, but most security teams don't yet have staff who understand AI-specific threats like prompt injection, model manipulation, and agent hijacking, it's the single largest cited cybersecurity skills gap of 2026 . The OWASP 2026 LLM Top 10 kept Prompt Injection at #1 for a second cycle, and, notably, kept it there even after incorporating real incident data (6,639 recorded incidents, weighted 25%) alongside expert consensus for the first time. OWASP's own read is a "defense effect": successful exploits are undercounted precisely because organizations are already investing in mitigations, not because the risk is smaller than the ranking suggests.
Prompt injection isn't a theoretical risk: it's how attackers get a chatbot to leak its system prompt, get an AI coding agent to exfiltrate secrets, or get an autonomous agent to call a "delete" (or "wire transfer," or "reconfigure router") tool it was never supposed to touch on its own.
This project is a practical, inspectable answer to that gap: a firewall you can actually read, test, and reason about, not a black box, and not a wrapper around someone else's undocumented model.
Who This Is For
Prompt injection isn't confined to consumer chatbots, anywhere an LLM reads text it didn't fully control and can take an action, the same risk shape applies:
Cybersecurity / AppSec teams adding an inspectable, offline control for LLM01 to an existing security stack, as a WAF-style gate in front of a model endpoint, a pre-commit/CI check (see cli.py --fail-on-block), or a SIEM-feeding sensor.
AI/ML platform engineers who need a fast first-layer filter in front of agentic tool-calling, before a model decides which function to invoke, not after.
Telecommunications, carriers are rolling out LLM copilots for network operations and customer support that ingest attacker-reachable text (trouble tickets, NMS alarms, self-service portal submissions) and can call tools against live infrastructure. See examples/telecom_network_ops_example.py .
Financial services, banking assistants and fraud-analysis copilots increasingly hold standing tool access to account data and, in agentic deployments, money movement itself. patterns.py's agent_hijack category is written specifically to catch this shape of request (tool_hijack, grant_permissions). See examples/fintech_transaction_assistant_example.py .
How It Works
Every request is scored by three independent layers, combined into a single 0-100 risk score:
Layer, What it catches, File
Pattern matching: 15 regex signatures across 6 attack categories (instruction override, role hijack, exfiltration, delimiter escape, obfuscation, agent hijack), patterns.py
Heuristics: Structural red flags a single regex can't catch: high-entropy encoded blobs, clustered imperative verbs, zero-width/control characters hiding text from human review, delimiter bursts faking a new context boundary, heuristics.py
TF-IDF similarity: Paraphrases of ~30 known jailbreak techniques, via cosine similarity against a curated corpus, catches attacks that don't match any single regex, similarity.py
The score maps to a verdict:
ALLOW (score < 25), nothing suspicious
FLAG (25-49), ambiguous; log it and let it through (or route to human review / step-up auth for high-stakes actions)
BLOCK (≥ 50), high-confidence injection attempt
A note on false positives
TF-IDF similarity is deliberately lightweight, no embedding model, no network call, which means it generalizes less precisely than a real semantic model. A benign sentence that happens to share vocabulary with a known attack (e.g. "please ignore my previous typo") can score into the FLAG tier. That's intentional: FLAG exists specifically so ambiguous cases get logged for review instead of being silently allowed or aggressively blocked. BLOCK requires a much stronger signal, an actual pattern match, not just vocabulary overlap. The evaluation notebook measures exactly how often this happens, instead of just asserting it.
Web UI
src/promptfirewall/api.py serves a self-contained, dependency-free web playground (webui/) alongside the API, open http://localhost:8000/ after starting the service (see Install or Docker) and you get:
A live scanner: paste text, hit Analyze (or Ctrl/Cmd + Enter), and see the verdict, 0-100 risk score, and every matched pattern / heuristic signal / similarity match that produced it.
One-click example prompts spanning all six attack categories plus benign controls (including a hard negative), for demos or quick sanity checks.
A session history panel, a live "how it works" explainer sourced from the running API (/api/info) so it never drifts out of sync with the detector, and copy-paste integration snippets (curl / Python / JS / CLI).
It talks to the same /analyze and /health endpoints documented under HTTP API, no separate backend or build step, and no external network calls (fonts, scripts, and styles are all bundled).
The UI is accessible by default: a skip link to the playground, a full ARIA tabs pattern (with arrow-key navigation) on the integration snippets, keyboard-operable session history, aria-live verdict announcements, visible focus states, and prefers-reduced-motion support. Colors are checked against WCAG 2.1 AA contrast.
Error handling
Errors surface in place, in plain language, no silent failures, no raw stack traces.
Oversized input is rejected with a 413 before it reaches the detector, and the UI reports exactly why:
If the backend isn't reachable, the status indicator in the header flips to API unreachable instead of leaving you guessing why Analyze isn't doing anything:
Every other input error (empty text, malformed JSON) returns a 422 with a detail message, see HTTP API.
Install
Prefer not to manage a Python environment at all? See Docker below, one command gets you the API, the web UI, and every dependency, with nothing to install on the host beyond Docker itself.
Docker
The whole service, API, detector, and web UI, runs from a single image with no host Python setup required. Works the same way on Linux, macOS, and Windows (Docker Desktop or WSL2).
Then open http://localhost:8000/ for the web UI, or call the API directly:
Prefer plain docker over Compose:
The image runs as a non-root user, exposes a container HEALTHCHECK against /health, and, matching the rest of this project, makes no external network calls at runtime: everything it needs (patterns, corpus, web UI assets) is baked into the image at build time.
Usage
As a library
Wrapping an LLM call
See examples/middleware_example.py for the integration pattern, check user input (and, for RAG/agentic apps, retrieved documents and tool output too, since indirect injection is just as real as direct input) before it reaches the model or an action executes.
CLI
For batch scanning a whole directory (log dumps, ticket exports, SOC triage) instead of one string at a time, see scripts/scan_prompt_logs.sh :
HTTP API
This also serves the web UI at http://localhost:8000/plus GET /health and GET /api/info / GET /api/examples (metadata the UI itself consumes, thresholds, layer descriptions, and the example gallery).
Industry Examples
Beyond the generic middleware_example.py , two runnable examples show the same integration pattern applied to sectors where an LLM agent has real-world blast radius:
Example, Scenario, Attack category exercised
examples/telecom_network_ops_example.py: An LLM NOC copilot triaging trouble tickets that can call tools against live infrastructure (router config, ACLs). Demonstrates indirect injection via a poisoned ticket description, not a live chat message., delimiter_escape, instruction_override
examples/fintech_transaction_assistant_example.py: A banking assistant with standing access to a wire_transfer tool. Treats FLAG (not just BLOCK) as a reason to require step-up verification, since the cost of a false negative here is a wire transfer, not a bad chat reply., agent_hijack (tool_hijack, grant_permissions), instruction_override
Run either directly:
Evaluation
notebooks/detection_evaluation.ipynb scores the shipped detector against a held-out set, 20 injection paraphrases (same techniques as corpus.py, different wording, so the TF-IDF layer is tested on generalization, not lookup) and 30 benign prompts including deliberate hard negatives, and reports precision, recall, F1, ROC-AUC, and average precision, plus which signal fired for every false positive and false negative.
This is a small, hand-curated set, not a public benchmark, so treat the numbers as a directional, reproducible check rather than a production accuracy claim:
Operating point, Precision, Recall, Notes
BLOCK only: 1.00, 0.05, High-confidence, low-volume, only paraphrases that also trip a strong pattern match
FLAG or BLOCK: 0.72, 0.65, The "log for review" tier catches most held-out paraphrases, at the cost of some false positives
Every false positive in the run came from the similarity layer alone (no pattern/heuristic fired), the concrete, measured version of the false-positive note above. Every false negative was a paraphrase with low enough vocabulary overlap to stay under both the similarity threshold and every regex, the expected failure mode of a bag-of-words layer, and the motivation for the embedding-backend roadmap item below.
Reproduce it:
Testing
45 tests covering benign-text false-positive avoidance, every pattern category, every heuristic, the similarity detector, verdict threshold configuration, and result serialization. CI runs the suite, plus ruff check ., on Python 3.10-3.12 on every push.
Limitations (read before relying on this in production)
Not a silver bullet. No prompt-injection defense is complete, this raises the bar and gives you visibility, it doesn't guarantee zero bypasses. Defense in depth (least-privilege tool access, output validation, human approval for destructive actions) still matters more than any single filter.
English-first. Patterns and the similarity corpus are English-language; non-English injection attempts will mostly rely on the heuristic layer alone.
TF-IDF, not embeddings. As noted above, similarity matching is bag-of-words-level, not true semantic understanding, see Evaluation for measured recall on paraphrased attacks.
Static corpus. New jailbreak techniques emerge constantly; patterns.py and corpus.py need to be maintained as the threat landscape evolves, PRs welcome.
Roadmap
Optional sentence-embedding backend for stronger similarity matching (the evaluation notebook's false negatives are the concrete case for this)
Per-tenant/per-app custom pattern packs
Structured logging sink (JSON lines) for SIEM ingestion
Benchmark against a public prompt-injection dataset, not just the hand-curated held-out set in notebooks/
Security
Found a way to bypass detection, or another security issue? Please don't open a public issue, see SECURITY.md for the private reporting process.
License
MIT
About
Lightweight, offline prompt-injection detector for LLM apps - pattern matching + heuristics + TF-IDF similarity, 45 tests, FastAPI + CLI
Topics
Resources
Readme
MIT license
Security policy
Security policy
Stars
0 stars
Watchers
0 watching
Forks
No releases published
Contributors 1 (1)
Languages
Like this project
Posted Oct 28, 2026
Offline prompt-injection detector for LLM apps: pattern matching, heuristics and TF-IDF similarity. Ships as a library, FastAPI service and CLI, with 45 tests.
Likes
0
Views
0
Timeline
Jul 29, 2026 - Oct 17, 2026
