Maturity
Field Value Status Arena entry Maturity Low Field-ready No — not clinical, not field-certified Last reviewed 2026-08-07
Gray Swan Safeguards Arena blue-team classifier prototype. Low maturity is not a flaw; unverified absolute claims are.
Black-X-Templar
FratresMedAI — layered safeguards for AI-agent deployments: prompt/tool-abuse signals, offline-capable scoring, and audit-oriented telemetry.
Contest reviewers typically care most about the
safeguards_adapter + mini_templar classification path (stateless API surface, deterministic tests).Optional ops checklist:
RUNPOD_DEPLOYMENT_CHECKLIST.md (legacy hosting notes). Local smoke helper: bash scripts/pre_flight_check.sh.Labeled dataset snapshot (illustrative held-out judge run)
Quick evaluation summary (real
tests/real_2026_training_dataset.jsonl, not the synthetic 100-prompt harness):Interpretation: 0% FNR on this adversarial slice (no missed attacks); 50% FPR reflects aggressive fail-closed blocking — call out explicitly for reviewers when submitting.
Loading
Unable to render rich display
Architecture
File Role
app.py Streamlit dashboard (demo mode only, not part of hardened headless core package) enforcer.py Passive MCP/tool-call signature monitor + rate-based detector osint_expert.py NVD CVE feed ingestion + cross-reference with threat log rebuff_engine.py Prompt-injection shield — detect & log hostile instructions ghost_monitor.py Encrypted-traffic fingerprinting via timing/length analysis (passive) vault_guardian.py Alert-driven mock credential-rotation workflow p2p_mesh.py Signed threat-intel sync across trusted peers whisper_detector.py Shannon entropy analysis for steganography detection mimicry_hunter.py TF-IDF cosine-distance behavioral drift analyzer neural_mirror.py Offline red-team simulation harness config.py Central configuration — all tunable thresholds and pathsAll detection events are written to
audit_log.db (SQLite) with SHA-256 payload hashes and HMAC-SHA256 signatures for chain-of-custody.Requirements
Python 3.11+
No root / admin privileges required for most modules
ghost_monitor.py live mode uses psutil only (no raw sockets)ghost_monitor.py --pcap mode optionally uses scapy (read-only)Installation
Optional labeled-data harness
When you have a judge backend configured locally,
test_harness.py can score JSONL datasets (flags vary by environment — run python test_harness.py --help). Repository fixtures include tests/real_2026_training_dataset.jsonl.Configuration
Environment variables use the
DARKSPACE_* prefix for continuity with older deployments and tooling (values shown below are unchanged).Option A — Environment variables (recommended)
Option B — .env file
.env fileCreate a
.env file in the project root (never commit it):Then load it before running:
Option C — Streamlit secrets (for app.py only)
app.py only)ThreatFox API key: Register free at https://threatfox.abuse.ch/api/
Running each module
Dashboard (demo mode only)
Opens at
http://localhost:8501For submission/hardened deployment, treat the dashboard as an optional demo artifact and run the core modules headlessly (CLI + k8s deployment path).
Enforcer — passive monitor (pipe log output into it)
OSINT Expert — CVE feed sync
Rebuff Engine — prompt injection shield
Ghost Monitor — traffic fingerprinting
Vault Guardian — mock key rotation
P2P Mesh — threat-intel sync
Whisper Detector — steganography check
Mimicry Hunter — behavioral drift
Neural Mirror — red-team simulation
Key thresholds (all tunable in config.py)
config.py)Setting Default Meaning
ENFORCER_RATE_THRESHOLD 10 Events per window before rate alert ENFORCER_RATE_WINDOW_SECONDS 60 Sliding window for rate detection VAULT_CRITICAL_SCORE_THRESHOLD 8.0 Risk score that triggers mock rotation WHISPER_ENTROPY_THRESHOLD 3.5 bits/char above which text is flagged MIMICRY_DRIFT_THRESHOLD 0.8 Cosine distance above which drift is flagged NEURAL_MIRROR_SAMPLE_LIMIT 42 Max synthetic samples per red-team runSafety guarantees
No TCP resets — zero raw socket writes
No active blocking — observation and logging only
No payload decryption — traffic analysis uses timing/size only
No offensive probing — all network calls are read-only GET/POST to approved public APIs
No credential mutation — vault rotation is mock-only
Audit chain — every detection is hashed (SHA-256) and HMAC-signed before storage
Database tables
Table Written by Contents
audit_log app.py ThreatFox search results, IOC records threat_log enforcer.py Signature & rate detections with HMAC sigs vuln_log osint_expert.py NVD CVEs vuln_correlation osint_expert.py CVE ↔ threat_log cross-references prompt_log rebuff_engine.py Prompt attempt hashes + verdicts ghost_log ghost_monitor.py Traffic timing/size metrics vault_events vault_guardian.py Mock rotation records mesh_inbox p2p_mesh.py Inbound peer intel summaries whisper_log whisper_detector.py Entropy scores mimicry_log mimicry_hunter.py Drift scores mirror_results neural_mirror.py Red-team run results4. Gaps & Risk Areas (Honest assessment - this is why we still do PoCs)
No active response layer yet (v1.0 is deliberately passive).
Acceptable for initial evaluation, but v2 must include kinetic hooks (auto-quarantine of agent session, forced re-auth, etc.).
Status:
kinetic_hooks.py and the Kinetic Hooks dashboard tab provide this v2 response path; validate in deployment with dry-run first.P2P Mesh trust model
Current design assumes pre-vetted peers.
In a real deployment we would want PKI + mutual TLS + zero-trust attestation (e.g., via SPIFFE).
False-positive tuning
Entropy thresholds and drift thresholds are good starting points but will need empirical calibration against real
.mil agent traffic.Streamlit dashboard
Convenient for demo, but not suitable for classified environments.
We would containerize the backend and replace the frontend with a hardened DoD-approved web framework or a thick-client.
ThreatFox dependency
Fine for unclass, but operational nodes should have an offline IOC cache or integration with classified feeds (e.g., ACES, CYBERCOM feeds).
Status: local SQLite IOC caching exists; classified-feed integration remains a deployment-phase integration task.
Neural Mirror realism limits
Current 100% scores (including
220/220) are on synthetic/red-team-generated corpus data.Real-world adversarial traffic is expected to expose additional gaps that require continuous corpus and rule updates.
Kinetic hooks are partially integrated
kinetic_hooks.py supports real wiring via optional Keycloak integration, but most deployment actions remain environment-specific integration work.Production rollout still requires mission IdP/SIEM-specific implementation and runbook validation.
Demo UI still exists in repo
Streamlit dashboard is explicitly demo-only and split from headless core dependencies.
Its presence is for demonstrations and operator exploration, not classified production deployment.
Static-analysis residual noise
Bandit medium/high blockers are enforced to zero.
Remaining Bandit findings are expected low-severity patterns (for example, assert/test/tooling noise) and are tracked as residual risk.
DevSecOps & Compliance Baseline
CI security workflow:
.github/workflows/security-ci.ymlRegression/compliance smoke suite (
python _smoke_v2.py)SAST (
bandit, semgrep)SCA (
pip-audit)SBOM generation (CycloneDX on CI runs, SPDX via Syft on releases)
Release provenance signing via Sigstore Cosign (keyless/OIDC)
Security disclosure and supply-chain posture:
SECURITY.mdNIST AI RMF + maturity mapping:
MATURITY.mdConfiguration versioning and threshold history:
CHANGELOG_CONFIG.mdRun security checks locally
Test suite (pytest)
Core security regression tests:
tests/test_core_security.pyRun directly:
SBOM artifacts
CycloneDX SBOM:
sbom.cdx.jsonSPDX SBOM:
sbom.spdx.jsonGenerate both formats locally:
Remaining Bandit noise is low-severity/expected test/tooling patterns, not medium/high risk blockers.
Hardened container deployment
OCI image definition:
DockerfileNon-root runtime user
Minimal Python base image
Kubernetes baseline manifests:
k8s/darkspace-deployment.yamlrunAsNonRoot, dropped Linux capabilitiesreadOnlyRootFilesystemseccompProfile: RuntimeDefaultBaseline security configuration
Set these for production deployments:
config.enforce_security_baseline() now blocks runtime startup when minimum baselines are not met.Air-gapped operation controls
Global offline switch:
DARKSPACE_OFFLINE_ONLY=trueosint_expert.py supports --offline-onlyapp.py ThreatFox fetch/search now respects offline-only mode and uses local IOC cacheDefault posture is offline-first (
DARKSPACE_OFFLINE_ONLY=true when unset)Pre-seed IOC cache before disconnected operation:
ioc_seed.json should be a JSON array of objects with fields such as: ioc, ioc_type, threat_type, malware_printable, confidence_level, reporter, raw_json.IdP integration example
Concrete Keycloak admin integration example:
contrib/keycloak_idp_example.pyIntended for deployment glue with
kinetic_hooks.py quarantine/re-auth action pointsSOC export roadmap
Current: SQLite-backed
Security Metrics tab in StreamlitAdded starter exporter:
export_security_metrics_prometheus.pyAdded Grafana starter dashboard:
grafana/darkspace_security_metrics_dashboard.jsonRun Prometheus exporter:
Metrics are exposed at
http://localhost:9109/metrics for Prometheus scrape integration.DoD pre-escalation checklist status
Must-have (non-negotiable)
Runtime security baseline enforcement: complete (
config.enforce_security_baseline())Kinetic safety controls (dry-run + toggles + audit): complete
Neural Mirror validation gate (
>=97% target): completeCalibration with audit trail (
calibrate.py): completeSBOM generation at release (CycloneDX + SPDX): complete
Structured metrics and observability: complete
P2P zero-trust controls (mTLS + attestation + fingerprint allowlist): complete
Hardcoded-secret checks in smoke suite: complete
Strongly recommended (before escalation)
Explicit CMMC/DFARS mapping in
MATURITY.md: completeConcrete IdP integration example in
contrib/: complete (contrib/keycloak_idp_example.py)NSA AISC alignment notes in
SECURITY.md: completeSigstore provenance in CI (
.github/workflows/security-ci.yml): completeApplication Security STIG full verification with official DISA tooling: pending external validation step (requires STIG Viewer/SCAP content in target environment)
1 like · 11 views · Aug 11, 2026
Developed a prototype AI-agent safeguard system with multiple security modules.