Your LLM traffic is unprotected.
Fix it in 5 minutes.

SafeLLM uses APISIX's serverless-pre-function plugin with Lua. The script reads the full request body from Nginx memory and POSTs it to SafeLLM's /auth endpoint. If the sidecar returns 200 — APISIX proxies to upstream. If 403 — APISIX blocks. No Base64 encoding, no header size limits. We can't use forward-auth because it only forwards headers, not body. The Lua integration is pre-configured in config/apisix.yaml.

Configurable via FAIL_OPEN. If true: all traffic passes through (graceful degradation). If false: all traffic blocked (fail-closed). The cache layer has a built-in circuit breaker — if Redis connection fails, it opens the circuit and bypasses cache without crashing the pipeline. Recovery is automatic when Redis comes back.

Three modes: Block (Enterprise) buffers the full response, scans, then delivers — increases TTFT but maximum security. Audit (OSS) streams to user instantly while scanning a copy in the background — zero latency impact. Streaming DLP (Enterprise beta) uses a sliding window to scan tokens as they stream via SSE and can redact PII mid-flight. Memory is capped by DLP_MAX_OUTPUT_LENGTH (default 500K chars).

Yes. The regex engine includes aggressive patterns for obfuscation detection — credit cards with spaces between digits (4 5 3 2 0 1 5 1...), SSNs with unusual separators. Credit card matches are validated with Luhn checksum and SSNs with SSA area/group/serial rules to minimize false positives. For IBAN, we restrict to 68 known country code prefixes. Enterprise adds GLiNER AI for context-aware detection (25+ entity types).

Prometheus metrics (/metrics): blocked requests by layer, scan latency histograms, cache hit rates, DLP detections. Enterprise adds tamper-resistant audit logs sent asynchronously via Redis queue to Loki/S3. Logs contain request ID, SHA256 prompt hash, verdict, layer, reason, and latency — but never the raw prompt (privacy by design). This covers EU AI Act documentation requirements, GDPR data minimization evidence, and SOC2 audit trails.

No GPU required. All ONNX models are optimized for CPU inference: L2 Neural Guard runs at 1,200+ RPS with P95 of 13.5ms on a standard AMD Ryzen 5 PRO. Models are shared via Copy-on-Write between workers (4 workers use ~500MB total). Enterprise optionally supports GPU for guard-class models (Llama, Gemma, Qwen) when you need maximum accuracy.

Set SHADOW_MODE=true (the default). Every layer runs its full detection pipeline but instead of returning 403, it logs the would-be block and returns 200. You deploy to production, observe what gets flagged via Prometheus/logs, tune thresholds and keyword lists, then set SHADOW_MODE=false when confident. One env var change to go from observe to enforce.

Yes. Set CUSTOM_FAST_PII_PATTERNS as a JSON dict: {"ACME_ID": "ACME-[0-9]{4}", "PROJ_CODE": "PRJ-[A-Z]{3}"}. Safety limits: max 50 patterns, max 256 chars per pattern, and patterns are skipped for texts longer than 20K chars (configurable) to prevent ReDoS. Enterprise adds GLiNER-based custom entity training for context-aware detection.

No. SafeLLM has zero telemetry mechanisms. No call-home, no usage analytics, no prompt fragments or statistics are ever sent to us or any third party. There is no outbound network connection to SafeLLM systems — the software runs entirely within your infrastructure. This is verifiable in the source code (OSS core is Apache 2.0). This zero-knowledge architecture is specifically designed for banking, government, and defense environments where any data exfiltration risk is unacceptable.

No. A DPA is not required because SafeLLM is not a data processor under GDPR. We deliver on-premise/VPC software — we never receive, store, process, or have access to your data. Your prompts, responses, logs, and encryption keys stay 100% within your infrastructure. When your legal team asks (and they will): "DPA is not applicable because SafeLLM is licensed software deployed on-premise. We are not a data processor — zero data is transmitted to our systems. Ever." This also means no sub-processor chain, no cross-border transfers, and no vendor audit obligations related to SafeLLM.