Why SafeLLM?

SafeLLM - Enterprise AI Security Gateway

SafeLLM is a high-performance, multi-layered security gateway (L0-L2) for Large Language Models (LLM). It acts as a sidecar for Apache APISIX, creating a protective barrier against Prompt Injection attacks, PII (Personally Identifiable Information) data leaks, and optimizing costs through intelligent caching.

Three Main Pillars of Protection

Deploying AI in an enterprise involves three main risks that SafeLLM addresses directly:

1. Security: Jailbreak and prompt injection attacks can force a model to break security policies or leak secret system instructions.
2. Privacy: End users may unknowingly send sensitive data (PII) to public models, violating regulations such as GDPR.
3. Cost and Performance (ROI): Repetitive queries generate unnecessary token costs and strain infrastructure.

SafeLLM solves these problems at the infrastructure level before the query reaches the model.

Use Cases

1. APISIX + LLM Gateway: Deploy SafeLLM as a sidecar for your Apache APISIX instance. This provides a unified entry point for all LLM traffic with centralized security enforcement, rate limiting, and caching without modifying your application code.
2. Air-Gapped / Private Cloud Deployment: SafeLLM is designed to run locally or in private clouds. Use it to secure LLM queries in regulated industries (Finance, Healthcare) where data cannot leave the internal network.
3. PII Compliance (GDPR/HIPAA): Automatically detect and redact sensitive information from both user prompts and model responses before they are transmitted, ensuring compliance with data protection laws.

What Problems We Solve

Metric	Impact
Latency	<1ms overhead for L1-L1.5 scans. <0.1ms for L0 Cache hits.
Cost Reduction	Up to 30-50% savings on token usage through intelligent L0 semantic caching.
Security	99%+ detection rate of known jailbreak and prompt injection patterns (L2).
False Positives	Optimized for low false-positive rates to ensure smooth developer experience.

Main Features

• [OSS] L0 Smart Cache: Cost and latency reduction (<0.1ms) for repetitive queries.
• [OSS] L1 Keyword Guard: Instant blocking of forbidden phrases and patterns.
• [Enterprise (Paid)] L1.5 AI PII (GLiNER): Recognition of over 25 types of sensitive data, including country-specific ones.
• [Enterprise (Paid)] L2 Neural Guard: Protection against advanced injections using neural networks (ONNX).
• [Enterprise (Paid)] DLP Output Scan: Scanning model responses for data leaks (block/anonymize).
• [Enterprise (Paid)] Dashboard: Administrative panel for managing rules and monitoring threats.

Operation Modes

1. Direct Guard API: The application directly queries SafeLLM before sending a prompt to the model.
2. APISIX Sidecar: Transparent protection at the network gateway level (Forward Auth).
3. DLP Audit Mode: Scanning responses in the background (zero latency), logging violations for compliance.