APISIX Integration

Integration with Apache APISIX

SafeLLM is designed to work seamlessly as a sidecar for Apache APISIX, providing a transparent security layer for all your LLM traffic.

Deep Content Inspection (Lua + resty.http)

Why Not forward-auth?

Standard forward-auth plugins do not send the request body to the auth service—only headers and URL. Since SafeLLM needs to scan the actual prompt content, we use a Lua-based integration that directly reads and forwards the body.

The integration uses APISIX’s serverless-pre-function plugin with the resty.http Lua library. This allows APISIX to:

1. Read the full request body from Nginx memory
2. POST it to SafeLLM’s /auth endpoint as a standard HTTP request
3. Block or allow based on the response

Benefits of This Approach

Feature	forward-auth	Lua + resty.http
Access to request body	❌ No	✅ Yes
Header size limits	⚠️ Problematic for large prompts	✅ No limits (body is POST)
Fail-open control	Limited	✅ Granular
Body stored in memory	N/A	✅ Direct Nginx access

How It Works

┌──────────┐    ┌─────────────────────────────────────┐    ┌──────────┐
│  Client  │───▶│ APISIX (Lua serverless-pre-function)│───▶│   LLM    │
└──────────┘    └─────────────────────────────────────┘    └──────────┘
                              │
                              │ POST /auth (body)
                              ▼
                       ┌────────────┐
                       │  SafeLLM   │
                       │  Sidecar   │
                       └────────────┘
                              │
                    ┌─────────┴─────────┐
                    ▼                   ▼
               200 OK              403 Forbidden
            (Continue)              (Block)

Flow Details

1. Request Arrival: A client sends an OpenAI-compatible request to APISIX.
2. Security Check (Pre-function): Lua script reads the body and POSTs it to SafeLLM’s /auth endpoint.
3. Validation: SafeLLM processes the request through its Waterfall Pipeline (Cache → Keywords → PII).
4. Decision:
   • If safe: Sidecar returns 200 OK, APISIX forwards to upstream.
   • If unsafe: Sidecar returns 403 Forbidden, APISIX blocks the request.

Route Configuration Example

routes:
  - uri: /api/*
    plugins:
      serverless-pre-function:
        phase: rewrite
        functions:
          - |
            return function(conf, ctx)
                local core = require("apisix.core")
                local http = require("resty.http")

                -- Configuration
                local FAIL_OPEN = false
                local TIMEOUT_MS = 5000

                -- Read body
                ngx.req.read_body()
                local body = ngx.req.get_body_data()

                if not body or #body == 0 then
                    return  -- No body to scan
                end

                -- Send to SafeLLM
                local httpc = http.new()
                httpc:set_timeout(TIMEOUT_MS)

                local res, err = httpc:request_uri("http://sidecar:8000/auth", {
                    method = "POST",
                    body = body,
                    headers = {
                        ["Content-Type"] = "application/json"
                    }
                })

                -- Handle failures
                if not res then
                    if FAIL_OPEN then
                        return  -- Allow traffic
                    else
                        core.response.exit(503, "Security service unavailable")
                    end
                end

                -- Check result
                if res.status == 403 then
                    core.response.exit(403, "Blocked by security")
                end
            end

Source of Truth

The canonical APISIX configuration is in config/apisix.yaml. Documentation examples are simplified—refer to the actual config for production deployments.

Performance Considerations

By using SafeLLM as a sidecar:

• Low Latency: Communication happens over localhost (~0.1ms network overhead).
• Scalability: SafeLLM instances scale horizontally alongside APISIX.
• Security: Prompts are checked before leaving your infrastructure.

Alternative: Proxy Mode

If Lua integration is too complex for your setup, you can run SafeLLM as a reverse proxy in front of APISIX:

Client → SafeLLM (proxy) → APISIX → LLM

This gives SafeLLM direct access to the body without Lua, but you lose some APISIX benefits (like early rate limiting).

Reference Deployment

For a ready-to-run APISIX evaluation stack:

• Docs: /deployments/apisix-reference/
• Repo folder: safellm-oss/examples/apisix-reference/

Long-form companion guides:

• /articles/how-to-run-apisix-reference-with-safellm/
• /articles/why-apisix-for-ai-gateway/