Kubernetes Deployment

SafeLLM provides a Helm chart for easy Kubernetes deployment.

Prerequisites

• Kubernetes 1.24+
• Helm 3.10+
• kubectl configured for your cluster

Quick Start

1. Add Helm Repository

helm repo add safellm https://safellm.github.io/charts
helm repo update

2. Install SafeLLM OSS

# Basic installation
helm install safellm safellm/safellm-oss

# With custom namespace
helm install safellm safellm/safellm-oss -n safellm --create-namespace

# With custom values
helm install safellm safellm/safellm-oss -f my-values.yaml

3. Verify Installation

kubectl get pods -l app.kubernetes.io/name=safellm-oss
kubectl logs -l app.kubernetes.io/name=safellm-oss

Configuration

Basic Values

values.yaml

replicaCount: 2

image:
  repository: ghcr.io/safellmio/safellm-apisix-gateway-sidecar
  tag: '2.0.0' # Pin to specific version!

safellm:
  edition: oss
  enableMetrics: true
  enableCache: true
  enableKeywords: true
  enablePII: true
  useFastPII: true
  failOpen: false
  shadowMode: false

resources:
  limits:
    cpu: 500m
    memory: 512Mi
  requests:
    cpu: 100m
    memory: 256Mi

Docker Hub alternative:

image:
  repository: docker.io/safellm/safellm-apisix-gateway-sidecar
  tag: '2.0.0'

Redis Configuration

Bundled Redis (default)

redis:
  enabled: true
  architecture: standalone
  auth:
    enabled: false
  master:
    persistence:
      size: 1Gi

External Redis

redis:
  enabled: false
  host: 'redis.example.com'
  port: 6379

High Availability

replicaCount: 3

affinity:
  podAntiAffinity:
    preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 100
        podAffinityTerm:
          labelSelector:
            matchLabels:
              app.kubernetes.io/name: safellm-oss
          topologyKey: kubernetes.io/hostname

Integration with APISIX

SafeLLM is designed to run alongside APISIX in Kubernetes.

Architecture

┌─────────────────────────────────────────────────────────┐
│                    Kubernetes Cluster                    │
├─────────────────────────────────────────────────────────┤
│  ┌─────────────┐    ┌─────────────┐    ┌─────────────┐  │
│  │   Ingress   │───▶│   APISIX    │───▶│     LLM     │  │
│  │  Controller │    │   Gateway   │    │   Service   │  │
│  └─────────────┘    └──────┬──────┘    └─────────────┘  │
│                            │                             │
│                     POST /auth                           │
│                            ▼                             │
│                    ┌─────────────┐                       │
│                    │   SafeLLM   │                       │
│                    │   Sidecar   │                       │
│                    └──────┬──────┘                       │
│                            │                             │
│                            ▼                             │
│                    ┌─────────────┐                       │
│                    │    Redis    │                       │
│                    │    Cache    │                       │
│                    └─────────────┘                       │
└─────────────────────────────────────────────────────────┘

APISIX ConfigMap

Configure APISIX to call SafeLLM:

apiVersion: v1
kind: ConfigMap
metadata:
  name: apisix-config
data:
  apisix.yaml: |
    routes:
      - uri: /v1/chat/completions
        plugins:
          serverless-pre-function:
            phase: rewrite
            functions:
              - |
                return function(conf, ctx)
                    local http = require("resty.http")
                    ngx.req.read_body()
                    local body = ngx.req.get_body_data()

                    local httpc = http.new()
                    local res = httpc:request_uri("http://safellm-safellm-oss:8000/auth", {
                        method = "POST",
                        body = body,
                        headers = { ["Content-Type"] = "application/json" }
                    })

                    if res.status == 403 then
                        ngx.exit(403)
                    end
                end

Monitoring

Prometheus Metrics

SafeLLM exposes metrics on /metrics:

podAnnotations:
  prometheus.io/scrape: 'true'
  prometheus.io/port: '8000'
  prometheus.io/path: '/metrics'

ServiceMonitor (if using Prometheus Operator)

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: safellm
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: safellm-oss
  endpoints:
    - port: http
      path: /metrics

Upgrading

# Check available versions
helm search repo safellm --versions

# Upgrade to specific version
helm upgrade safellm safellm/safellm-oss --version 2.0.0

# Upgrade with new values
helm upgrade safellm safellm/safellm-oss -f new-values.yaml

Uninstalling

helm uninstall safellm

Troubleshooting

Pod not starting

kubectl describe pod -l app.kubernetes.io/name=safellm-oss
kubectl logs -l app.kubernetes.io/name=safellm-oss --previous

Health check failing

kubectl exec -it deploy/safellm-safellm-oss -- curl localhost:8000/health

Redis connection issues

kubectl exec -it deploy/safellm-safellm-oss -- python -c "import redis; r=redis.Redis(host='safellm-redis-master'); print(r.ping())"