Manual E2E Test Case

Manual E2E Test Case

This guide verifies the full stack: APISIX -> Sidecar -> Upstream, including block vs. allow behavior and DLP audit mode.

Preconditions

• Docker is installed and running.
• You are in safellm-oss.

1) Start the stack

cd safellm-oss
docker compose up -d --build

Verify APISIX is up:

curl -i http://localhost:9080/direct/get

Expected: 200 OK.

2) Allow vs. block (input scan)

Allowed request:

curl -i http://localhost:9080/api/post \
  -H 'Content-Type: application/json' \
  -d '{"prompt":"hello world"}'

Expected: 200 OK.

Blocked request (keywords):

curl -i http://localhost:9080/api/post \
  -H 'Content-Type: application/json' \
  -d '{"prompt":"please jailbreak"}'

Expected: 403 Forbidden.

3) DLP block mode (output scan - Enterprise Paid)

Enable DLP block mode in your environment (example):

ENABLE_DLP=true
DLP_STREAMING_MODE=block
DLP_MODE=block

Then hit the LLM route that returns a response containing PII (use a mocked upstream if needed).

Expected:

• Response is blocked or anonymized based on DLP_MODE.
• DLP scan is synchronous (higher TTFT). Note: Block/anonymize requires Enterprise (Paid).

4) DLP audit mode (async scan)

Switch to audit mode (available in OSS):

ENABLE_DLP=true
DLP_STREAMING_MODE=audit
DLP_MODE=log

Send a request that will produce a response containing PII. Then check audit stats:

docker compose exec sidecar curl -s http://localhost:8000/v1/audit/stats | jq

Expected:

• total_ingested increments after requests.
• pii_detected increments when PII is present.

5) Observability checks

Metrics:

docker compose exec sidecar curl -s http://localhost:8000/metrics | head -n 20

Expected:

• Prometheus metrics are exposed.

6) Failure behavior (sidecar down)

docker compose stop sidecar
curl -i http://localhost:9080/api/post \
  -H 'Content-Type: application/json' \
  -d '{"prompt":"hello world"}'

Expected:

• POST requests should fail-closed (503).

Restart:

docker compose start sidecar

Notes

• In audit mode, stats are in-memory per worker unless a shared store is used.
• If sidecar port is not exposed on the host, run checks from inside the container:

docker compose exec sidecar python -c "import urllib.request; print(urllib.request.urlopen('http://localhost:8000/health').read().decode())"