Performance & Tuning

Guard cost spans four decades, and the miss rate sets the tail

Most guards finish in well under a millisecond. Two kinds dominate the tail: content classifiers and external providers.

An external guard’s hit and miss differ by three orders of magnitude, so its miss rate sets the pipeline’s tail. TtlCache takes a per-entry TTL on insert and evicts on expiry or at capacity. A longer TTL lowers the miss rate and a larger capacity lowers eviction, and both lengthen how stale a verdict can be when it admits a call, so set them against your freshness budget.

A WASM guard’s cost is a policy choice: its fuel ceiling. chio guard bench reads the fuel a module burns on a fixed request, the floor the per-module ceiling has to clear:

chio guard bench target/wasm32-unknown-unknown/release/custom_pii_guard.wasm \
  --iterations 100

=== Guard Benchmark ===

File: target/wasm32-unknown-unknown/release/custom_pii_guard.wasm
Iterations: 100
Fuel limit: 1,000,000

Latency:
  p50:  539.24 us
  p99:  3510.78 us
  min:  483.28 us
  max:  3510.78 us
  mean: 658.48 us

Fuel consumed:
  p50:  14,344
  p99:  14,344
  min:  14,344
  max:  14,344
  mean: 14,344

The module burns the same 14,344 units on every iteration, so a ceiling of 1,000,000 leaves room for inputs about seventy times larger. The p99 includes the first iterations, which compile and warm the module; the p50 is the steady per-call cost. Build a histogram per guard from chio_guard_eval_duration_seconds on the target deployment before sizing it.

Read more in WASM Guard Runtime.

Pipeline shape bounds throughput per core

The slowest resource in the pipeline sets its rate. These targets describe pipeline shapes; measure your own deployment for service levels.

Adding replicas adds receipt stores and checkpoint chains, one per replica. Measure the write path and evidence aggregation along with CPU throughput.

Receipt volume sizes the disk

A row in chio_tool_receipts keeps the canonical receipt JSON verbatim in raw_json, beside the columns the store projects out of it for queries. Each indexed projection pays again in a b-tree.

Two stores were each written by 200 preflight chio check calls against the starter policy chio init writes, every call admitted, one with a small parameter object and one with an added 2 KiB string. The measurement script reads page totals through SQLite’s dbstat table, so the bytes are the bytes on disk:

$ python3 receipt-footprint.py

chio_tool_receipts: 16 columns, 9 named indexes, 1 implicit
  idx_chio_tool_receipts_capability
  idx_chio_tool_receipts_cost
  idx_chio_tool_receipts_cost_global
  idx_chio_tool_receipts_decision
  idx_chio_tool_receipts_grant
  idx_chio_tool_receipts_subject
  idx_chio_tool_receipts_tenant
  idx_chio_tool_receipts_timestamp
  idx_chio_tool_receipts_tool

per receipt, averaged over each store:
parameter object       receipts  raw_json    table  indexes   total
{"name": "..."}             200    2630 B   4116 B    532 B  4649 B
plus a 2 KiB string         200    4640 B   5140 B    532 B  5673 B

A canonical receipt for a two-field tool call runs about 2.6 KB and costs about 4.6 KB of store, roughly 1.8 times its JSON. Two thousand more bytes of parameters add about 2 KB of JSON and about 1 KB of store, pulling the multiplier to about 1.2.

The index bytes hold steady between the stores, because the nine named indexes and the implicit b-tree behind the UNIQUE on receipt_id key on projected columns and never touch raw_json. The table grows in whole SQLite pages, so a receipt smaller than a page still pays for one.

Take the small-payload figure as the floor and scale by the parameter objects the deployment sends. A kernel sustaining 5K req/s writes 432 million receipts a day, about two terabytes a day at that floor before payload, and RetentionConfig in chio-kernel bounds it. Guard evidence rows, larger tool responses, and cost fields make receipts larger, so rerun the script against a store your deployment produced before committing to a disk budget.

Four contention points decide where to tune

Profile these four before adding replicas. Each has its own remedy.

1. Receipt store I/O. Every allowed or denied call writes one receipt, so SQLite INSERT latency is the floor for evaluation throughput on one node. The store runs in WAL mode and refuses any other journal mode; checkpoint batch size and per-tenant store sharding are the levers.
2. Session journal locks. DataFlowGuard and BehavioralSequenceGuard each hold an Arc<SessionJournal> and take its Mutex per evaluation, so concurrent calls on one session serialize. Shard sessions across journals, or batch low-stakes calls outside the session. VelocityGuard and AgentVelocityGuard hold a guard-wide Mutex over their own token-bucket map instead (VelocityGuard keys buckets by (capability_id, grant_index)), so relieving them means sharding that map.
3. WASM fuel. A guard that exhausts its fuel returns Verdict::Deny with reason_class = "fuel" on chio_guard_deny_total. The tail before that deny is the full fuel ceiling, so a lower per-module limit shortens the tail and narrows the damage a slow module does.
4. External guard circuit breaker. A degraded provider can stall a synchronous pipeline. The breaker stops the stall and refuses calls during its open window; tune RetryConfig::max_retries and CircuitBreakerConfig::reset_timeout to the provider’s service level.

Read more in Backpressure & Limits, which covers the limits these knobs keep a deployment inside and what each returns.

Six replicas carry 5K requests a second

This fleet runs the default pipeline, a policy-wired content-safety provider, and a custom WASM classifier, about 833 req/s per replica. Each key is a ChioConfig field in chio-config, and every section is deny_unknown_fields, so a typo fails to parse.

kernel:
  signing_key: "${CHIO_SIGNING_KEY}"

adapters:
  # At least one adapter is mandatory. An absent or empty adapters block
  # parses but fails validation with "at least one adapter is required".
  - id: pricing
    protocol: openapi
    upstream: "http://127.0.0.1:9440"

receipts:
  store: "sqlite:///var/lib/chio/receipts.db"
  # Receipts between Merkle checkpoints. Larger batches amortize signing
  # cost; at 833 req/s/replica, 500 checkpoints about every 0.6s.
  checkpoint_interval: 500
  # Live retention window. Aged receipts move to a read-only archive on
  # rotation and stay verifiable against their checkpoint roots.
  retention_days: 30

logging:
  level: info
  format: json

telemetry:
  enabled: true
  endpoint: "http://otel-collector:4317"
  service_name: chio-pricing-edge

guards:
  # Force these guards to run on every request, regardless of route.
  required:
    - internal-network
    - agent-velocity

wasm_guards:
  # wasm_guards is a LIST of entries, each with its own fuel_limit.
  - name: content-classifier
    path: /etc/chio/guards/content-classifier/content_classifier.wasm
    fuel_limit: 5000000
    priority: 100

The content-safety provider is an external guard wired through a HushSpec policy (see External Guards). Of the retention fields, only retention_days has a chio.yaml key; max_size_bytes, archive_path, check_interval_secs, and session-journal sharding keep their Rust defaults. Circuit-open and rate-limited verdicts keep their deny defaults, since CircuitOpenVerdict::Allow and RateLimitedVerdict::Allow are reserved for advisory guards.

Each replica keeps its SQLite receipt file on the kernel’s local disk, apart from the agent, and receipts aggregate across replicas out of band through archive rotation or a streaming receipt sink. Per-replica stores produce per-replica checkpoint chains. For one canonical chain per tenant, consolidate through a single-writer store or the federated-evidence import path; never merge checkpoint records by hand, because the chain links through previous_checkpoint_sha256.

Tuning knobs and resident bounds

import sqlite3

c = sqlite3.connect("file:small.db?mode=ro", uri=True)
names = [r[0] for r in c.execute(
    "SELECT name FROM sqlite_master WHERE type='index' "
    "AND tbl_name='chio_tool_receipts' AND name NOT LIKE 'sqlite_%' ORDER BY name")]
auto = [r[0] for r in c.execute(
    "SELECT name FROM sqlite_master WHERE type='index' "
    "AND tbl_name='chio_tool_receipts' AND name LIKE 'sqlite_%' ORDER BY name")]
cols = [r[1] for r in c.execute("PRAGMA table_info(chio_tool_receipts)")]
c.close()
print("chio_tool_receipts: %d columns, %d named indexes, %d implicit"
      % (len(cols), len(names), len(auto)))
for n in names:
    print("  " + n)
print()
print("per receipt, averaged over each store:")
print("%-22s %8s %9s %8s %8s %7s" % ("parameter object", "receipts", "raw_json", "table", "indexes", "total"))
for label, path in (('{"name": "..."}', "small.db"), ("plus a 2 KiB string", "large.db")):
    c = sqlite3.connect("file:%s?mode=ro" % path, uri=True)
    n = c.execute("SELECT COUNT(*) FROM chio_tool_receipts").fetchone()[0]
    avg = c.execute("SELECT AVG(LENGTH(raw_json)) FROM chio_tool_receipts").fetchone()[0]
    tbl = c.execute("SELECT SUM(pgsize) FROM dbstat WHERE name = 'chio_tool_receipts'").fetchone()[0]
    idx = c.execute("SELECT SUM(pgsize) FROM dbstat WHERE name LIKE '%chio_tool_receipts%'"
                    " AND name <> 'chio_tool_receipts'").fetchone()[0]
    print("%-22s %8d %7.0f B %6.0f B %6.0f B %5.0f B"
          % (label, n, avg, tbl / n, idx / n, (tbl + idx) / n))
    c.close()

Next: watch the deployment you sized