Sidecar HTTP Service
Run the kernel in its own container beside your application, holding the signing key, and reach it over localhost from any language.
On this page
A Chio sidecar runs the kernel in its own container next to your application. It holds the signing key and the receipt store, listens on port 9090, and admits each request before your upstream sees it.
Any language with an HTTP client can use it, and one image serves Rust, Python, Go, Node, .NET, and Java hosts. Cloud Run, ECS Fargate, and Azure Container Apps run it as a second container with localhost networking and startup ordering.
StartOnly the sidecar container holds the signing key
Service: only the sidecar holds the key
An application in any language reaches the kernel over HTTP on the sidecar’s listener, 127.0.0.1:9090 by default, and the sidecar proxies admitted calls to the upstream API. The signing seed is mounted only into the sidecar container; container and network settings decide how much isolation that achieves. HTTP service deployment.
- admitted route
- HTTP :9090 to the kernel, the reverse proxy, then the upstream
- signing key
- seed file mounted into the sidecar container only
- stores
- SQLite receipt store, set by --receipt-store
- trust boundary
- the sidecar container; the app container has no key
In Rust · types.rs:11811 lines
/// Optional SQLite database path for durable receipt persistence.
#[arg(long, global = true)]
pub(crate) receipt_db: Option<PathBuf>,
// …
/// Optional file path for a persistent capability-authority seed.
#[arg(long, global = true)]
pub(crate) authority_seed_file: Option<PathBuf>,
// …
/// Optional SQLite database path for durable admission, trust authority, and remote MCP session state.
#[arg(long, global = true)]
pub(crate) session_db: Option<PathBuf>,
Mount the Ed25519 signing seed into the sidecar container and leave it out of the application container. Protect the sidecar process, its secret mount, and its signing API so a compromised application gains no signing authority; container and network settings decide the isolation a deployment achieves.
The split also lets a separate team roll the kernel forward without redeploying the host. Each call adds local HTTP transport and serialization to kernel evaluation, so measure the configured pipeline with your request sizes and concurrency before you set a latency budget.
| Property | In process | Sidecar |
|---|---|---|
| Holds the signing key | Host process | Sidecar container |
| Cost per kernel call | A function call | Local HTTP and serialization; measure it |
| Scaling | With the host | Kernel and host separately |
| Rolling policy configuration | Rebuild the embedded host configuration | Roll the sidecar revision with a drain |
| Host language | Rust | Any language with an HTTP client |
| Fits | Trusted single-tenant binaries | Untrusted code, multi-tenant, third-party agents |
1. Pull the sidecar image
Every release publishes the sidecar as a signed two-architecture image. The steps put it in front of the pricing-service example (chio new pricing-service --example pricing-service), running on port 8000 with its openapi.yaml in the current directory.
docker pull ghcr.io/backbay-labs/chio-sidecar:0.2.0| Variant | Base | Runs as |
|---|---|---|
| Alpine (default) | Alpine 3.22 and tini, musl-linked | uid and gid 10001 |
| Distroless | distroless cc-debian12 nonroot | uid 65532 |
The Alpine image is the small default and runs /usr/local/bin/chio under tini. The distroless variant installs /usr/local/bin/chio-sidecar in a multi-arch shared-library layout and enters it directly, with no shell and no tini. It bakes in curl and its shared libraries so its HEALTHCHECK runs without a shell. Container Images compares every image and shows how to verify its signature.
2. Run it in front of your API
Flags configure the sidecar; there is no config file to mount. chio api protect puts a reverse proxy in front of an OpenAPI host. This local run mounts the spec, persists the generated seed under the image's writable CHIO_HOME, and keeps receipts in memory; chio mcp serve-http fronts an MCP server the same way, as the reference shows.
docker run --rm \
-p 9090:9090 \
-v "$(pwd)/openapi.yaml:/etc/chio/spec/openapi.yaml:ro" \
ghcr.io/backbay-labs/chio-sidecar:0.2.0 \
--authority-seed-file /var/lib/chio/authority.seed \
api protect \
--upstream http://host.docker.internal:8000 \
--spec /etc/chio/spec/openapi.yaml \
--listen 0.0.0.0:9090 \
--allow-ephemeral-receipts--upstream is the only required flag. --spec is discovered when omitted, and the kernel derives its route and scope table from that OpenAPI document. --listen defaults to 127.0.0.1:9090; every reference manifest binds 0.0.0.0:9090, so the application container reaches the kernel at http://localhost:9090 over the network namespace its pod, task, or revision shares.
In a deployment, point --receipt-store at a SQLite audit log in place of --allow-ephemeral-receipts. The signing seed is the global --authority-seed-file flag: a file path the kernel generates and persists at mode 0600 when it is missing, and never an environment variable carrying key material. A bare docker run prints --help and exits successfully, because every subcommand needs operator input, so override CMD at deploy time.
3. Gate the application on readiness
The health routes are fixed at /chio/health and /chio/live. Check both once the sidecar is up.
curl -fsS http://localhost:9090/chio/health
curl -fsS http://localhost:9090/chio/live{"status":"healthy","version":"0.1.0","receipt_backend":"ephemeral","revocation_backend":"ephemeral"}
{"status":"healthy","version":"0.1.0","receipt_backend":"","revocation_backend":""}You should see healthy, with readiness naming the receipt and revocation backends. /chio/health is dependency-aware: it returns 503 when the receipt store can no longer persist, which pulls the instance from rotation, and startup and readiness probes gate on it. /chio/live is process-only and returns 200 while the process runs, so a dependency blip leaves a serving container running.
Both return a HealthResponse. status is healthy, degraded, or unhealthy; on readiness, receipt_backend and revocation_backend report durable or ephemeral, with remote as a third revocation value. A spec or durable receipt store that fails to open at startup exits the process non-zero: the orchestrator marks the container unhealthy (ECS, Azure) or fails the revision (Cloud Run). The distroless image's HEALTHCHECK polls liveness.
HEALTHCHECK --interval=10s --timeout=5s --start-period=15s --retries=3 \
CMD ["/usr/bin/curl", "-fsS", "http://localhost:9090/chio/live"]4. Supervise it under systemd or your orchestrator
On a VM or bare-metal host, run the sidecar under a unit that pins the binary, mounts the secret directory, confines the service, and restarts on failure.
[Unit]
Description=Chio sidecar HTTP service
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User=chio
Group=chio
ExecStart=/usr/local/bin/chio \
--authority-seed-file /etc/chio/seed/authority.seed \
api protect \
--upstream http://127.0.0.1:8000 \
--spec /etc/chio/spec/openapi.yaml \
--listen 0.0.0.0:9090 \
--receipt-store /var/lib/chio/receipts.db
Environment=RUST_LOG=info
EnvironmentFile=/run/secrets/chio.env
Restart=on-failure
RestartSec=2s
TimeoutStopSec=30s
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
PrivateTmp=true
ReadWritePaths=/var/lib/chio
[Install]
WantedBy=multi-user.targetTimeoutStopSec=30s gives the drain its window. On multi-container platforms the orchestrator handles startup ordering and health gating, with manifests under deploy/. Cloud Run uses run.googleapis.com/container-dependencies and an httpGet startup probe, ECS Fargate uses dependsOn with condition: HEALTHY and a curl-based healthCheck, and Azure Container Apps uses Bicep multi-container probe sequencing.
5. Send a refused request and an admitted one
This run uses the same binary outside a container, against the pricing service and its OpenAPI document, with a durable receipt store, a trust control plane, and the trust file whose issuer pin admits capabilities your kernel issued. Its startup line states what the sidecar can verify later.
chio --trust ./trust.toml \
--control-url http://127.0.0.1:8940 --control-token "$TOKEN" \
api protect \
--upstream http://127.0.0.1:8000 \
--spec ./openapi.yaml \
--listen 127.0.0.1:9090 \
--receipt-store ./state/sidecar-receipts.sqlite3WARN chio::sidecar message=no --authority-seed-file: a fresh signer is generated per boot, so receipts signed before a restart are unverifiable
Without --authority-seed-file each boot generates a fresh signer, so pass the flag wherever receipts must verify after a restart. POST /v1/refund is a side-effecting route in the pricing document. Send a refund for order-77 with no capability header:
curl -si -H 'content-type: application/json' \
--data '{"order":"order-77"}' \
http://127.0.0.1:9090/v1/refundHTTP/1.1 403 Forbidden
content-type: application/json
x-chio-receipt-id: rcpt-01c3…0078
content-length: 228
{"error":"chio_access_denied","message":"side-effect route requires a capability token","receipt_id":"rcpt-01c3…0078","suggestion":"provide a valid capability token in the X-Chio-Capability header or chio_capability query parameter"}You should see a 403 with a JSON body, and the pricing service never receives the call: no money moves. The receipt id appears in the body and in the x-chio-receipt-id header, so a caller that discards the body still holds the evidence pointer. At the same instant the sidecar names the guard on stderr; a guard that cannot answer, or answers no, fails the request closed.
WARN chio_kernel::kernel::evaluation::async_evaluation_core message=guard denied request_id=019fbe3e-82c3…3939 reason=guard denied the request: guard "http_projection_policy" error (fail-closed): guard denied the request: side-effect route requires a capability tokenNow send a quote with the outside buyer's capability from Quickstart part 2, which grants quote and nothing else:
curl -si -H 'content-type: application/json' \
-H "X-Chio-Capability: $(cat basalt.capability.json)" \
--data '{"sku":"sku-118","percent":10}' \
http://127.0.0.1:9090/v1/quoteHTTP/1.1 200 OK
server: uvicorn
content-type: application/json
content-length: 64
x-chio-receipt-id: rcpt-01c2…274b
{"sku":"sku-118","percent":10.0,"price":21.6,"currency":"USD"}Both calls are recorded. The route's effect class in the OpenAPI document decides whether it needs a capability token in X-Chio-Capability; the token's scope then decides which side-effecting routes it reaches. The same token sent to /v1/refund is refused as out of scope. Connect an Application runs the pricing service end to end.
Requests reach the kernel as JSON over HTTP
In chio api protect mode the sidecar is the front door. It terminates the inbound request, validates the capability, runs guards, forwards an admitted request to the upstream, and signs a receipt on the way out, and the upstream host never sees the original capability token.
Hosts that call the kernel directly use its endpoints for capability validation, tool dispatch, and receipt retrieval. Every request is JSON, binary payloads are base64-encoded, and Wire Protocol defines the format.
SIGTERM drains admitted requests before the process exits
ringressKchio-sidecar :9090dhost app :8080draingrace
SIGTERM runs the drain sequence
The sidecar stops accepting connections, lets admitted requests finish and emit receipts, flushes the receipt log, drains the signing task with a final checkpoint where a boundary is crossed, and exits 0. A 30-second grace holds the 25-second drain. Graceful shutdown.
- Stop accepting. The listen address takes no new connections.
- Finish in-flight requests. A request that has passed the guard pipeline completes and emits its receipt.
- Flush the receipt log to the SQLite database at
--receipt-store, or to the in-memory log under--allow-ephemeral-receipts. - Drain the signing task and emit a final checkpoint if a checkpoint boundary is crossed.
- Exit 0.
The drain lasts the upstream hop ceiling plus five seconds. The default --upstream-timeout-secs of 20 seconds derives a 25-second drain, which a 30-second grace holds. A 30-second hop ceiling asks for 35 seconds, so raise the orchestrator's grace with it: past the grace the platform escalates to SIGKILL, and receipts for calls still in flight are lost.
In the Alpine image tini is PID 1 and forwards signals to the chio binary. The distroless image enters chio-sidecar directly as PID 1, and the binary installs its own SIGTERM handler.
Read more in Node Lifecycle.
Three symptoms trace back to how the sidecar started
The container exits right after it starts
Exit 0 means the image ran with no subcommand and printed --help, so the health endpoint never opened. Give it a full command: chio api protect requires --upstream, and chio mcp serve-http requires --policy, --server-id, and the trailing server command. A non-zero exit means the spec or the durable receipt store failed to open.
Readiness answers 503
The receipt store can no longer persist. The instance is out of rotation while liveness keeps the container running; restore the store's disk or volume, and readiness returns the instance to rotation.
Receipts from before a restart fail to verify
The sidecar started without --authority-seed-file and logged the fresh-signer warning above. Pass a persistent seed file, mounted into the sidecar only.
Image contents, other commands, and environment variables
Try CHIO_SIDECAR_CONTROL_TOKEN, --server-id, or OPENSSL_STATIC.
What the image is built with
protoc.chio-envoy-ext-authz, reachable through the workspace, runstonic-buildat compile time. CI installsprotobuf-compilerfor the same reason, and matching it keeps a dependency change from breaking the image unnoticed.- Static OpenSSL.
openssl-dev,openssl-libs-static, andOPENSSL_STATIC=1.chio-custody-hw, reached throughchio-kernel -> chio-custody-hw, pulls inwebauthn-rs-core, which linksopenssl-sys. Static linking keeps the runtime layer to CA roots plustini, with nolibssl.sodependency. - The whole workspace.
wit/feedschio-wasm-guardsthroughwasmtime::component::bindgen!(viachio-cli -> chio-wasm-guards), andcontracts/is embedded bychio-web3-bindingsthrough Alloy'ssol!macro.bench/,examples/,formal/,tests/,integrations/(the aws-bedrock control plane and mcp-adapter members), andxtask/are workspace members thatCargo.lockresolution needs.sdks/is copied as well.
chio mcp serve-http
chio mcp serve-http fronts an MCP tool server over HTTP. It requires --policy, --server-id, and the wrapped server command after --.
chio mcp serve-http \
--policy /etc/chio/policy.yaml \
--server-id pricing \
-- python mcp_server.pychio start
chio start is a convenience alias for api protect aimed at SDK quickstart and chio-hermes users. It runs the same router with no upstream proxy and durable receipts by default (--allow-ephemeral-receipts for the in-memory quickstart). Its default --listen 127.0.0.1:9090 matches ChioClient.DEFAULT_BASE_URL in the Python SDK, and --print-config prints a chio-hermes config snippet on startup.
Environment variables
The listen address, OpenAPI spec, receipt store, and signing seed are the --listen, --spec, --receipt-store, and --authority-seed-file flags. Kernel configuration and the policy source are flags too, and the health paths are fixed. chio api protect reads the CHIO_* variables below at startup, in crates/products/chio-cli/src/cli/runtime.rs; RUST_LOG sets the log level.
| Variable | Default | Purpose |
|---|---|---|
RUST_LOG | info | Log level for tracing_. The image sets it, and the manifests also carry a plain CHIO_LOG_LEVEL value by convention. |
CHIO_TRUST (or --trust <FILE>) | the project's .chio/trust.toml, then ~/.chio/trust.toml | Trust file whose issuer pins admit presented capabilities. |
CHIO_ | none | Bearer token for the admin and /metrics endpoints. Loopback callers need no token; remote callers present it. Falls back to CHIO_. |
Next: run the pair on a container platform
Deploy and operate · 1 of 3
NextSecrets & Keys