Backpressure & Limits

Serve sites

Every site wires the connection cap and the drain. They differ on the blanket request timeout and the body cap, and each site states its choice.

The proxy writes its receipt in the handler after the upstream hop returns. An outer timeout would drop the handler mid-hop and skip receipt finalization for a call that may already have reached the upstream, so the configured upstream timeout bounds the hop and the drain sits a margin above it.

Trust-control’s budget authorize parks in a rollback-aware quorum wait with its own bound. A blanket timeout firing after the local exposure write would drop the handler before its rollback branch and leave a charged, leader-visible write the client saw fail. The MCP edge answers GET and POST with Server-Sent Event streams that stay open while a session waits for notifications, and a blanket 408 would close a healthy idle stream and push resumable clients into reconnect churn.

On these three sites the drain deadline is the last bound on an idle request that neither streams nor waits on a bounded operation, and it arms once a stop signal fires.

Load-shed layer order

In a ServiceBuilder the first layer listed is outermost, giving HandleError(LoadShed(ConcurrencyLimit(router))). Both inner services are fallible, so a HandleErrorLayer turns the shed error into a status and keeps the result an infallible Router:

async fn shed_to_status(error: BoxError) -> StatusCode {
    if error.is::<tower::load_shed::error::Overloaded>() {
        StatusCode::SERVICE_UNAVAILABLE
    } else {
        StatusCode::INTERNAL_SERVER_ERROR
    }
}

acquire_owned errors only on a closed semaphore, and the listener treats that case as serving without a permit, so it can never wedge the accept loop. Embedders that serve the kernel behind tower add chio-tower, whose per-tenant concurrency limiter maps a kernel Overloaded to the service’s shed edge and refuses with TenantTableFull when its tenant table is full.

Body caps

DefaultBodyLimit inserts a limit into the request extensions that FromRequest extractors built on Bytes consume; it counts no bytes itself. A handler that drains the raw body brings its own bound. The MCP edge refuses an oversized Content-Length up front, then reads with an explicit 8 MiB ceiling, and answers a JSON-RPC-shaped 413 either way. The proxy reads without a Content-Length pre-check, and an over-limit read falls out of to_bytes as a plain 400 reading failed to read request body, the same answer a truncated upload gets.

Trust-control sets 1 MiB service-wide and overrides it on four routes: 384 MiB for internal admission authority, 128 MiB on each of the two receipt-append routes, and 64 MiB for evidence import. A stream receipt carries one 64-hex-character chunk digest per retained chunk, and DEFAULT_MAX_STREAM_CHUNKS is 1,048,576, so a full receipt reaches 64 MiB of digest text and the 128 MiB cap leaves room for the envelope. DefaultBodyLimitService inserts its limit unconditionally, so the innermost route-local layer writes last and wins.

Relay frame caps

The signals relay adds structural caps to its byte cap. A batch over max_batch_frames (128 in the production profile) is refused, a catch-up request over max_catchup_frames (256) is refused, and a catch-up response is bounded by max_catchup_bytes (1 MiB). The values are per peer, from the signed peer directory, and directory validation refuses a peer entry that declares zero or a value above the profile ceiling.

Operating system settings

The soft ceiling is the in-process counterpart of a cgroup hard limit. It works when the OS lets allocations fail early and locally. The four settings, in the order they take effect:

1. memory.high and memory.max, for example 3.5G and 4G, with rss_soft_limit_bytes at about 85 to 90 percent of memory.max. The shed fires at or shortly before aggressive reclaim, well under the OOM boundary.
2. vm.overcommit_memory = 2 with a tuned vm.overcommit_ratio. A large allocation fails at malloc or mmap time, so the try_reserve paths turn it into a typed deny instead of an abort.
3. oom_score_adj: +500 for the untrusted sidecar, 0 for the edge, -500 for the mediator. If the OOM killer runs anyway, it takes the least-trusted process first and the kernel last.
4. RLIMIT_AS with soft and hard set to memory.max, a coarse address-space backstop.

Under systemd these are MemoryHigh=, MemoryMax=, OOMScoreAdjust=, and LimitAS=, with the untrusted sidecar unit at OOMScoreAdjust=500. The reference units under docs/release/systemd/ set KillSignal=SIGTERM and TimeoutStopSec=35s for the drain and name the directives each deployment adds (MemoryMax, MemoryHigh, OOMScoreAdjust, LimitNOFILE, LimitAS), so both units share one OOM ordering. A drop-in for the mediator, with the example values above:

[Service]
MemoryHigh=3.5G
MemoryMax=4G
OOMScoreAdjust=-500
LimitAS=4G

Bounded collections

A node stays under its ceiling because its long-lived collections have capacity policies. crates/core/chio-bounded holds one invariant: no long-lived collection in a serving process without a capacity policy and a live size metric. It depends on no kernel or federation crate and exports three types. Ring<T> is a fixed-capacity append-only buffer that evicts oldest-first and hands the evicted item back. BoundedMap<K, V> is a capacity-bounded cache with an optional idle sweep that runs inline every 256 inserts. SizeGauge is a cloneable atomic handle anything can read and only the crate can write. On both collections capacity == 0 is a disabled mode that stores nothing and hands the item straight back.

BoundedMap evicts by oldest insert: a re-insert moves a key to newest, and get refreshes only the idle timestamp, so a key read constantly and never rewritten can be evicted ahead of a colder one. The idle sweep runs on inserts, so a map that stops taking inserts holds its entries past their TTL until the next write.

In the kernel these are a receipt mirror capped at 4096, a child receipt mirror at the same cap, and two federation artifact caches at 8192 with a one-hour idle TTL. Receipt lookups read the durable store first, so the mirror is a cache. An evicted receipt resolves when the store can point-load it through ReceiptStore::load_chio_receipt; with an append-only or remote store that cannot, a dependent call-chain claim fails closed with a deny, and a test asserts it. Each structure reports chio_mem_entries and chio_mem_capacity under its structure label, the process reports chio_process_rss_bytes and chio_process_rss_soft_limit_bytes, and chio_overload_total counts sheds by resource.

Memory budget caps

MemoryBudgetConfig::defaults() carries five more caps. Two of them refuse rather than evict.

Streamed results

The kernel bounds an oversized stream at finalize: it truncates the retained chunks and marks the receipt incomplete. A hard refusal at the dispatch seam would have to unwind the monetary charge for a stream that already executed. invoke_stream returns a materialized result, so a connector bounds its own accumulation with the public helpers push_chunk_bounded and enforce_stream_byte_limit; a connector that skips them is bounded by the process ceiling.

Tested behavior

• request_timeout_returns_408 and body_limit_returns_413: each request control returns its status at its boundary.
• load_shed_returns_503_over_concurrency_limit: a request over the concurrency limit sheds with 503 while the admitted request still returns 200.
• max_conn_listener_caps_concurrent_connections: a second connection over the cap is established at TCP level but never accepted until a permit frees.
• slow_request_times_out_cleanly_inside_the_drain_window: a 30-second handler under a 100ms timeout inside a 600ms drain returns 408 and the drain finishes clean. default_request_timeout_stays_within_drain_window keeps the two defaults ordered.
• rss_shed_persists_signed_overload_deny_receipt: a top-level shed records exactly one signed deny receipt naming the resource and returns Overloaded. nested_admission_denied_while_rss_shedding finds a matching signed receipt for the nested flow and verifies it.
• mcp_rate_limiter_caps_session_window and mcp_rate_limiter_caps_tracked_keys: the MCP limiter caps requests per window and tracked keys, and reports the seconds left in the window.
• kernel_bounded_registry_lists_every_labelled_structure: bounded_structure_gauges() enumerates every labeled kernel structure.