Rate Limit Guards

Each key drains its own bucket and refills with time

A bucket starts full at its capacity. Whenever it is checked, it refills from the monotonic time elapsed since the last check, then admits the call only if one whole token remains.

fn burst_capacity(per_window: u64, burst_factor: f64) -> u64 {
    ((per_window as f64 * burst_factor).round() as u64).max(1)
}

The arithmetic is integer. One logical token is 1,000 milli-tokens (MT_PER_TOKEN), and the refill rate is milli-tokens per millisecond, rounded down and floored at one. That floor is one token per second, so a configured rate slower than that refills faster than asked; a slower or exact window ceiling belongs to the durable budget store.

Rounding also lowers fast rates. With 100 calls per minute and burst_factor = 1.5, capacity is 150 tokens (150,000 milli-tokens) and refill is one milli-token per millisecond: 60 tokens per minute instead of the requested 100. An idle bucket admits a burst of 150, and admission then follows the achieved rate.

VelocityGuard gives every grant its own limit

VelocityGuard keys its buckets by (capability_id, grant_index). Two grants inside one capability each get a bucket; two capabilities that carry the same ID share one. When either limit is configured, the guard needs the grant the kernel matched, and a missing matched_grant_index returns Err(KernelError::Internal), which the kernel refuses on.

Setting max_spend_per_window adds a second bucket that meters money. Each call draws the matched grant's max_cost_per_invocation.units, the declared ceiling for one call. Spend limiting applies when max_spend_per_window is Some, the kernel matched a grant, and that grant carries max_cost_per_invocation; any other combination returns an error and the call is refused.

The guard resolves both buckets under one state lock, in three phases, so a refusal from one limit leaves the other untouched:

1. Reject impossible spend. A declared cost larger than the spend bucket's burst ceiling can never fit, so the guard refuses before creating any bucket, and the refused call takes no table slot.
2. Reserve. The guard reserves a slot for every new bucket the call needs across both maps at once. A new key with both limits on needs two, so a call refused for lack of room cannot leave an unpaired invocation bucket in the last free slot.
3. Peek, then commit. can_consume runs on both buckets before consume runs on either, so a call one limit refuses drains nothing from the other.

AgentVelocityGuard limits an agent across its capabilities

AgentVelocityGuard runs the same bucket over different keys. max_requests_per_agent keys a bucket on agent_id alone, and max_requests_per_session keys one on (agent_id, capability_id). The guard context exposes no session ID, so the capability stands in for the session: one agent holding two capabilities counts as two sessions.

Both count in integer milli-tokens over monotonic Instant time, and both return an error the kernel refuses on when their state lock is poisoned.

A full bucket table refuses newcomers and keeps live limits

Each guard holds both of its bucket maps behind one mutex, so checking the combined count and inserting a bucket happen as one step, and no evaluation reads a stale size of the sibling map.

bucket_cap bounds that combined count. It defaults to chio_kernel::MemoryBudgetConfig::defaults().velocity_bucket_cap, so lowering the process memory budget tightens the table. When a new key would pass the cap, the guard calls prune_refilled to reclaim buckets that have refilled to capacity, which are equivalent to new ones. If only buckets holding live state remain, the new key is refused.

Evicting a live bucket would let a caller reset a depleted limit by presenting a new capability ID, so the depleted bucket stays and the newcomer is refused. The bound and the pruning both come from the guard itself.

Read more in Fail-Closed Semantics.

Two guards in series cap billing and abuse separately

An embedding host can place both guards in one pipeline. The agent-grain buckets bound abuse across capabilities, and the grant-grain buckets bound what one grant may bill. Their state is independent, so a call that clears one can still be refused by the other.

use chio_guards::velocity::VelocityConfig;
use chio_guards::{AgentVelocityConfig, AgentVelocityGuard, VelocityGuard};

let mut pipeline = chio_guards::GuardPipeline::new();

// Agent-grain throttle (cross-capability, broad)
pipeline.add(Box::new(AgentVelocityGuard::new(AgentVelocityConfig {
    max_requests_per_agent: Some(600),
    max_requests_per_session: Some(120),
    window_secs: 60,
    burst_factor: 1.0,
})));

// Capability-grain throttle (per-grant, narrow)
pipeline.add(Box::new(VelocityGuard::new(VelocityConfig {
    max_invocations_per_window: Some(60),
    max_spend_per_window: Some(1_000),
    window_secs: 60,
    burst_factor: 1.5,
})));

// Install on the existing mutable kernel.
kernel.add_guard(Box::new(pipeline));

The spend bucket charges each grant's declared per-call ceiling and refills by the arithmetic above; it never reconciles actual charges. An enforceable aggregate spending budget lives in the durable budget store, whose hold reserves the worst case before dispatch, and a family cap bounds what sibling grants spend together. Byte volume across a session is the data-flow guard's job.

Read more in Budgets.

Next: bound what a session reads and runs