Secure a Swarm

Five scenarios run on any Unix host

Use the installed chio, Python 3.11 or newer, and a directory your non-root user owns. chio process init --help and chio process serve --help confirm the process host commands first, and the process quickstart introduces the host itself.

chio new security-swarm --example security-swarm
cd security-swarm
python3 run.py

ALLOW reference: all graphs completed within the family budget
DENY authority: replayed or widened authority refused
DENY revocation: revoked capability refused for every graph
RECOVER host-crash: restart resumed without repeating an effect
DENY budget: shared family budget exhausted across graphs

You should see five lines, each printed only after the host's own responses show the outcome it names. run.py builds five hosts from one plan, one per scenario in .chio/<scenario>/, serves each with chio process serve, and keeps each scenario's state and receipts in .chio/<scenario>/state. A repeat run takes python3 run.py --reset; without it the script exits 2 and leaves the existing state alone.

The project holds run.py, inspect.py, tasks.json, host-config.json, and policy.yaml. Its six workers are direct children of the root with 1,600 basis-point shares, each granted only send_jobs on the native jobs mailbox, so the example needs no separate tool server or signed launch policy. The script itself is the caller and no tool runs confined; the Linux matrix below adds confinement.

The plan binds three graphs to one family

The plan fixes every call: the child that makes it, its operation key, the tool, and the arguments. Initialization issues task authority for that exact child and request, so no caller can sign a substitute at invocation time.

{
  "schema": "chio.process.swarm-plan.v2",
  "profile_id": "security-swarm",
  "graphs": [
    {
      "graph_id": "issue-417",
      "calls": [
        {"process": "triage-417", "operation_key": "triage-417-send", "server_id": "chio-ipc", "tool_name": "send_jobs", "arguments": {"message_key": "triage-417-job", "payload": {"graph": "issue-417", "from": "triage-417"}}},
        {"process": "fixer-417", "operation_key": "fixer-417-send", "server_id": "chio-ipc", "tool_name": "send_jobs", "arguments": {"message_key": "fixer-417-job", "payload": {"graph": "issue-417", "from": "fixer-417"}}}
      ]
    },
    {
      "graph_id": "issue-418",
      "calls": [
        {"process": "triage-418", "operation_key": "triage-418-send", "server_id": "chio-ipc", "tool_name": "send_jobs", "arguments": {"message_key": "triage-418-job", "payload": {"graph": "issue-418", "from": "triage-418"}}},
        {"process": "fixer-418", "operation_key": "fixer-418-send", "server_id": "chio-ipc", "tool_name": "send_jobs", "arguments": {"message_key": "fixer-418-job", "payload": {"graph": "issue-418", "from": "fixer-418"}}}
      ]
    },
    {
      "graph_id": "issue-419",
      "calls": [
        {"process": "triage-419", "operation_key": "triage-419-send", "server_id": "chio-ipc", "tool_name": "send_jobs", "arguments": {"message_key": "triage-419-job", "payload": {"graph": "issue-419", "from": "triage-419"}}},
        {"process": "fixer-419", "operation_key": "fixer-419-send", "server_id": "chio-ipc", "tool_name": "send_jobs", "arguments": {"message_key": "fixer-419-job", "payload": {"graph": "issue-419", "from": "fixer-419"}}}
      ]
    }
  ]
}

run.py binds it with chio process init --config host-config.json --state <dir> --aggregate-invocations 6 --swarm-plan tasks.json. The root becomes a family root carrying a signed AggregateInvocationBudget of six, and for each graph the host signs:

1. A task graph and a delegation witness for each root-to-child hop.
2. A route plan and a single-use continuation token for each call, and a revocation epoch for the graph.
3. A SwarmBudgetPool whose total_units equal the family budget, one unit per call.

The host checks every bundle with verify_swarm_authority_for_admission before it writes any. A chio.process.swarm-plan.v2 plan holds two to eight graphs of two to 32 calls, at most 32 in all; every call needs its own process, the root makes none, and a process named twice is refused with "a bounded fan-out plan needs one call per distinct child".

{
  "schema": "chio.process.host.v1",
  "policy": "policy.yaml",
  "mailboxes": [{"id": "jobs"}],
  "limits": {"max_processes": 7, "max_depth": 1, "max_calls": 20},
  "children": [
    {"id": "triage-417", "parent": "root", "budget_share_bps": 1600, "tools": [{"server_id": "chio-ipc", "tool_name": "send_jobs"}]},
    {"id": "fixer-417", "parent": "root", "budget_share_bps": 1600, "tools": [{"server_id": "chio-ipc", "tool_name": "send_jobs"}]},
    {"id": "triage-418", "parent": "root", "budget_share_bps": 1600, "tools": [{"server_id": "chio-ipc", "tool_name": "send_jobs"}]},
    {"id": "fixer-418", "parent": "root", "budget_share_bps": 1600, "tools": [{"server_id": "chio-ipc", "tool_name": "send_jobs"}]},
    {"id": "triage-419", "parent": "root", "budget_share_bps": 1600, "tools": [{"server_id": "chio-ipc", "tool_name": "send_jobs"}]},
    {"id": "fixer-419", "parent": "root", "budget_share_bps": 1600, "tools": [{"server_id": "chio-ipc", "tool_name": "send_jobs"}]}
  ]
}

Read more in Swarm authority.

The family allowance stops the third graph

The budget scenario gives the family four invocations. inspect.py reads each scenario's retained decisions and queries the stopped SQLite mailboxes independently.

python3 inspect.py

budget: triage-417=allow, fixer-417=allow, triage-418=allow, fixer-418=allow, triage-419=deny, fixer-419=deny
budget: exact retry retained original; mailbox contains 4 messages across the family

Four sends are allowed and triage-419 and fixer-419 are refused with "invocation budget exhausted for capability". The first worker's exact retry returns its original response and receipt, so the mailbox holds four messages and no fifth effect.

Two ceilings apply. The swarm verifier checks each graph's allocations against its pool's total_units, and the kernel's durable admission charges every call, as an ordinary tool call on a capability descended from the root, against the root's one family budget. Every graph fits its own pool, and the family still runs out.

def scenario_budget(sock_root):
    family_budget = 4
    swarm = Swarm("budget", sock_root, family_budget=family_budget)
    swarm.host.start()
    try:
        results = []
        exact_retry = None
        for index, process in enumerate(swarm.calls):
            result = swarm.run_planned(process)
            results.append((process, result))
            if index == 0:
                exact_retry = swarm.run_planned(process)
    finally:
        swarm.host.stop()
    allowed = [process for process, result in results if sent_sequence(result)]
    exhausted = [process for process, result in results
                 if refused(result, "invocation budget exhausted for capability")]
    if (len(allowed) != family_budget or exhausted != ["triage-419", "fixer-419"]
            or exact_retry != results[0][1]):
        raise ScenarioFailure(f"the family budget did not stop the third graph: {results}")
    retain_results(swarm, dict(results))
    (swarm.dir / "exact-retry.json").write_text(json.dumps(exact_retry, indent=2) + "\n")
    print("DENY budget: shared family budget exhausted across graphs")

Each call carries the intent the host wrote for it

The Swarm initializer issues each worker its own credential and loads the host-produced swarm-calls.json. Every invocation sends the original operation key with its governed intent.

def invoke(self, process, planned, **changes):
    """Present one planned call, optionally altered, as the named process."""
    operation = {"op": "invoke", "operation_key": planned["operation_key"],
                 "server_id": planned["server_id"], "tool_name": planned["tool_name"],
                 "arguments": planned["arguments"], "governed_intent": planned["governed_intent"]}
    operation.update(changes)
    frame = (json.dumps({"protocol": PROTOCOL, "credential": self.credentials[process],
                         "operation": operation}, separators=(",", ":")) + "\n").encode()
    with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as client:
        client.settimeout(60)
        client.connect(str(self.host.socket_path))
        client.sendall(frame)
        buffer = b""
        while b"\n" not in buffer:
            chunk = client.recv(65536)
            if not chunk:
                raise ScenarioFailure("chio process serve closed the connection without a response")
            buffer += chunk
    response = json.loads(buffer.decode())
    if response.get("protocol") != PROTOCOL or response.get("ok") is not True:
        raise ScenarioFailure(f"call was not evaluated: {response}")
    return response["result"]

The governed intent references each piece of the graph's authority, from the task graph to the budget pool, by digest. Once the host serves, the kernel requires swarm context on every tool call and refuses a call without it with "kernel policy requires swarm context on every tool call". Replacing the process, tool, or operation key changes a different binding, and the retained response names the first boundary that refused.

Changed calls stop at the control that owns them

The authority, revocation, and budget scenarios break the swarm on purpose. Each refusal is a completed call: the host answers ok: true with a signed Deny verdict, no output, and a receipt.

Both binding codes sit in the receipt's chio_runtime.failure_code. The swarm verifier's own SwarmAuthorityError::Rejected reaches the caller only as the swarm-bound denial, the code a mismatched task binding also produces, with the verifier's detail text kept out of the response. The out-of-scope call changes only the tool name; its intent, still naming send_jobs, would also fail the target check, and the capability scope check answers first.

The revocation refusal names the root's capability id because the host checks every ancestor of the caller before it evaluates the call. The budget text is the message of KernelError::BudgetExhausted, raised when the durable store denies the family's invocation hold and signed into the Deny.

To repair, present the child's own bound task, request an authorized grant change for a new tool instead of editing the call, and continue the original operation under its key, since a new task needs new authority. An exhausted family waits for a reviewed new allowance, which an exact retry never creates, and withdrawal belongs to the root's authority owner.

Read more in Invocation limits, Delegation boundaries, Replay protection, Revocation, and Unknown outcomes.

The Linux matrix adds confined file and network scenarios

The qualifier ships in the CLI as chio process qualify. It runs two scenarios that need the cage and repeats the other five with confined tools and worker containers supervised by chio process run, which restarts declared workers within a bound and runs only on Linux. It needs:

• Linux x86_64 on the cage profile, as a non-root operator. The command refuses any other platform, and a root user or group.
• A verified release. The qualifier records the running chio release, its build provenance, and its own digest; run chio release verify first, because a binary that fails it refuses to qualify.
• The static chio-cage-init, the repository reader, and the enforcement probe from the reference-runtime archive, installed with install -m 755: the cage refuses a group- or world-writable or setuid or setgid helper, so mode 775 build copies fail.
• An immutable sha256: worker image id for linux/amd64; the run refuses another id or platform.
• A fresh output directory, since the run refuses an existing one, and an existing private rollback-anchor directory with no scenario subdirectories, on another filesystem device, such as one under /dev/shm kept for the run. The receipt store refuses an anchor on its database's device, since one snapshot could roll both back.
• Docker, for the worker containers.

chio process qualify run \
  --cage-init ./stage/chio-cage-init \
  --probe ./stage/enforcement-probe \
  --reader ./stage/chio-tool-repo-reader \
  --worker-image sha256:<worker-image-id> \
  --output ./matrix-run \
  --receipt-rollback-anchor-root /dev/shm/larkfield-anchor

The run records identity.json first: the hashes of every executable, the chio release and its provenance, the platform, and the worker image. It runs all seven scenarios serially, with tools under the Enforced cage, and refuses an executable that changes mid-run; a failed step stops the run, keeps its logs, and writes no matrix.json.

You should see matrix.json with every scenario's signed worker outcomes, operator-pins.json, and the run's own verification.json and negative-verification.json. The portable crash lands after a send committed and its budget is spent serially; here the crash lands while a confined write is outstanding, and four workers from two graphs contend for a family budget of two:

Read more in Process confinement.

A second machine verifies the matrix against separately sent pins

Copy matrix.json to the verifying machine and send operator-pins.json through a separate trusted channel. The pins hold the runtime and signer identities read from operator state and the exact digest of matrix.json, so pins that travel with the artifact prove nothing about it. On the verifying machine, install the same release independently and check it with chio release verify before you trust its verdict.

chio release verify
chio process qualify verify \
  --artifact ./received/matrix.json \
  --trusted-pins ./trusted/operator-pins.json
chio process qualify test-evidence \
  --artifact ./received/matrix.json \
  --trusted-pins ./trusted/operator-pins.json

Each command prints one JSON report. verify checks the digest against the pins, rechecks the original signatures with the independently installed chio, and joins every observation to the same worker identities, responses, launches, and durable accounting. test-evidence then requires 17 substitutions to be rejected: ten changed fields in matrix.json, such as another capability, receipt, or operation, a changed budget count, or an edited signed outcome, and seven structural ones, such as reordered or re-coded authority denials, a stale capture digest, a missing scenario, a swapped launch record, or a changed runtime id in the pins. Most also update the capture digest, so their rejection comes from the signatures and cross-links.

A verified matrix covers seven scenarios on one recorded host

A verified matrix shows that the seven scenarios behaved as recorded on the host, kernel, and worker image that identity.json names, with the executables it hashes; another host, kernel, or image needs its own run. The report marks itself as an evidence record: it always carries m5_acceptance_complete: false, its unchecked list always names designated_runner_authorization and combined_foundation_qualification, and the list adds execution_nonces or receipt_log_inclusion whenever a scenario's report lacks that check.

Kernel and launch signatures prove what they state. An operator's file, socket, contention, or PID observation is an assertion by the operator who captured and pinned the bundle, authenticated only by the separately retained capture digest, which the report names external_observation_authority. Verification never turns such an observation into a tool receipt or a platform authorization.

Extend it: change the scopes, then rerun the refusals

Change the child scopes, task inputs, and shared allowance in tasks.json and host-config.json. Keep the root out of tool calls and one governed intent per child operation, then repeat the allowed path and the authority, revocation, and budget refusals; a graph that completes shows nothing about its forbidden alternatives. For one governed agent, follow Secure an agent.

Next: size the family allowance for your work