Threat coverage
See which boundary stops each threat, which test refutes the claim, and the mutation evidence that the test catches a broken defense.
On this page
Each row of Chio's threat inventory pairs an attacker action with a conformance test that fails when the defense it names is removed. The evidence is mutation testing: a deliberately broken defense that the test must catch.
A row records the attacker-controlled input, the enforcing boundary, the test, and the scope its evidence supports. The tool_server_escape row is the worked example: a capability for one tool, two connections that accept anything, and a kernel that must refuse a pivot before either connection counts an extra call.
Both pivots stop at the kernel
Changing the tool to exec_privileged or the server to pivoted-server returns a signed denial. The admitted counter stays at 1 and the pivoted counter at 0; both connections accept every invocation, so only the kernel's scope check can have stopped the calls. Least privilege.
Each row names one enforcing boundary and the observation that would refute it
A row names the component that must prevent the unwanted effect and the counterexample that would show it failed.
| Threat family | Enforcing boundary | Counterexample to retain |
|---|---|---|
| Capability theft or delegation abuse | Identity, scope, expiry, audience, root binding | Replayed or broadened authority |
| Cumulative disclosure | Retained flow state and egress clearance | Sensitive read, then incompatible outbound data |
| Native tool escape | Cage launch and OS resource restrictions | Denied file, syscall, or network operation |
| Credential misuse | Broker destination, request constraints, attempt state | Changed request or repeated captured use |
| Key-history substitution | Contiguous log, witnesses, activation, artifact time | Omitted event, split view, or backdating |
| Unsafe containment recovery | Per-effect ownership and rollback | Remaining control reported as lifted |
| Interrupted external effect | Durable capture and original-operation recovery | Resend or refund after an unknown outcome |
The table is a reading map. The evidence record behind each row sets the supported scope for a particular test and deployment, so read the claim, the exact test, the mutation result, and the residual scope together before extending a conclusion to another host, adapter, or deployment.
A count of tests shows little: several tests can exercise one parser while an alternate dispatch path goes untested. A small test with an independent resource observation establishes a specific property. Each evidence record answers six questions.
- Threat ID and surfaces. Which attacker action and entrypoint are covered.
- Test path and baseline command. How to reproduce the defended behavior.
- Closure scope. The exact sub-vector the test establishes.
- Mutation and caught result. Whether breaking the defense makes the test fail.
- Survivors. Which altered behaviors escaped the assertions.
- Run identity and retained artifacts. Which source, toolchain, and execution produced the evidence.
Take the brokered issue tool the triage agent uses to open issue #419. A refutable claim says that changing the approved repository after authorization never makes the broker contact the substituted receiver. Build the valid request and observe the permitted receiver accept it; then alter the destination while keeping the original signed authority, and observe both the broker's result and the substituted receiver.
The resource assertion is the oracle: an implementation that sends first and reports denial afterward passes an error-string check and fails this one. The mutation removes or bypasses the production destination comparison. Other adapters, provider behavior, credential custody, parent revocation, captured-attempt replay, redirects, credential export, and provider outcomes each fail differently and need their own claims and oracles.
The tool_server_escape test counts dispatches, so a connector cannot hide a missing check
A capability permits admitted-server/read_public. The test changes the tool to exec_privileged, or the server to pivoted-server, and both registered connections accept every invocation, so a connector's own refusal cannot stand in for the kernel's scope check.
The test sends the same three requests to the portable verified-core evaluator and to the hosted kernel: the admitted call is allowed and the admitted counter reaches 1, then both pivots return signed denials while the admitted counter stays at 1 and the pivoted counter at 0.
The complete test
#[test]
fn threat_tool_server_escape_kernel_mediation_blocks_tool_and_server_pivots() {
// covers: tool_server_escape
//
// The connection deliberately accepts every invocation. Only the kernel's
// capability mediation can prevent these pivots. A positive control first
// proves the admitted path can run. Both pivots must then be denied without
// incrementing either registered connection's invocation count.
let mut kernel = escape_test_kernel();
let admitted_invocations = Arc::new(AtomicUsize::new(0));
let pivoted_invocations = Arc::new(AtomicUsize::new(0));
kernel.register_tool_server(Box::new(PermissiveEscapeServer {
id: "admitted-server",
invocations: Arc::clone(&admitted_invocations),
}));
kernel.register_tool_server(Box::new(PermissiveEscapeServer {
id: "pivoted-server",
invocations: Arc::clone(&pivoted_invocations),
}));
let subject = Keypair::from_seed(&[0x52; 32]);
let capability = match kernel.issue_capability(&subject.public_key(), admitted_scope(), 300) {
Ok(capability) => capability,
Err(error) => panic!("issue mediation capability: {error}"),
};
let clock = FixedClock::new(current_unix_seconds());
let guards: &[&dyn chio_kernel_core::Guard] = &[];
let portable_admitted = kernel.evaluate_portable_verdict(
&capability,
&portable_escape_request(
"tool-server-escape-portable-admitted",
&capability,
"admitted-server",
"read_public",
),
guards,
&clock,
None,
);
assert_eq!(portable_admitted.verdict, PortableVerdict::Allow);
let portable_tool_pivot = kernel.evaluate_portable_verdict(
&capability,
&portable_escape_request(
"tool-server-escape-portable-tool-pivot",
&capability,
"admitted-server",
"exec_privileged",
),
guards,
&clock,
None,
);
assert_eq!(portable_tool_pivot.verdict, PortableVerdict::Deny);
let portable_server_pivot = kernel.evaluate_portable_verdict(
&capability,
&portable_escape_request(
"tool-server-escape-portable-server-pivot",
&capability,
"pivoted-server",
"read_public",
),
guards,
&clock,
None,
);
assert_eq!(portable_server_pivot.verdict, PortableVerdict::Deny);
let admitted = kernel.evaluate_tool_call_blocking(&escape_request(
"tool-server-escape-admitted",
&capability,
"admitted-server",
"read_public",
));
let admitted = match admitted {
Ok(response) => response,
Err(error) => panic!("admitted control request must complete: {error}"),
};
assert_eq!(admitted.verdict, Verdict::Allow);
assert_eq!(admitted_invocations.load(Ordering::SeqCst), 1);
let tool_pivot = kernel.evaluate_tool_call_blocking(&escape_request(
"tool-server-escape-tool-pivot",
&capability,
"admitted-server",
"exec_privileged",
));
let tool_pivot = match tool_pivot {
Ok(response) => response,
Err(error) => panic!("tool pivot must return a signed denial: {error}"),
};
assert_eq!(tool_pivot.verdict, Verdict::Deny);
assert_eq!(admitted_invocations.load(Ordering::SeqCst), 1);
let server_pivot = kernel.evaluate_tool_call_blocking(&escape_request(
"tool-server-escape-server-pivot",
&capability,
"pivoted-server",
"read_public",
));
let server_pivot = match server_pivot {
Ok(response) => response,
Err(error) => panic!("server pivot must return a signed denial: {error}"),
};
assert_eq!(server_pivot.verdict, Verdict::Deny);
assert_eq!(admitted_invocations.load(Ordering::SeqCst), 1);
assert_eq!(pivoted_invocations.load(Ordering::SeqCst), 0);
}The supporting kernel, request, and permissive-connection declarations are in the complete conformance file, with a source-copy manifest. The download is the exact test source; every release runs it as part of the threat suite.
Two caught mutants show the test notices a broken matcher
The promoted record for tool_server_escape comes from a cargo-mutants run (timestamp_kind: cargo-mutants-run, needs_real_run: false). It records two caught production matcher mutations and no survivors for closure scope hosted-and-portable-kernel-capability-mediation: hosted-kernel and portable verified-core mediation of tool-name and server-ID pivots.
The same test also exercises WASM host-import and fuel containment. Generated metadata is never a mutation run: a positive count without its execution provenance shows no mutant was caught.
When a mutant survives, keep the altered code and the test result, then decide whether the edit is behaviorally equivalent in the claimed scope, the fixture missed the branch, or the assertion is too weak, and record that disposition. Deleting an inconvenient survivor breaks the link between the coverage label and the defended behavior.
Read more in Mutation testing, which covers the catch ratio and how candidates are discovered.
Two gates hold the inventory to its evidence
Every Chio release passes the threat-coverage gate before it is tagged: one check holds each row's coverage state and the shape of its backing test, and the other requires promoted mutation evidence for the defended scope and rejects placeholder or generated-only records. The release workflow signs the resulting report, and chio release verify --evidence threat-coverage shows it for the release you run.
chio release verify --evidence threat-coveragebinary /usr/local/bin/chio chio 0.2.0 x86_64-unknown-linux-gnu checksum matches SHA256SUMS for v0.2.0 signed by release.yml on backbay-labs/chio, issuer token.actions.githubusercontent.com provenance SLSA v1, 5 subjects, tag v0.2.0 threat coverage report bound to v0.2.0: every row covered or explicitly out of scope mutation promoted mutation evidence for the defended scope verified
You should see the coverage and mutation lines bound to the tag you run. The mutation report records 20 passed, 0 weak, and 0 bootstrap placeholders. Both checks read recorded evidence: neither reruns cargo-mutants, so a fresh claim needs the selected mutant run and its protected-resource assertion inspected. The gate scripts themselves live on the contributor qualification page.
For a row the mutation check rejects, retain the mutation campaign before updating its status. For a missing backing test, restore its behavioral scope. Keep a valid neighboring call in the test: a defense that returns Deny for every request passes the hostile control and fails that one.
Read more in Behavioral gates, which covers suite ownership and CI triggers.
A row covers the boundary its test exercised
Components that are each correct can still fail at a handoff. A label library can order restrictions correctly while an adapter drops the destination clearance, and a durable store can capture a request atomically while the connector sends a changed body. A response adapter can remove its own contribution while the scheduler reports another as lifted.
For each seam, test a mismatch between what the sender commits to and what the receiver consumes, keeping the original authority and varying the transferred identity, fields, or order. A test across the deployed path shows it still prevents an effect that crosses a process or service boundary; small tests isolate the invariant and keep mutants fast to diagnose.
Neighboring claims stay separate. A cumulative byte-ceiling test that allows a session at 900 bytes, denies one at 1,200 bytes under a 1,000-byte ceiling, and catches an inverted comparison establishes that boundary; cross-session aggregation, compression-aware measurement, semantic limits, and the deployment's classification policy are separate claims. Isolation of in-process connector code, OS or container sandboxing, filesystem access, and outbound-network restrictions come from the host, as the assurance model sets out, and a Linux confinement observation covers that deployment's restriction while other host profiles carry their own evidence.
- Changing an adapter adds the positive path and the hostile variation through that adapter.
- Changing recovery adds process death at the physical boundary it depends on.
- Changing classification or response logic adds mutants that would silently allow disclosure or falsely report cleanup.
Read more in Adversarial testing and the threat model.