Testing Guards & Policies

chio check: dry-run a policy

chio check evaluates a single tool call against a policy with no server, no agent, and no MCP connection. It runs the same compiled guard pipeline the kernel runs at request time (see Compile Model) through a short-lived kernel session, so the guard names and receipt ids it reports come from that session and not from a separate simulator.

Preflight vs full mode

Most guards only need the request to reach a verdict, so the default preflight mode never touches a tool server. Guards that inspect what the tool returned, such as Response Sanitization, have no response to inspect in preflight. Rather than evaluate them against nothing, chio check refuses to run preflight at all when the compiled policy has a post-invocation pipeline, and says what to do instead.

Every chio check transcript on this page runs against examples/policies/canonical-hushspec.yaml copied to ./policy.yaml. That policy declares rules.secret_patterns, which inspects what the tool returned, so preflight has nothing to evaluate it against:

$ chio --session-db ./admission-0.db --receipt-db ./receipts.db \
  check --policy ./policy.yaml --tool read_file \
  --params '{"path": "/workspace/README.md"}'

error [urn:chio:error:cli:other]: chio check preflight cannot evaluate post-output guards; use --mode full --output-fixture <JSON> so output-sensitive policy is evaluated against explicit fixture output
context: {"domain":"cli","severity":"error","stability":"deprecated","string_code":"CHIO-CLI-OTHER"}
suggested fix: Preserve the original message and migrate the call site to a specific registry code when touched.

The refusal is symmetric, and both halves exit 1. --mode full without a fixture says chio check --mode full requires --output-fixture <JSON>; use --mode preflight for admission-only checks, and --output-fixture in preflight says --output-fixture requires --mode full. The fixture is a JSON file holding the value the tool would have returned, and there is no default.

A run prints eight lines, nine on a deny: verdict, tool, server, reason (deny only), receipt_id, the compiled policy and source content hashes, then mode and fixture. On a guard deny the reason names the compiled pipeline rather than the child guard; the guard that fired is on the receipt, in the evidence array. --json prints the same fields as one object, with the two hashes spelled policy_hash and policy_source_hash.

The two database flags are global options and work on either side of the check subcommand; the transcripts here pass them first. --session-db holds durable admission state and --receipt-db holds receipts. The two paths must differ, and pointing both at one file fails with durable admission database must not alias receipt database. Every chio check run uses the fixed request id check-001, which the receipt records under metadata.receipt_context.request_id, so give each case in a suite its own admission database and let them share one receipt database.

Exit codes

Smoke-test cases

The examples in these docs, from the Quickstart to Wrap an MCP Server, test a policy with an allowed call and a denied call for each relevant guard.

Three cases cover the shape. The first is a call that must survive; the second trips a guard; the third asks for a tool the policy never granted. Write the output fixture once with echo '{"content":"ok"}' > output-fixture.json and reuse it. Case 1 is a shell command none of the policy's three forbidden_patterns regexes match:

$ chio --session-db ./admission-1.db --receipt-db ./receipts.db \
  check --policy ./policy.yaml --tool run_command \
  --params '{"command": "ls -la /workspace"}' \
  --mode full --output-fixture ./output-fixture.json

verdict:    ALLOW
tool:       run_command
server:     *
receipt_id: 5f65ab2d79acf48f7c6c8d7ddeb202ea775a090ae4ea751040d5eb6d9e78ad5c
policy:     89e8cd101e3fb0c311cceb9bd7b988a6a65d371310b4fccceb08a015333885da
source:     11e1aa04a3696c42034ae77afb4f9aee4e5c16c4e38a58f2a21fc2fa137779d8
mode:       full
fixture:    true

Case 2 reads a path forbidden_paths covers with **/.env:

$ chio --session-db ./admission-2.db --receipt-db ./receipts.db \
  check --policy ./policy.yaml --tool read_file \
  --params '{"path": "/workspace/.env"}' \
  --mode full --output-fixture ./output-fixture.json

verdict:    DENY
tool:       read_file
server:     *
reason:     guard denied the request: guard "guard-pipeline" denied the request
receipt_id: 0b8dd0573f189324215246c2b4509ee6f867724000d795bbc0f66b65cc016c33
policy:     89e8cd101e3fb0c311cceb9bd7b988a6a65d371310b4fccceb08a015333885da
source:     11e1aa04a3696c42034ae77afb4f9aee4e5c16c4e38a58f2a21fc2fa137779d8
mode:       full
fixture:    true

WARN chio_kernel::kernel::evaluation::async_evaluation_core message=guard denied request_id=check-001 reason=guard denied the request: guard \"guard-pipeline\" denied the request

Case 3 asks for a tool that is not in tool_access.allow:

$ chio --session-db ./admission-3.db --receipt-db ./receipts.db \
  check --policy ./policy.yaml --tool delete_file \
  --params '{"path": "/workspace/out.txt"}' \
  --mode full --output-fixture ./output-fixture.json

verdict:    DENY
tool:       delete_file
server:     *
reason:     requested tool delete_file on server * is not in capability scope
receipt_id: 0e570662187463441276e407bcb5e46c5b049d7cf59c614756c2acb5431149ee
policy:     89e8cd101e3fb0c311cceb9bd7b988a6a65d371310b4fccceb08a015333885da
source:     11e1aa04a3696c42034ae77afb4f9aee4e5c16c4e38a58f2a21fc2fa137779d8
mode:       full
fixture:    true

WARN chio_kernel::kernel::evaluation::async_evaluation_core message=capability rejected request_id=check-001 reason=requested tool delete_file on server * is not in capability scope

The allow case proves the policy is not so strict it blocks legitimate use. The two deny cases separate the two ways a call dies: case 2 reached the guard pipeline and a guard refused it, case 3 never reached a guard because tool_access.allow compiles to the capability scope and delete_file is not in it. The stderr line above each verdict says which: guard denied against capability rejected. Both print exit 2 and both write a signed receipt. Write a Policy builds this checklist by hand; Testing in CI below turns it into a script.

What a dry run cannot decide

One guard answers differently under chio check than it will in a running node, and the canonical policy enables it. PathAllowlistGuard gates on the session's enforceable filesystem roots before it consults its own allowlist, and a root set that is present but empty matches nothing. A chio check session declares no roots, so every path-bearing call denies whatever the allowlist says, including the read the policy is written to permit:

$ chio --session-db ./admission-4.db --receipt-db ./receipts.db \
  check --policy ./policy.yaml --tool read_file \
  --params '{"path": "/workspace/README.md"}' \
  --mode full --output-fixture ./output-fixture.json

verdict:    DENY
tool:       read_file
server:     *
reason:     guard denied the request: guard "guard-pipeline" denied the request
receipt_id: fbe57d052bcb6ad1b79b0f8ff2d5613b4203c9f8cc31fe4ca75b8443b82075dc
policy:     89e8cd101e3fb0c311cceb9bd7b988a6a65d371310b4fccceb08a015333885da
source:     11e1aa04a3696c42034ae77afb4f9aee4e5c16c4e38a58f2a21fc2fa137779d8
mode:       full
fixture:    true

WARN chio_kernel::kernel::evaluation::async_evaluation_core message=guard denied request_id=check-001 reason=guard denied the request: guard \"guard-pipeline\" denied the request

chio check is therefore the wrong instrument for rules.path_allowlist. Test that rule against a session that declares roots, through Filesystem Guards or a native-guard unit test built with an explicit session_filesystem_roots, as the native guard example below does. Every other rule on the canonical policy decides the same way in both places.

Read the denial back

The cases shared one receipt database, so the record of the refusal is already on disk. chio receipt explain takes the receipt_id printed by case 2, and chio receipt list is the way to find it again without scrolling back:

$ DENY=$(chio --receipt-db ./receipts.db receipt list --admin-all \
    | jq -r 'select(.action.parameters.path == "/workspace/.env") | .id')
$ chio --receipt-db ./receipts.db receipt explain "$DENY" --admin-all

receipt: 0b8dd0573f189324215246c2b4509ee6f867724000d795bbc0f66b65cc016c33
schema: chio.receipt.v1
identity: 0b8dd0573f189324215246c2b4509ee6f867724000d795bbc0f66b65cc016c33
decision: deny
reason: guard denied the request: guard "guard-pipeline" denied the request
guard: kernel
policy_hash: 89e8cd101e3fb0c311cceb9bd7b988a6a65d371310b4fccceb08a015333885da
scope_diff: requested scope vs granted scope is not embedded in this receipt
parents: 0
repair_hint: inspect the guard and policy_hash, then mint or narrow a matching capability

chio receipt list prints the whole record, one JSON receipt per line. The deny receipt for case 2 carries the parameters that were refused, their hash, the policy hash, an evidence row naming the guard, and an Ed25519 signature over the canonical body. The del(.metadata) below drops the four metadata keys, which are attribution, budget authority, the receipt signing nonce, and receipt_context.request_id:

$ chio --receipt-db ./receipts.db receipt list --admin-all \
    | jq 'select(.action.parameters.path == "/workspace/.env") | del(.metadata)'

{
  "id": "0b8dd0573f189324215246c2b4509ee6f867724000d795bbc0f66b65cc016c33",
  "timestamp": 1788551990,
  "capability_id": "cap-01a06e01-8ae8-7fd0-a7b0-b3593f7a666f",
  "tool_server": "*",
  "tool_name": "read_file",
  "action": {
    "parameters": {
      "path": "/workspace/.env"
    },
    "parameter_hash": "1708328df9dddb4147395f5741ee7df3cf5dc4922554c08feb6a73bc0d82f245"
  },
  "decision": {
    "verdict": "deny",
    "reason": "guard denied the request: guard \"guard-pipeline\" denied the request",
    "guard": "kernel"
  },
  "receipt_kind": "mediated_decision",
  "boundary_class": "prevent",
  "tool_origin": "caller_executed",
  "redaction_mode": "none",
  "content_hash": "74234e98afe7498fb5daf1f36ac2d78acc339464f950703b8c019892f982b90b",
  "policy_hash": "89e8cd101e3fb0c311cceb9bd7b988a6a65d371310b4fccceb08a015333885da",
  "evidence": [
    {
      "guard_name": "forbidden-path",
      "verdict": false,
      "details": "action=deny; reason=guard denied request"
    }
  ],
  "trust_level": "mediated",
  "kernel_key": "722344ad01d71706e495a5049569bf3a5dbc419f5a95920379829a206f762553",
  "signature": "4e1d438f45b249195039733149f1e9b027dc7e5fd5a98d275b424f585c147832bdf98d0d01c357a947ab600ecf1ec0d95ced5a4e1ccb5052e2ae365d5e150f00"
}

evidence[0].guard_name is the only place on this receipt that names forbidden-path, which is why a test that asserts on the reason string alone cannot tell which guard fired. decision.guard carries the fixed kernel-scope literal kernel on this path, not the denying guard.

Rerunning the cases reproduces every line above except receipt_id, capability_id, timestamp, the keys, and the signature. The receipt id is a hash over a body that carries a fresh timestamp and signing nonce, so it is new on every run. The three content hashes are not: policy, source and parameter_hash are functions of the policy file and the arguments, so they are the assertion a test should make.

HushSpec evaluation paths

HushSpec exposes two independent ways to ask what a policy would do with one action, and they are not interchangeable. Confusing them is the most common mistake in tooling built on Chio.

The evaluate() interpreter

chio_policy::evaluate is a pure-function reference interpreter. It walks a resolved HushSpec directly against one EvaluationAction (a tool call, egress, a file read or write, a patch, a shell command, computer-use input, or input injection) and returns an EvaluationResult. It never touches a kernel session or writes a receipt to a store, and its decision has three states, not two:

pub enum Decision {
    Allow,
    Warn,
    Deny,
}

pub struct EvaluationResult {
    pub decision: Decision,
    pub matched_rule: Option<String>,
    pub reason: Option<String>,
    pub origin_profile: Option<String>,
    pub posture: Option<PostureResult>,
}

Warn is not a softer deny. It fires when a preference-level rule is not met but nothing required it: a tool_access.prefer_runtime_assurance_tier the caller falls short of produces Warn with the reason runtime assurance tier '{actual}' is below preferred '{preferred}'; the equivalent require_runtime_assurance_tier produces a hard Deny. Call the interpreter directly from a test to see the difference:

use chio_policy::{evaluate, Decision, EvaluationAction};

#[test]
fn preferred_tier_shortfall_warns_not_denies() {
    let spec = policy_with_preferred_tier(RuntimeAssuranceTier::Attested);
    let result = evaluate(
        &spec,
        &EvaluationAction {
            action_type: "tool_call".to_string(),
            target: Some("payments.charge".to_string()),
            ..Default::default()
        },
    );

    assert_eq!(result.decision, Decision::Warn);
}

evaluate_with_context(spec, action, context, conditions) is the same interpreter with conditional activation applied first. evaluate_audited(spec, action, config) wraps either path with timing and a SHA-256 policy hash into a DecisionReceipt, still entirely in-memory, no store, no kernel. For fast, side-effect-free answers about what a policy would do, call this API directly from Rust instead of invoking chio check in a loop. chio-policy itself leans on this property: its own property-test suite drives evaluate() through invariants like deny_overrides_warn_and_allow, scaled by a PROPTEST_CASES environment variable (256 cases per pull request, 4096 nightly), a reasonable model if you build logic on top of it.

Testing custom native guards

A native guard implements the synchronous Guard trait directly in Rust instead of compiling to WASM. It is small enough to unit-test with no kernel, policy file, or receipt store at all:

pub trait Guard: Send + Sync {
    fn name(&self) -> &str;
    fn evaluate(&self, ctx: &GuardContext<'_>) -> Result<Verdict, crate::KernelCoreError>;
}

GuardContext carries a &PortableToolCallRequest (request_id, tool_name, server_id, agent_id, arguments, plain fields with no signature required), a &ChioScope (ChioScope::default() is enough for scope-blind guards), denormalized agent_id / server_id strings, and two optional fields the kernel populates before guards run: session_filesystem_roots and matched_grant_index. Construct one directly, no kernel required:

use chio_kernel_core::{Guard, GuardContext, PortableToolCallRequest, Verdict};
use chio_core_types::capability::scope::ChioScope;

#[test]
fn denies_paths_outside_the_workspace_root() {
    let request = PortableToolCallRequest {
        request_id: "req-1".to_string(),
        tool_name: "read_file".to_string(),
        server_id: "srv-files".to_string(),
        agent_id: "agent-1".to_string(),
        arguments: serde_json::json!({ "path": "/etc/passwd" }),
    };
    let scope = ChioScope::default();
    let ctx = GuardContext {
        request: &request,
        scope: &scope,
        agent_id: "agent-1",
        server_id: "srv-files",
        session_filesystem_roots: Some(&["/workspace".to_string()]),
        matched_grant_index: None,
    };

    let guard = WorkspaceRootGuard::new(vec!["/workspace".into()]);
    assert_eq!(guard.evaluate(&ctx).unwrap(), Verdict::Deny);
}

In-tree guards register against the fuller chio_kernel::Guard, signature-compatible but backed by the complete ToolCallRequest (a signed CapabilityToken, DPoP proof, and other kernel-only fields the portable trait omits). crates/guards/chio-data-guards/tests/warehouse_cost_guard.rs provides an example: it mints a throwaway keypair, signs a capability, and evaluates the guard.

Testing WASM guards

WASM guards run through the same chio guard lifecycle described in Custom WASM Guards (new, build, inspect). Testing sits between build and publish, with two dedicated subcommands: chio guard test --wasm <path> <fixtures...> --fuel-limit <n> replays YAML fixtures, and chio guard bench <path> --iterations <n> --fuel-limit <n> measures latency and fuel across warmup and timed runs.

Fixture format

Each fixture file is a YAML list. Every case runs against a fresh Wasmtime backend, so fuel and memory never leak between fixtures:

- name: allows a tool that is not on the denylist
  request:
    tool_name: read_file
    server_id: srv-files
    agent_id: agent-1
    arguments:
      path: "./workspace/README.md"
  expected_verdict: allow

- name: denies a tool on the denylist
  request:
    tool_name: delete_file
    server_id: srv-files
    agent_id: agent-1
    arguments:
      path: "./workspace/output.txt"
  expected_verdict: deny
  deny_reason_contains: "denylist"

Run those two fixtures against the guard built in Worked Example: Tool Denylist Guard:

$ chio guard test \
    --wasm target/wasm32-unknown-unknown/release/tool_denylist_guard.wasm \
    ./fixtures/tool-denylist.yaml --fuel-limit 1000000

[PASS] allows a tool that is not on the denylist
[PASS] denies a tool on the denylist

2 passed, 0 failed out of 2 total

Any failed fixture makes chio guard test return a nonzero exit, so a CI step can fail on an unexpected fixture result. In chio check, a deny is often the expected outcome.

What a failing fixture prints

The failure line carries the guard's own refusal. A fixture that asserts expected_verdict: allow for a tool the denylist covers gets the deny reason back verbatim, which is what makes the fixture suite a regression test on the wording as well as the verdict:

- name: wipe_database is still allowed
  request:
    tool_name: wipe_database
    server_id: srv-db
    agent_id: agent-1
    arguments: {}
  expected_verdict: allow

$ chio guard test \
    --wasm target/wasm32-unknown-unknown/release/tool_denylist_guard.wasm \
    ./fixtures/wrong.yaml --fuel-limit 1000000

[FAIL] wipe_database is still allowed: expected allow, got deny: tool 'wipe_database' is on the denylist

0 passed, 1 failed out of 1 total

error [urn:chio:error:guard:denied]: 1 test(s) failed
context: {"domain":"guard","severity":"error","stability":"stable","string_code":"CHIO-KERNEL-GUARD-DENIED"}
suggested fix: Inspect the guard verdict and adjust the prompt, tool input, policy, or output before retrying.

A fixture that never reaches a verdict reads differently: the line says evaluation error rather than naming a verdict, because the runner reports the backend error in place of a comparison. Fuel exhaustion arrives that way, and dropping --fuel-limit far enough stops the guard inside its own request deserialization, before any policy logic runs. The fuel a guard actually spends, and the deserialization floor underneath it, are measured on Custom WASM Guards.

chio guard bench is the companion for latency and fuel regressions: five warmup iterations against a fixed bench_tool request, then the requested count, reporting p50, p99, min, max and mean for both latency and fuel. Every iteration gets its own backend, so the numbers include module load. Its output is on Custom WASM Guards.

Unit-test guard logic

Put the decision logic in a plain helper called by the macro-annotated function, then unit-test that helper directly. This avoids Wasmtime, fuel metering, and a linear-memory round trip:

use chio_guard_sdk::prelude::*;
use chio_guard_sdk_macros::chio_guard;

fn decide(req: &GuardRequest) -> GuardVerdict {
    if req.tool_name == "dangerous_tool" {
        GuardVerdict::deny("tool is blocked by policy")
    } else {
        GuardVerdict::allow()
    }
}

#[chio_guard]
fn evaluate(req: GuardRequest) -> GuardVerdict {
    decide(&req)
}

#[cfg(test)]
mod tests {
    use super::decide;
    use chio_guard_sdk::prelude::*;

    #[test]
    fn denies_the_dangerous_tool() {
        let req = GuardRequest {
            tool_name: "dangerous_tool".to_string(),
            server_id: "srv".to_string(),
            agent_id: "agent-1".to_string(),
            arguments: serde_json::json!({}),
            ..Default::default()
        };
        assert!(decide(&req).is_deny());
    }
}

Keep both layers: fast native tests against decide() for iteration, and chio guard test fixtures for the slower integration test that the compiled .wasm module exports the right ABI and behaves the same under fuel metering.

Testing in CI

The building blocks above are individually scriptable; the remaining work is deciding what a passing pipeline run means.

Golden deny/allow fixtures. Because a deny is often the correct, expected result, a CI step cannot simply run chio check and fail on any non-zero exit. Keep a small table of expected outcomes and compare against it instead:

#!/usr/bin/env bash
set -euo pipefail

# name|tool|params|expected_exit  (params contain colons, so do not split on ':')
cases=(
  'shell-list|run_command|{"command": "ls -la /workspace"}|0'
  'env-forbidden|read_file|{"path": "/workspace/.env"}|2'
  'delete-not-granted|delete_file|{"path": "/workspace/out.txt"}|2'
)

state=$(mktemp -d)
failures=0
i=0
for case in "${cases[@]}"; do
  IFS='|' read -r name tool params expected <<< "$case"
  i=$((i + 1))
  set +e
  chio --session-db "$state/admission-$i.db" --receipt-db ./ci-receipts.db \
    check --policy ./policy.yaml --tool "$tool" --params "$params" \
    --mode full --output-fixture ./output-fixture.json > /dev/null 2>&1
  actual=$?
  set -e
  if [ "$actual" -ne "$expected" ]; then
    echo "FAIL $name: expected exit $expected, got $actual"
    failures=$((failures + 1))
  else
    echo "PASS $name"
  fi
done

[ "$failures" -eq 0 ]

$ ./scripts/check-policy.sh

PASS shell-list
PASS env-forbidden
PASS delete-not-granted

The case table is the assertion. Change env-forbidden to expect 0 and the script reports the guard doing its job as a table error, which is the signal you want when a policy edit quietly stops denying something:

$ sed -i "/env-forbidden/s/|2'/|0'/" scripts/check-policy.sh
$ ./scripts/check-policy.sh

PASS shell-list
FAIL env-forbidden: expected exit 0, got 2
PASS delete-not-granted

Two details keep the script honest. The delimiter is | rather than : because the JSON in --params contains colons, and a split on : puts {"path" in $params and the rest of the line in $expected, where the integer comparison errors out and every case reports PASS. And each case gets its own admission database under one throwaway directory, because the fixed request id check-001 collides with the retained operation from the previous case otherwise.

WASM guard fixtures. chio guard test already exits non-zero on failure, so it works as a CI step once chio is on PATH:

steps:
  - name: Golden allow/deny fixtures
    run: ./scripts/check-policy.sh
  - name: WASM guard fixtures
    run: |
      chio guard build
      chio guard test --wasm target/wasm32-unknown-unknown/release/*.wasm \
        ./fixtures/*.yaml

Next steps

• HushSpec Policy Format · the schema and compile model behind chio check
• chio.yaml Configuration · guards.required and advisory promotion
• Custom WASM Guards · manifest format, signing, hot reload, and canaries
• Custom Guards · the guide-form walkthrough, including sign/verify, install, blocklist
• Node Observability · what to watch once a tested policy is running