Mutation Testing

A mutant asks whether a test notices one changed decision

An edit replaces a Boolean result, changes a comparison, or removes a condition; the tool builds the changed program and runs the configured tests against it. In the credentials crate, one listed candidate replaces the schema check with true:

fn is_supported_passport_schema(
    schema: &str,
) -> bool {
-    schema == PASSPORT_SCHEMA
+    true
}

The mutant asks one thing: whether any test fails when this helper accepts an unsupported schema. A failing test catches the mutant, and the failure shows whether the intended assertion noticed it. A passing suite leaves a survivor, a review task for a missing assertion, unexercised behavior, or an equivalent edit. A catch shows test sensitivity to that edit, whether or not the edit resembles a realistic defect. The original helper compares the supplied schema name with PASSPORT_SCHEMA.

The generated list is the mutation set

The workspace configuration selects files in chio-kernel-core, chio-policy, chio-guards, chio-credentials, chio-attest-verify, and chio-anchor: decision, guard, credential, attestation, and anchoring code. An examine_globs entry states selection intent, and the list shows what the tool found. cargo-mutants generates candidates from functions declared in the files it parses, so code pulled in with include! produces none.

For chio-kernel-core alone, cargo-mutants 25.3.1 lists hundreds of candidates; listing parses them and runs no tests. The count follows the source, the configuration, and the tool version, which is why the baseline below records all three. Maintainers list and run candidates from source as Qualification: formal gates describes.

Read the workspace selection, the relevant per-crate configuration, and the generated list together; the configuration documentation describes explicit config selection, and behavior follows the tool version. A rationale checker requires an explanation for each configured exclusion, which leaves discovery and the testing of excluded behavior to be read from the list and the tests.

The committed baseline catches 115 of 375 scoreable mutants

The baseline, generated April 29, 2026 with cargo-mutants 25.3.1, aggregates complete per-crate sweeps and bounded shards across the six crates. It lists 2,369 candidates and evaluates 442 of them.

The denominator is 442 − 67 = 375, so the catch ratio is 115 ÷ 375 ≈ 30.7 percent against an 80 percent per-crate release floor. The ratio measures sensitivity to the evaluated edits under their selected tests. Code the tool did not discover and the separate proof-mutation campaigns sit outside it.

The release gate blocks once its activation evidence records two consecutive green nightly full sweeps and measured per-crate catch ratios; it then compares each crate with the threshold and refuses missing outcomes even when the tool exits zero, while zero scoreable outcomes with a zero exit pass. Before activation it reports advisory success. The gate script also documents its missing-data handling and explicit overrides.

generated_at = "2026-04-29"
tool = "cargo-mutants"
tool_version = "25.3.1"

[aggregate]
scope = "six-crate trust-boundary mutation baseline"
crate_entries = 6
listed_mutants_total = 2369
evaluated_mutants_total = 442
caught_total = 115
missed_total = 259
unviable_total = 67
timeout_total = 1
measured_kill_rate_excluding_unviable = 30.7
baseline_status = "aggregate complete from existing per-crate baselines; mixed full sweeps and bounded shards"

[mutants]
# The release floor is 80 percent per crate. Proof and specification mutation
# lanes use rotating samples and remain advisory until a full-inventory
# coverage ratchet is promoted.
initial_merge_tag = "v0.0.0-mutation-gate"
target_catch_ratio_percent = 80
activation_threshold_percent_per_crate = 80
required_consecutive_nightly_successes = 2
observed_consecutive_nightly_successes = 0
activation_evidence = """
status: advisory
evidence_pending: true
gate_mode: advisory
observed_consecutive_nightly_successes: 0
required_consecutive_nightly_successes: 2
nightly_runs: []
per_crate_kill_rate_percent:
  chio-policy: pending_full_sweep
  chio-credentials: pending_full_sweep
  chio-attest-verify: pending_full_sweep
  chio-kernel-core: pending_full_sweep
  chio-guards: pending_full_sweep
  chio-anchor: pending_full_sweep
note: "Blocking activation requires confirmed green mutants-nightly full sweeps and measured per-crate kill rates. Until then cycle_end_tag stays empty."
"""
cycle_end_tag = ""
pr_survivor_issue_budget = 5
nightly_wall_budget_hours_per_crate = 4
trust_boundary_crates = [
    "chio-policy",
    "chio-credentials",
    "chio-attest-verify",
    "chio-kernel-core",
    "chio-guards",
    "chio-anchor",
]

Pull requests mutate the diff, and the nightly sweep mutates the package

• Pull request. A path-selected package with its explicit per-crate config, a merge-base diff, and --test-workspace false, at --timeout 300 with two jobs. Shared resource-budget setup can stop the run.
• Scheduled or manual sweep. The selected package with the workspace-root config and no diff filter, which adds workspace tests excluding chio-cpp-kernel-ffi, at --timeout 600 with two jobs inside a four-hour per-package job limit. The sweep is advisory.

The workspace timeout multiplier and minimum apply when the tool derives a timeout, and an explicit --timeout replaces that derivation. A per-crate config loads only when --config names it. The PR gate step preserves its outcome so triage output still runs, and a later enforcement step fails the job on a failed gate; branch protection and hosted execution are configured outside this file.

A full campaign for one crate takes substantial time, so the workflows establish a stable unmodified test run first. A diff-scoped run mutates only the changed lines, and an empty diff mutates nothing. The mutation baseline travels with each release in its threat-coverage report, which chio release verify --evidence threat-coverage checks for the release you run.

on:
  # PR runs are scoped to trust-boundary source and mutation-control changes.
  # Untouched matrix packages stop before installing Rust or cargo-mutants.
  pull_request:
    paths:
      - ".cargo/mutants.toml"
      - ".github/workflows/mutants.yml"
      - "crates/kernel/chio-kernel-core/mutants.toml"
      - "crates/kernel/chio-kernel-core/src/**"
      - "crates/guards/chio-guards/mutants.toml"
      - "crates/guards/chio-guards/src/**"
      - "crates/guards/chio-policy/mutants.toml"
      - "crates/guards/chio-policy/src/**"
      - "crates/trust/chio-attest-verify/mutants.toml"
      - "crates/trust/chio-attest-verify/src/**"
      - "crates/trust/chio-credentials/mutants.toml"
      - "crates/trust/chio-credentials/src/**"
      - "crates/economy/chio-anchor/mutants.toml"
      - "crates/economy/chio-anchor/src/**"
      - "releases.toml"
      - "scripts/check-fuzz-budget.sh"
      - "scripts/check-mutants-rationale.sh"
      - "scripts/mutants-*.sh"
  schedule:
    # 05:00 UTC nightly. Sits after fuzz.yml (03:23) and the existing
    # nightly proptest lane (04:23) so the GHA runner pool is not
    # over-subscribed.
    - cron: "0 5 * * *"
  workflow_dispatch:
    inputs:
      package:
        description: "Force a single crate (default: full matrix)"
        required: false
        type: string

A survivor becomes a test decision, and a fuzz corpus can replay it

Reproduce the edit and read the assertion that should have noticed it.

1. Behavior changed and matters. Add a test that fails for the mutant and passes for the original.
2. The edit is equivalent for the contract. Record why, alongside the configured exclusion and its rationale.
3. The run timed out or failed in setup. Repair that execution before assigning a semantic verdict.

The PR workflow keeps a summary and, for same-repository requests, files survivors beyond its inline limit as issues for triage. The cross-fuzz script replays a fuzz corpus against each surviving mutant, choosing the corpus through a filename-to-target mapping, and writes a replay log per survivor. Read that log with the result: a corpus failure on a mutant is a catch by a second oracle, while a setup failure or an unmapped file leaves the survivor a test decision.

The same question applies to the policy you write

A policy test table can pass while a rule it claims to cover does nothing. chio policy test --mutate asks the mutation question of your policy: it weakens one rule at a time (drops a forbidden path, widens a grant, raises a ceiling, removes an approval), reruns the case table against each mutant, and reports every weakening no case noticed. It runs on the pricing project's policy.yaml and the case table from Testing Guards & Policies:

chio policy test --mutate

policy   ./policy.yaml
baseline 3 cases pass
mutants  6 generated: 5 caught, 1 missed
MISSED   forbidden-path  removed **/.env
         the only .env case reads /secrets/.env, which the grant refuses before the guard runs
6 mutants: 5 caught, 1 missed

The survivor is real. fixer reads /secrets/.env expects a deny and gets one, but the workspace grant already refuses the path, so the forbidden-path guard never decides it. Add a case the guard alone decides, a read of /workspace/.env, which the grant admits and **/.env refuses, and the mutant is caught. The command exits 1 while any mutant survives, so the same CI step that runs the table holds the line on what the table proves.

Next: run the assembled kernel against generated adversaries