Node State on Disk

Sidecar layout on disk

A sidecar started with chio api protect --receipt-store /var/lib/chio/receipts.db produces this layout.

Provisioning creates every authority schema in authority.db (budget, revocation, FROST, the economic state cache, fiscal, and the rest), and the sidecar wires two of them; its revocation list stays in the sibling file. The same authority store opened by DurableAdmissionRuntime::open in chio-control-plane puts its lock root at <path>.locks and points the budget and revocation stores at the authority file.

Receipt writer and reader pools

The receipt store keeps two pools in chio-store-sqlite/src/lib.rs: DEFAULT_READER_POOL_MAX_SIZE = 8 and DEFAULT_WRITER_POOL_MAX_SIZE = 1. Every write runs on the writer through the commit actor's run_write, including session anchors, request lineage, lineage statements, child receipts, and manual checkpoint creation. Both pools open the file read-write; routing keeps the readers read-only, and reader_pool_never_begins_a_write_transaction forces PRAGMA query_only = ON on all eight reader connections, then drives the whole routed write path to completion.

• RECEIPT_GROUP_COMMIT_MAX_BATCH = 64. The most appends the actor drains into one transaction.
• RECEIPT_GROUP_COMMIT_FLUSH_DELAY. 500 microseconds, the longest the actor waits for each next append.
• Channel depth. A bounded channel sixteen times the batch deep (1024); a full channel returns ReceiptStoreError::Pool("sqlite receipt commit queue saturated").

The deadline variants append_chio_receipt_with_timeout and run_write_receipt_with_timeout fail closed with ReceiptStoreError::Timeout { operation, timeout_ms }. That is a caller-side deadline only: the command is already on the channel, so the actor still owns it and may still commit it, and expiry leaves the in-flight counter elevated for the liveness probe to read.

At start the actor seeds a verified head once, running full claim-log validation and a full checkpoint-chain verification on the writer connection. Each later append verifies only the delta: one indexed latest-checkpoint read plus a digest compare, and an aggregate over rows past the head. Writer health reports a poisoned head until seeding succeeds. SqliteStoreOptions::incremental_verification is fixed at open; set false, it keeps full per-append verification for A/B checks on a suspect database. The authority database has no pool: one connection, one mutex, cloned into every logical store.

Provisioning

SqliteAuthorityStore::provision runs before anything serves. A missing path is created with create_new, mode 0600, and O_NOFOLLOW. An existing path is adopted only when it is a regular file, mode exactly 0600, owned by the effective user, with link count one, and only when its authority tables are pristine. The database's parent directory and the lock root must be owned by the effective user and neither group- nor world-writable.

It mints a canonical UUID-v7 store id, creates exactly one <store_uuid>.lock inode in the lock root (mode 0600, link count one, the lock root's own uid and gid), and records the device and inode of both files. The owner table and its single row are created in one IMMEDIATE transaction, so an interrupted provision rolls both back and leaves a recoverable database. Files and both parent directories are fsynced. A repeat provision is idempotent only when the UUID and inode metadata match exactly. It runs from DurableAdmissionRuntime::open and from the sidecar's startup path.

Opening and the epoch claim

open_serving canonicalizes the path, takes an advisory lock on the lock root, re-reads the provisioning record, and refuses a database whose path, device, inode, mode, or link count no longer matches; symlink and hardlink aliases are refused. It then takes try_lock on the lock inode for the serving lifetime and re-validates the record with the lock held, so a race cannot slip a different file underneath.

In one IMMEDIATE transaction it closes the previous lease row at the current admission-authority head, increments owner_epoch, writes a fresh lease_id, and inserts the new open lease. Lease history in chio_serving_leases is append-and-close-only: triggers ABORT any delete and any update other than the one legal transition from open to closed. A startup check requires the leases to form a dense chain of epochs whose authority intervals tile without gaps.

The rollback anchor in the lock file has two slots recording commit heads and chain digests. The path-identity marker is local continuity evidence: restoring or deleting the lock root removes the marker and the anchor together. Serving ownership requires Unix file identity; metadata_device and metadata_inode return Invalid on other targets.

Open refusals

SqliteRevocationStore::open on a provisioned authority file reports that the store requires the joint serving owner at its current epoch, and writes nothing.

Schema stamp refusals

/// ASCII "CHIO" as a big-endian `i32`, stamped into every Chio operator store.
pub const CHIO_SQLITE_APPLICATION_ID: i32 = 0x4348_494f;

/// Table holding each co-located store's schema revision, keyed by a stable
/// per-store identifier. `PRAGMA user_version` is database-wide and so cannot
/// give the receipt and approval stores (which the sidecar keeps in one file)
/// independent revisions; a keyed table can.
const SCHEMA_VERSION_TABLE: &str = "chio_store_schema_versions";

/// Failure to validate or apply a store's schema stamp.
#[derive(Debug, thiserror::Error)]
pub enum SchemaVersionError {
    #[error("sqlite error: {0}")]
    Sqlite(#[from] rusqlite::Error),
    #[error("database application_id {found:#x} is not a Chio store (expected {expected:#x})")]
    ForeignDatabase { found: i32, expected: i32 },
    #[error(
        "database carries the Chio application_id but none of this store's tables ({expected_anchors:?}); refusing to open another store's database"
    )]
    MismatchedStore { expected_anchors: Vec<String> },
    #[error(
        "database schema version {found} is newer than this binary supports ({supported}); refusing to open"
    )]
    FutureSchema { found: i32, supported: i32 },
}

The revision lives in chio_store_schema_versions under a stable key per store, because PRAGMA user_version is database-wide and co-located stores need independent revisions. The receipt store and the approval store in one sidecar file each sit at their own revision.

Anchors identify the store: the receipt store answers to chio_tool_receipts and to the table names http_receipts and tool_receipts; the joint authority open passes capability_grant_budgets and revoked_capabilities. Because the two shorter receipt names are generic, an unstamped file adopted on either also needs a receipt payload column (raw_json or receipt_json).

A database with no user tables is adoptable, so a fresh file and an unstamped one both open. The application_id is read first, so a file stamped by another application is ForeignDatabase even with no tables. A refused foreign database keeps application_id 0, a future-schema database keeps its revision, and a table-driven test sweeps binary revisions 0 to 3 against on-disk revisions 0 to 5, asserting that a refusal never mutates the file and an accepted open never downgrades it. The gate runs on the open path only, so appends pay nothing for it.

Pragmas per store

All durable stores set WAL, synchronous = FULL, and busy_timeout = 5000. The serving owner, revocation, budget, approval, and receipt stores set foreign_keys = ON; the capability authority and execution-nonce stores do not, so neither file enforces referential integrity. The receipt store adds auto_vacuum = INCREMENTAL and is the store that reads its settings back: journal_mode must be WAL, synchronous 2, busy_timeout at least 5000, and foreign_keys 1, or the open is refused, so a filesystem that declines WAL stops the process at startup.

let journal_mode: String = connection.query_row("PRAGMA journal_mode", [], |row| row.get(0))?;
if !journal_mode.eq_ignore_ascii_case("wal") {
    return Err(ReceiptStoreError::Conflict(format!(
        "sqlite receipt store journal_mode must be WAL, got {journal_mode}"
    )));
}

is_in_memory_sqlite_path recognizes :memory:, file::memory:, and any file:...?mode=memory URI as non-durable.

The ReceiptStore trait

Any backend that implements ReceiptStore can persist receipts. Two methods are required and the rest carry defaults, so a backend overrides only what it persists. The block shows five methods; the rest of the trait (receipt_store.rs:418-909) covers timeout-aware appends, canonical-bytes reads, pending settlement observations, capability lineage, federated evidence, and retention hooks.

pub trait ReceiptStore: Send + Sync {
    fn append_chio_receipt(&self, receipt: &ChioReceipt) -> Result<(), ReceiptStoreError>;
    // ...
    fn append_chio_receipt_returning_seq(
        &self,
        receipt: &ChioReceipt,
    ) -> Result<Option<u64>, ReceiptStoreError> {
        self.append_chio_receipt(receipt)?;
        Ok(None)
    }
    // ...
    fn append_child_receipt(&self, receipt: &ChildRequestReceipt) -> Result<(), ReceiptStoreError>;
    // ...
    fn store_checkpoint(&self, _checkpoint: &KernelCheckpoint) -> Result<(), ReceiptStoreError> {
        Err(ReceiptStoreError::Conflict(
            "receipt checkpoint storage is not supported by this receipt store".to_string(),
        ))
    }
    // ...
    fn load_checkpoint_by_seq(
        &self,
        _checkpoint_seq: u64,
    ) -> Result<Option<KernelCheckpoint>, ReceiptStoreError> {
        Ok(None)
    }
    // ...
}

The checkpoint defaults are behavior: a store without checkpointing refuses to store one with “receipt checkpoint storage is not supported by this receipt store” and reports none on read, so checkpoint_batch_size and store capability must agree before a deployment claims a Merkle chain. The kernel queries receipts through ReceiptQuery:

pub struct ReceiptQuery {
    /// Filter by capability ID (exact match).
    pub capability_id: Option<String>,
    /// Filter by tool server name (exact match).
    pub tool_server: Option<String>,
    /// Filter by tool name (exact match).
    pub tool_name: Option<String>,
    /// Filter by decision outcome (maps to decision_kind column:
    /// "allow", "deny", "cancelled", "incomplete").
    pub outcome: Option<String>,
    /// Include only receipts with timestamp >= since (Unix seconds, inclusive).
    pub since: Option<u64>,
    /// Include only receipts with timestamp <= until (Unix seconds, inclusive).
    pub until: Option<u64>,
    /// Include only receipts with financial cost_charged >= min_cost (minor units).
    /// Receipts without financial metadata are excluded when this filter is set.
    pub min_cost: Option<u64>,
    /// Include only receipts with financial cost_charged <= max_cost (minor units).
    /// Receipts without financial metadata are excluded when this filter is set.
    pub max_cost: Option<u64>,
    /// Currency for cost filters. Required when either cost bound is present.
    pub cost_currency: Option<String>,
    /// Cursor for forward pagination: return only receipts with seq > cursor (exclusive).
    pub cursor: Option<u64>,
    /// Maximum number of receipts to return per page (capped at MAX_QUERY_LIMIT).
    pub limit: usize,
    /// Filter by agent subject public key (hex-encoded Ed25519). Resolved through
    /// capability_lineage JOIN -- does not replay issuance logs.
    pub agent_subject: Option<String>,
    /// Optional tenant narrowing filter. This is never authority by itself:
    /// callers must also provide a matching explicit `read_context`.
    pub tenant_filter: Option<String>,
    /// Explicit read context resolved from authenticated authority.
    pub read_context: Option<ReceiptReadContext>,
}

limit is capped at MAX_QUERY_LIMIT, which is 200. cost_currency is required whenever either cost bound is set, and receipts without financial metadata drop out of a cost-filtered result. tenant_filter narrows; the explicit read_context authorizes and shapes the query with a boundary (AdminAll or TenantScoped), a source (LocalOperator, AdminService, or AuthenticatedTenant), and an include_null_tenant flag. Remote and control-plane callers build it from authenticated context before the query reaches the store; see multi-tenant isolation.

SqliteReceiptStore schema

The reference implementation in chio-store-sqlite reads through the pool and writes through the commit actor. The tool receipt table:

CREATE TABLE IF NOT EXISTS chio_tool_receipts (
    seq INTEGER PRIMARY KEY AUTOINCREMENT,
    receipt_id TEXT NOT NULL UNIQUE,
    timestamp INTEGER NOT NULL,
    capability_id TEXT NOT NULL,
    subject_key TEXT,
    issuer_key TEXT,
    grant_index INTEGER,
    tool_server TEXT NOT NULL,
    tool_name TEXT NOT NULL,
    decision_kind TEXT NOT NULL,
    policy_hash TEXT NOT NULL,
    content_hash TEXT NOT NULL,
    raw_json TEXT NOT NULL,
    cost_currency TEXT CHECK (
        cost_currency IS NULL OR (
            typeof(cost_currency) = 'text' AND
            length(cost_currency) = 3 AND
            cost_currency NOT GLOB '*[^A-Z]*'
        )
    ),
    cost_charged_be BLOB CHECK (
        (cost_currency IS NULL AND cost_charged_be IS NULL) OR
        (
            cost_currency IS NOT NULL AND
            typeof(cost_charged_be) = 'blob' AND
            length(cost_charged_be) = 8
        )
    )
);

CREATE INDEX IF NOT EXISTS idx_chio_tool_receipts_timestamp
    ON chio_tool_receipts(timestamp);
CREATE INDEX IF NOT EXISTS idx_chio_tool_receipts_capability
    ON chio_tool_receipts(capability_id);
CREATE INDEX IF NOT EXISTS idx_chio_tool_receipts_subject
    ON chio_tool_receipts(subject_key);
CREATE INDEX IF NOT EXISTS idx_chio_tool_receipts_grant
    ON chio_tool_receipts(capability_id, grant_index);
CREATE INDEX IF NOT EXISTS idx_chio_tool_receipts_tool
    ON chio_tool_receipts(tool_server, tool_name);
CREATE INDEX IF NOT EXISTS idx_chio_tool_receipts_decision
    ON chio_tool_receipts(decision_kind);

A migration adds tenant_id with ALTER TABLE chio_tool_receipts ADD COLUMN tenant_id TEXT and an index beside it, so the column is nullable and rows written before multi-tenancy read back with a NULL tenant. cost_currency must be three uppercase letters or NULL, and cost_charged_be an 8-byte blob present exactly when a currency is, so no row records an amount without its unit. The canonical receipt JSON lives in raw_json; the columns exist for query performance, and the store rebuilds receipts from raw_json on read and re-verifies signatures.

Contracts and their evidence

1. One mutable owner per file, by an OS lock at open and an epoch fence in every mutation transaction: serving_owner.rs, tests concurrent_process_open_is_fenced_before_serving, provisioned_store_rejects_independent_mutable_opens, stale_serving_epoch_fences_budget_and_revocation_access.
2. No write into a foreign, mistargeted, or future-schema file, and no change to it on refusal: schema_version.rs and its test module.
3. Receipt writes serialize on one connection through the group-commit actor: run_write and the actor loop in receipt_store.rs, test reader_pool_never_begins_a_write_transaction.
4. A restored stale snapshot is detected: serving_owner/rollback_anchor.rs, tests budget_only_snapshot_rollback_is_rejected_by_the_global_anchor and revocation_only_snapshot_rollback_is_rejected_by_the_global_anchor.
5. The kernel refuses on an unhealthy writer before any tool runs: ReceiptWriterLiveness::healthy in chio-kernel/src/receipt_store.rs. The receipt is durable before the allow returns.