Retention & Archive

Retention configuration

• retention_days sets the time cutoff, now - retention_days * 86400.
• max_size_bytes is compared with the freelist-adjusted live size, not the on-disk size.
• archive_path is created on the first rotation, then pinned for the life of the store.
• check_interval_secs is clamped to at least 1 second.
• explicit_cutoff_unix_secs is internal: archive_receipts_before sets it to bypass both thresholds, and it has no serialized form.

ReceiptsConfig in crates/platform/chio-config/src/schema.rs is deny_unknown_fields and holds three keys: store, checkpoint_interval, and retention_days. The archive path, size ceiling, tenant scope, and check interval come from RetentionConfig::default(); set them on the returned KernelConfig when you embed the kernel. to_kernel_config always emits retention_config: Some(..), so a file-configured kernel runs retention hourly, and its archive lands at the relative default path, resolved against the process working directory on the first rotation.

Attaching a store is where a policy that could never be honored is refused. try_set_receipt_store_handle refuses three configurations instead of spawning a worker that would fail every interval: a store whose supports_retention is false; a tenant_id against a store whose supports_tenant_scoped_retention is false, which is every store, SqliteReceiptStore included; and retention alongside checkpoint_batch_size == 0, since the watermark advances only to a checkpoint boundary and could never leave zero.

Checkpoint signer

Retention frees only what a checkpoint has committed, so its throughput is the checkpoint signer's. The attach that spawns the maintenance thread installs a BackgroundCheckpointSigner { keypair, max_batch } as one more command on the writer actor, with max_batch from checkpoint_batch_size (DEFAULT_CHECKPOINT_BATCH_SIZE = 100; 0 disables checkpointing). A store that advertises kernel-signed checkpoints but fails to install a signer has its attach refused, so it can never append forever without producing one.

The build runs after commit_receipt_batch has sent every append durability response, so it adds nothing to append latency. It runs on the same single writer connection and before the co-drained flush waiters are released, which makes flush_receipt_writes a checkpoint barrier as well as a durability barrier. Each checkpoint is O(batch), and the loop repeats while the head owes max_batch entries. A build failure, a caught panic included, records last_error and leaves the head untouched; the append that triggered it is already durable and stays committed.

Archive file, ledger, and tombstones

The archive is a separate SQLite database stamped with the Chio application_id and the receipt-store schema revision. create_archive_schema creates thirteen evidence tables plus the schema-version table: nine receive copied rows, and four checkpoint-projection tables stay empty so that opening the archive as a receipt store finds every table. Archive tables carry no REFERENCES clauses; the archive is a write-once evidence copy.

receipt_retention_watermark records each rotation's archived_through_entry_seq, cutoff, and archive path. Three triggers make it append-only and strictly increasing at the database level, because a later verifier trusts this value to skip work. Archive identity comes from re-deriving signed roots from the archived rows.

receipt_retention_tombstones keeps one small row per archived receipt id. Deleting a live receipt also deletes its UNIQUE(receipt_id) sentinel, so without a tombstone an archived id would look new to the append path's ON CONFLICT DO NOTHING. Two BEFORE INSERT triggers on the receipt tables abort any insert of a tombstoned id, and two more make the tombstones immutable.

Watermark query

SELECT COALESCE(MAX(kc.batch_end_seq), 0)
FROM kernel_checkpoints kc
WHERE NOT EXISTS (
    SELECT 1 FROM claim_receipt_log_entries e
    WHERE e.entry_seq <= kc.batch_end_seq
      AND e.timestamp >= ?1
)
AND NOT EXISTS (
    SELECT 1 FROM chio_authorization_receipt_consumptions ac
    JOIN claim_receipt_log_entries ae ON ae.receipt_id = ac.authorization_receipt_id
    JOIN claim_receipt_log_entries ce ON ce.receipt_id = ac.consumer_receipt_id
    WHERE ae.entry_seq <= kc.batch_end_seq
      AND ce.entry_seq > kc.batch_end_seq
)
-- ... three further NOT EXISTS clauses follow, plus a sixth on settle_attempts
-- ... when that table is present; see the table below

The first clause is the age rule; the others share one shape, never archiving a row that live evidence still points at. Each hold has its reason:

• Age. The unit of archival is a whole checkpoint batch.
• Authorization consumption. The binding row is a RESTRICT foreign key on the authorization, so archiving the pair separately would strand the live consumer's replay binding or fail the delete.
• Receipt lineage. Parent resolution for governed call-chain validation reads the live receipt tables, so a live child would lose its verified parent.
• Settlement and metered reconciliation. Both upsert paths need the live receipt row; archiving it would turn an actionable item into a NotFound.
• Settlement attempts. Every attempt row is pending or retryable work. Stores without the table skip the clause.

In incremental mode the result is capped at the newest checkpoint boundary the pre-flight audit verified, so a checkpoint row appended by a second handle since the head was seeded cannot widen the prune. A W of zero, or one that does not exceed the recorded high-water mark, returns zero archived without touching anything.

Copy, verify, delete

Before the rotation runs, the actor refuses if the writer's verified head is poisoned, then runs a full verify_checkpoint_chain_integrity (a Merkle rebuild per checkpoint above W, plus the archive re-hash below it) and a full claim-log set-equality validation that loads and sorts every source and projection row, in both verification modes. Only the delete and incremental_vacuum scale with the batch. An incremental head attests new appends only, so it would miss a checkpoint-covered source row and its projection row both deleted out of band after open; rotating over that drift would archive the survivors, delete the rest, and stamp a watermark the archive cannot back.

• Vet the path. The archive path is checked before the ATTACH. A target that vanishes on DETACH, or one that aliases the live database by path, symlink, or inode, is refused, since the identity checks would compare a table with itself while the delete removed the only copy. The path is then canonicalized, so the ledger records an absolute location that resolves the same way from any working directory.
• Copy idempotently. Append-only evidence uses INSERT OR IGNORE. The two reconciliation tables use INSERT OR REPLACE, because later reconciliation mutates their rows in place, and a stale archived copy would fail the identity re-check on every later rotation and stall the prefix.

verify_co_archival_complete checks identity, not presence. For each of nine tables it counts the live prefix rows whose archive counterpart matches on the table's key column and, NULL-safely, on every other column. Any shortfall raises RetentionArchiveIncomplete { table, live, archived } before a single delete; a count-only check would pass a reused archive holding the right ids under divergent bytes.

DROP TRIGGER IF EXISTS chio_tool_receipts_reject_delete;
DROP TRIGGER IF EXISTS chio_child_receipts_reject_delete;
DROP TRIGGER IF EXISTS claim_receipt_log_entries_reject_delete;

INSERT OR IGNORE INTO receipt_retention_tombstones
    (receipt_id, receipt_kind, archived_through_entry_seq, tombstoned_at)
    SELECT receipt_id, receipt_kind, {w}, {now}
    FROM claim_receipt_log_entries WHERE entry_seq <= {w};

DELETE FROM settlement_reconciliations WHERE receipt_id IN (
    SELECT receipt_id FROM claim_receipt_log_entries
    WHERE entry_seq <= {w} AND receipt_kind = 'tool_receipt');
DELETE FROM metered_billing_reconciliations WHERE receipt_id IN (
    SELECT receipt_id FROM claim_receipt_log_entries
    WHERE entry_seq <= {w} AND receipt_kind = 'tool_receipt');
DELETE FROM chio_authorization_receipt_consumptions WHERE authorization_receipt_id IN (
    SELECT receipt_id FROM claim_receipt_log_entries
    WHERE entry_seq <= {w} AND receipt_kind = 'tool_receipt');
DELETE FROM chio_tool_receipts WHERE seq IN (
    SELECT source_seq FROM claim_receipt_log_entries
    WHERE entry_seq <= {w} AND receipt_kind = 'tool_receipt');
DELETE FROM chio_child_receipts WHERE seq IN (
    SELECT source_seq FROM claim_receipt_log_entries
    WHERE entry_seq <= {w} AND receipt_kind = 'child_receipt');
DELETE FROM claim_receipt_log_entries WHERE entry_seq <= {w};

The deletion batch, the watermark insert, and the recreation of every immutability guard run inside one BEGIN IMMEDIATE transaction. SQLite executes CREATE and DROP TRIGGER transactionally, so a rollback restores the rows and the guards together. The pre-flight checks are fast fails; the transaction repeats them because a second handle can commit a dependent row or repoint the ledger in the window between.

The source tables and the projection lose exactly the same receipt-id set in that transaction, which is why the set-equality validator passes afterward with no exemption of its own. Cross-database atomicity is never required: the copy finishes first, the delete touches only main, and a crash between them leaves the live store intact. The first rotation on a store still reading auto_vacuum = NONE runs migrate_auto_vacuum_incremental_if_needed from rotate_on_writer_connection, because SQLite applies a changed auto_vacuum mode only to a new file or after a full VACUUM.

Rotation refusals

A panic is caught at two levels: the actor's catch_unwind returns a typed writer-panic error and takes a fresh connection for the next command, and the maintenance thread catches anything that escapes and retries at the next interval.

Retention repair

chio receipt retention repair --archive <path> recovers a store damaged by a delete that left orphaned projection rows. It opens through open_existing, so diagnosing a damaged store never fails at open. It removes only claim-log rows absent from both source tables, only when the named archive holds a byte-identical copy of each, only up to the smallest checkpoint boundary covering them, and only when every row in that boundary's prefix is itself an orphan whose archived rows re-derive the signed roots.

If a watermark already covers that boundary, the --archive you pass must be the one the ledger pinned, because the tombstones are stamped from it. Anything else aborts untouched, and both output modes exit non-zero when the drift survives. Every receipt command, from health through retention repair, resolves a local --receipt-db.

Contracts and their tests

• Retention runs on the single writer actor, never on a reader-pool connection. dispatch_rotate and the Rotate arm; reader_pool_never_rotates.
• Archived and live receipt-id sets partition the appended history. Over random interleavings of tool appends, child appends, and rotations, the store stays appendable, reopenable, and healthy, with no overlap and no loss; prop_retention_preserves_append_invariant, a state-machine proptest.
• A rotated store keeps working. The next append, a fresh open(), checkpoint status, and further checkpoint creation succeed; retention_then_append_and_reopen_succeeds, checkpoint_chain_watermark_exemption.
• An unbacked or tampered watermark skips nothing. bogus_watermark_does_not_skip_verification, watermark_trust_requires_backing_archive, watermark_trust_rejects_tampered_archive_contents, boundary_matching_watermark_over_live_prefix_does_not_skip_verification.
• A dependent row that escapes the copy survives. The delete fails closed instead of removing it unarchived; delete_fails_closed_when_a_dependent_row_escapes_the_copy, delete_rechecks_archive_path_under_the_write_lock.
• Size-driven rotation converges. It shrinks the measured live size, and a repeat under the same configuration archives nothing or leaves the store no larger; the tests assert convergence rather than one pass under the threshold. size_rotation_converges_below_threshold, size_rotation_archives_when_median_timestamp_is_shared.
• An archived id stays archived. It cannot be appended again, and its tombstone cannot be updated or deleted; archived_receipt_id_cannot_be_reappended, retention_tombstones_are_immutable.
• Tenant-scoped rotation is refused and changes nothing. Every store reports supports_tenant_scoped_retention() == false; tenant_scoped_rotation_rejected, tenant_scoped_rotation_is_rejected_and_leaves_counts_unchanged.