Chio/Docs
LOGIN · JOIN

PlatformDurable State

Node

Approval Store & Channels

The tables a paused call waits in, the channels that carry it to a human, and the transaction that resolves it.

The verdict next door, the state here

Approval & HITL owns the decision: what makes ApprovalGuard demand a human, which binding checks a presented token must survive, and what HitlVerdict comes back. That is one mediated call, and the value it can return is The Three Verdicts. This page owns everything that outlives the call: the rows on this node’s disk, the channels that leave the process, and the resume path back in.

The half that remembers

An approval store answers two questions for one process: what is still outstanding, and what has already been spent. Both are answerable by reading one node’s own filesystem, which is what files them here rather than a rung up. Nothing on this page contacts a peer. Running two nodes is precisely where the single-use guarantee stops holding, and that limit is stated below rather than papered over.

The contract is ApprovalStore in crates/kernel/chio-kernel/src/approval.rs, deliberately synchronous because the kernel does not run on an async executor and both implementations (an in-memory reference and SqliteApprovalStore) are synchronous too. The store is real and durable. Be exact about what drives it.

  • The guard is test-only. ApprovalGuard is constructed in one file, chio-kernel/src/kernel/tests/approval_flow.rs; everywhere else it is a re-export in lib.rs. The seven-guard pipeline does not stand in for it either: kernel/dispatch.rs treats a Guard that returns PendingApproval as an unsupported state and fails closed.
  • The sidecar writes rows over HTTP. POST /approvals/submit takes operator-supplied context, materializes an ApprovalRequest with governed_intent: None and a default policy_id of policy-hermes-hitl, and names the sidecar’s own signing key as the sole entry in trusted_approvers. Its doc comment states the boundary: manual flow only, the held call is not auto-resumed. Resolving records a decision; it does not release anything.
  • The kernel’s own pending path skips this store. Verdict::PendingApproval is raised by budget admission when a grant’s cumulative-approval requirement trips with no token attached. It signs a ThresholdApprovalProposal, parks it on the durable admission operation, and returns a Deny receipt reading cumulative approval required. No chio_hitl_pending row is written on that path.

Durability is inherited from the receipt store

chio api protect selects SqliteApprovalStore only when it was given a durable --receipt-store path. It refuses to boot without one unless --allow-ephemeral-receipts is set, and in that mode the approval store is InMemoryApprovalStore, the threshold collector is its in-memory twin, and the durable admission stores are absent, so every pending row and every collected vote dies with the process. An in-memory SQLite path counts as no path. Everything below about files applies to the durable configuration.

Five tables, two stores

SqliteApprovalStore creates every table below on open, all with CREATE TABLE IF NOT EXISTS, alongside the shared chio_store_schema_versions row it stamps. One type implements two traits against them: ApprovalStore for the first three, ThresholdApprovalCollectorStore for the last two.

TableHoldsKeyed on
chio_hitl_pendingOutstanding requests, stored as the serialized ApprovalRequest in payload with eight fields lifted into columns for querying. Indexed on subject_id and on expires_at.approval_id
chio_hitl_resolvedThe audit row a resolution leaves: outcome, timestamp, approver hex, token id. Indexed on (subject_id, policy_id, outcome), which is what count_approved reads.approval_id
chio_hitl_consumed_tokensThe replay registry. One row per token actually spent.(token_id, parameter_hash)
chio_threshold_approval_collectorsOne signed threshold proposal per row, with its requirement, its state, an optimistic version, and the whole record as canonical JSON in record_json. Reads come from that blob.proposal_id
chio_threshold_approval_collector_votesIndividual approver tokens, written so SQL enforces uniqueness rather than to be read back: nothing selects from this table outside the v1 migration.(proposal_id, token_id)

Batch approvals are a second store type entirely. SqliteBatchApprovalStore takes its own path and holds one table, chio_hitl_batches, indexed on (subject_id, revoked). No shipped binary opens it.

Both are own-file stores in the sense Node State on Disk defines: they take a path, open it directly, and never join the serving-owner arrangement the authority database uses. The schema gate is the shared one, under the store key approval at revision 2 and batch_approval at revision 0. What is specific here is the anchor set. A standalone open accepts exactly one anchor, chio_hitl_pending, so a path mistargeted at a receipt, revocation, budget, or authority database is refused rather than written with HITL tables. open_colocated_with_receipt_store widens the set with three receipt anchors, because chio api protect keeps both stores in one file and opens the receipt store first, so the approval store meets a database already carrying somebody else’s anchor and no approval table yet. The pair is pinned by standalone_open_refuses_a_receipt_sidecar_that_colocated_open_adopts.


Pending row to resolved row

Writing the request

store_pending is idempotent on approval_id, but only for a byte-identical payload. The whole rule is one statement:

crates/platform/chio-store-sqlite/src/approval_store.rssql
INSERT INTO chio_hitl_pending (...) VALUES (?1, ..., ?9)
ON CONFLICT(approval_id) DO UPDATE SET payload = excluded.payload
  WHERE chio_hitl_pending.payload = excluded.payload
RETURNING payload

A retry carrying the same request updates a row to itself and returns it. A retry carrying a different one fails the WHERE, so the update is skipped, RETURNING yields nothing, and the call reports approval_id ... already exists with different payload rather than overwriting in-flight HITL state. The in-memory store reaches the same outcome with an equality compare before insert. Two things about that refusal are worth knowing. It is an ApprovalStoreError::Backend, not a typed conflict, so a duplicate /approvals/submit comes back as 500 internal_error. And the comparison is on the serialized string, not on JSON semantics, so a payload that changes shape across a binary upgrade reads as a conflicting retry rather than a matching one.

Leaving the process

A channel is a delivery sink, and the trait is two methods: name and dispatch. Two implementations exist in-tree and neither has a caller outside tests, because the only code that dispatches channels is ApprovalGuard::evaluate. In the shipped sidecar, notification is poll-only: GET /approvals/pending is the whole delivery mechanism, and it returns every matching row unless the caller passes a limit.

WebhookChannel is a blocking ureq POST with a 5s default timeout and one optional static header, deliberately one because every real caller uses a single auth secret. Being static it cannot be a per-body HMAC, whatever the doc comment suggests, and the channel retries nothing even though the trait says implementations own retries. Its body is { event, approval, callback_url } with event fixed at approval_requested, approval the entire serialized request including its trusted approver keys, and callback_url a host-less relative path. RecordingChannel appends every dispatch to an unbounded in-memory vector; its doc comment attributes it to an api-poll dispatch mode that does not exist in the tree.

Where the guard does run, dispatch failure is not a refusal. It iterates channels sequentially, logs a redacted warning on Transport, Remote, or Config, and leaves the pending row in place so the poll route still serves it. A webhook outage delays a call; it does not drop or allow one.

Doc comments describing behavior that is not there

dispatch returns a ChannelHandle of channel, channel_ref, and an optional action_url, and its doc comment says the kernel records it alongside the request so the dispatch can be cancelled later. Neither half is true in the tree: the guard discards the Ok value, no table has a column for it, and the trait has no cancel method. ApprovalRequest::callback_hint is documented as something a dispatcher fills in after sending and is None at every construction site in the repo. The module header links ChioKernel::evaluate_tool_call_with_hitl, which no longer exists. Treat all three as reserved shape, not behavior.

The resolve transaction

resolve is one SQLite transaction doing six things in a fixed order: read the pending row’s policy_id and parameter_hash inside the transaction to avoid a check-then-act race, refuse if the bound token is already in the consumed registry, refuse if a resolution row already exists, insert the resolved row by selecting straight from the pending row, insert the consumed-token row, and delete the pending row. Either the request stops being pending and the token stops being spendable, or neither happens. That shape is why record_consumed and is_consumed are separate trait methods rather than folded into resolve, as the trait comments say. One consequence of the order: because the pending row is read first and deleted last, a second resolve of the same id returns NotFound, not AlreadyResolved. The already-resolved branch is only reachable when a pending row was reinserted after a resolution.

The caller above it is resume_with_decision, and its ordering carries a stated reason. It loads the pending record, reports already resolved: {id} ({outcome}) when only a resolution exists, checks the replay registry, verifies the token against the stored request, and only then compares the HTTP envelope’s outcome to the signed decision. That last check runs before the store is touched: a mismatched pair, where the token says denied and the body says approved, would otherwise have already flipped the request to resolved and moved the approved counter before the error was returned.

ConditionWhat resume returnsStatus on respond
URL id disagrees with the signed request_idRejected before the store is read400
No pending row and no resolutionApprovalRejected("unknown approval id")403
A resolution row already existsApprovalRejected("already resolved")403
The token is already in chio_hitl_consumed_tokensApprovalRejected("approval token already consumed (replay)")409
The store raises Replay from inside the transactionApprovalRejected("replay detected")409
Binding, approver-trust, time, lifetime, or signature check failsApprovalRejected(reason)403
A concurrent respond deleted the pending row between the load and the transactionInternal("approval store: approval request not found")500

Read that third column carefully. The resume path collapses every distinction into one string-carrying variant, and the HTTP mapping recovers 409 by testing whether the message contains replay. Anything the resume path does not classify as ApprovalRejected becomes 500. The typed mapping that turns NotFound into 404 and AlreadyResolved into 409 is real, but it applies only where a handler calls the store directly, as GET /approvals/{id} does. Match on the code, and do not read a 403 on respond as a signature failure.

The batch endpoint, POST /approvals/batch/respond, runs the same path per entry and never aborts the set: each decision lands as resolved or rejected with its own message, under one summary of total, approved, denied, and rejected. It is a loop over independent transactions, not a transaction over the loop. The list is uncapped, and the response is 200 even when every entry was rejected, so the summary is the only thing worth reading.

The TTL ceiling, and the deadline that is not one

MAX_APPROVAL_TTL_SECS = 3600 caps the token lifetime: expires_at - issued_at above an hour is rejected in verify_against, because the single-use registry’s retention horizon is pinned to that value and a longer token could outlive its own replay entry. The sidecar clamps in the same units on the way in, ttl_seconds.min(3600) with 3600 as the default when the field is zero or absent, and mints 600-second tokens on the operator-respond shortcut.

A request’s expiry is advisory

ApprovalRequest::expires_at is written, indexed, and never enforced. verify_against compares the clock only to the token’s own window, and no resolve path reads the request’s deadline, so a request that lapsed weeks ago still resolves cleanly. Nothing sweeps expired rows out of chio_hitl_pending either: the only DELETE in the store is the one inside resolve. Filtering is opt-in, through ApprovalFilter::not_expired_at on list_pending, which the caller must pass as a query parameter. Operators who need auto-deny at the deadline should treat that as unbuilt.

Two paths beside the single request

Threshold: m-of-n on one node

The threshold collector is the shipped multi-approver path. chio api protect routes four endpoints at it under /approvals/threshold/proposals for create, get, respond, and deliver, and opens SqliteApprovalStore a second time against the same file to back it. Those routes are the only writers of the collector tables; the kernel’s own cumulative-approval proposal is parked on the durable admission operation instead. A proposal must be signed by a trusted policy authority, and in the sidecar that set holds one key, the sidecar’s own signer, which is also the key its embedded kernel signs proposals with. It must further agree with its requirement on threshold and eligible-set digest before it is stored.

Each submitted token must agree with the proposal on request id, intent hash, subject, and the proposal’s own artifact digest, sit inside three proposal-relative bounds (issued_at at or after creation, before the deadline, and expires_at no later than the deadline) as well as its own validity window against now, come from an approver in the eligible set, and survive a submitter-separation check when one is required. Its decision must be Approved: the collector has no way to record a denial, only cancel.

Storage is optimistic. Every mutation reads the record inside a transaction, refuses if the version moved or the state is terminal, and writes with the expected version and previous state in the WHERE clause; a row count other than one is a conflict, not a retry. The vote table enforces the rest in SQL:

crates/platform/chio-store-sqlite/src/approval_store.rssql
canonical_token_digest TEXT NOT NULL UNIQUE,
...
PRIMARY KEY (proposal_id, token_id),
UNIQUE (proposal_id, approver_fingerprint),
UNIQUE (proposal_id, canonical_token_digest),
FOREIGN KEY (proposal_id)
    REFERENCES chio_threshold_approval_collectors(proposal_id)

The column-level constraint is the strict one: a canonical token digest is unique across the whole file, not per proposal, so one signed token cannot be submitted to two proposals even where the rest would allow it. The foreign key is weaker than it looks, because PRAGMA foreign_keys is set only on the connection that ran the migration.

Revision 2 exists for the primary key. The v1 table made token_id globally unique, so two proposals collided on an id that only has to be unique within one; opening a v1 database rebuilds the table scoped to the proposal and copies every row across in one transaction, asserted by open_migrates_v1_threshold_vote_ids_to_proposal_scope. Delivery has one more subtlety worth knowing: token expiry is bounded above by the proposal deadline but not below, so a set that was quorate at submission can hold lapsed tokens by delivery time. deliver filters to currently valid tokens and fails with threshold approval quorum is no longer satisfied rather than delivering a stale set, and the proposal stays Ready so the same signer can replace their expired token.

Batch: pre-approving a class

A BatchApproval lets one human pre-approve a class of calls: a subject, a server and tool pattern, an optional per-call and total amount, an optional call count, a time window, and used counters carried on the row so consumption can be reconciled without going back to the receipt log. find_matching narrows in SQL to non-revoked rows for the subject inside the window, then applies pattern, count, and amount filters in Rust and returns the first hit. There is no ORDER BY, so which batch wins when several match is unspecified. Pattern matching is exact, full wildcard, or one trailing-star prefix. Amount handling has one rule worth stating plainly: a call carrying no monetary intent matches only a batch that constrains neither per-call nor total amount, so putting a cap on a batch narrows it to priced calls and nothing announces that. Consumption is not atomic with matching either. record_usage is a separate unconditional UPDATE, so concurrent callers can both clear a max_calls check and overshoot it, and store is INSERT OR REPLACE, so re-storing a batch id overwrites its used counters.

Nothing consults it yet

BatchApprovalStore has both backends, a schema, and a passing test. It has no caller. find_matching and record_usage are invoked from exactly one kernel unit test, against the in-memory backend, and nowhere in any product crate, so no mediated call is currently satisfied by a batch approval. It is a finished store waiting for a guard.

Guarantees and limits

StatusClaimEvidence
ShippedResolution is atomic: the resolved row, the consumed-token row, and the pending delete commit together or not at all, with both the replay and double-resolve checks inside the same transaction.SqliteApprovalStore::resolve
Proved by testPending rows, the consumed registry, and resolutions survive a process restart: a second store handle on the same file lists the pending request and resumes it to Approved.persistence_survives_restart, chio-store-sqlite/tests/approval_store.rs
Proved by testA token spent once cannot be spent again, even after the pending row is reinserted; a second record_consumed for the same pair is a Replay error rather than a silent no-op.resolve_rejects_replay, record_consumed_is_idempotent_on_first_write_only
Proved by testA threshold proposal recovers its collected votes across three separate store opens and records delivery before returning the token set.threshold_collector_recovers_votes_and_persists_delivery_before_return, chio-store-sqlite/src/approval_store.rs
LimitSingle-use is per store. The registry is a local table keyed on (token_id, parameter_hash); two kernels with separate stores cannot reject each other’s replays. Sharing one file instead runs into the unsupported row below.chio_hitl_consumed_tokens; InMemoryApprovalStore::consumed_key
LimitIn the sidecar the approver is the sidecar. /approvals/submit installs the process signing key as the only trusted approver and /approvals/{id}/operator-respond signs the decision with it, falling back to that same key for the token subject when the request carries none. The human factor is reaching the route, not holding a key.chio-api-protect/src/proxy/approval.rs
LimitEvery approval route sits behind the sidecar-control gate, and that gate admits any loopback caller when no control token is configured. The token comes from CHIO_SIDECAR_CONTROL_TOKEN in the environment, so an operator who never sets it makes approving a local, unauthenticated POST.require_sidecar_control_request in proxy/sidecar.rs
LimitWithout --authority-seed-file the sidecar generates a keypair per boot, which it warns about at start. Pending rows outlive the key named on them, so after such a restart operator-respond returns sidecar signer is not a trusted approver for this request and the row is unresolvable.signer_seed_hex in proxy/state.rs; approver check in operator_respond_approval_handler
LimitPragmas are issued once, on the pooled connection that ran the migration: journal_mode, synchronous, busy_timeout, foreign_keys. WAL is a property of the file and persists; the other three are per-connection and are not reissued, so the other connections the pool hands out carry SQLite defaults and do not enforce the vote table’s foreign key. Sibling stores in the same crate install busy_timeout through with_init; this one does not.approval_store.rs::run_migrations against execution_nonce_store.rs and governed_approval_replay_store.rs
LimitA request’s expires_at is recorded and indexed but never enforced, and no sweeper removes lapsed pending rows.No comparison against expires_at in verify_against, resume_with_decision, or resolve; the only delete is inside resolve
Not durable by defaultUnder --allow-ephemeral-receipts with no durable receipt path, the approval store and the threshold collector are the in-memory implementations and the durable admission stores are absent, so the kernel’s cumulative-approval path cannot even park a proposal.durable_receipt_db branch in proxy/state.rs
Not wiredApproval channels. Neither WebhookChannel nor RecordingChannel is constructed outside kernel tests, because only ApprovalGuard dispatches them. Notification in the sidecar is polling.Repo-wide grep for both types in crates/
Not wiredBatch approvals. Both backends and the schema exist; no product crate calls find_matching.One kernel unit test is the only caller in crates/
Not wiredThe single-request ApprovalGuard in a shipped product. The sidecar drives the store over HTTP instead.ApprovalGuard::new appears only in chio-kernel/src/kernel/tests/approval_flow.rs
UnsupportedTwo processes writing one approval database. Unlike the authority file, this store takes no serving lock and mints no epoch fence, so nothing detects a second writer; the arbitration is SQLite’s file locking alone, on connections that mostly lack a busy timeout.No serving_owner reference in approval_store.rs; contrast the serving-owner row
Design onlyTimeout actions and escalation chains. The HITL protocol document specifies timeout_action, tiered escalation, and an Escalated approval state; no TimeoutAction or EscalationChain type exists in any crate and no approval type here has an escalated variant. The unrelated Escalated case state in chio-governance is a different mechanism.docs/protocols/HUMAN-IN-THE-LOOP-PROTOCOL.md sections 2 and 15

Two replay registries, one word

The table in this store records approval tokens already spent on a resolution. It is not the same mechanism as GovernedApprovalReplayStore, which reserves a (subject_id, request_id, intent_hash) tuple at the dispatch boundary before any payment or tool side effect and leaves the marker blocking when a commit result is unknown. Different table, different key, different crate module. A deployment that needs replay safety across restart installs a durable implementation of both.

Next Steps

  • Approval & HITL · the guard, the constraints that trigger it, and the seven binding checks a presented token must survive
  • The Three Verdicts · who may construct PendingApproval, and what every other runner does on sight of it
  • Node State on Disk · the schema stamp this store passes through, the sidecar file it shares, and the fence it does not take
  • Revocation Store · the other node-local list the kernel consults before an allow
  • Fail-Closed Semantics · what the node does when a store refuses to open mid-flight
Approval Store & Channels · Chio Docs