Tenancy & Read Scoping

One admin token, N read tokens

TrustServiceConfig carries a single service_token and a BTreeMap<String, String> named tenant_read_tokens, mapping a tenant id to the token that speaks for it (trust_control/service_types/config.rs). The admin token is the whole service: every write, every operator report, and a read that crosses all tenants. A tenant read token authorizes nothing but reads, and those reads are confined to receipt rows carrying that tenant’s tag.

The mechanism is shipped and deliberately small. Seven call sites resolve a control read principal. Five of them serve a tenant token real data; one resolves the principal only to refuse it; the seventh is resolve_admin_report_read_context, which twenty-seven registered report routes call and which turns any tenant token into a 403. Every other public route, including every mutating one, runs validate_service_auth against the admin token and has no tenant concept at all; the deliberately public passport and discovery routes take no credential.

The internal replication routes are the exception in both directions. They authenticate with the keyed peer digest rather than a bearer token (validate_cluster_peer_auth), and handle_internal_tool_receipts_delta and its child-receipt twin then read with ReceiptReadContext::admin_service(). A peer credential is a second admin-all read of the same log, governed by the allowlist rather than by anything on this page.

The log being scoped is the replicated one (see Replication & Convergence), so a tenant token presented to any node reads that node’s copy. What that buys is convergence, not simultaneity: the same token can see one more row on the leader than on a follower mid-round. That last sentence is read off the replication model, not a test. The single-process view of the same store is Node State on Disk. The CLI draws the line itself: passing a read-boundary flag alongside --control-url is refused with receipt list read-boundary flags apply to local --receipt-db; remote reads derive scope from the control token.

The principal, and what it may ask

Resolution produces one of two values, and each maps to exactly one read context:

Where the two diverge per route is a list, not a rule. Each handler decided separately.

Child receipts have no tenant column, so that handler refuses rather than serve rows it cannot filter, and the message says so.

Resolution, in order

1. Configuration, twice

The CLI accepts repeated --tenant-read-token tenant=token pairs, split on the first = only, so a token may contain that character and a tenant id may not. parse_tenant_read_tokens rejects a spec with no separator, either half padded, either half carrying a control character, either half empty, and a duplicate tenant id. It then refuses a tenant token equal to --service-token before the config is even built.

TrustServiceConfig::validate repeats all of it against the assembled map, which is what catches an embedder that never went through the CLI:

for (tenant_id, token) in &self.tenant_read_tokens {
    if tenant_id.trim().is_empty() {
        return Err(CliError::cli_other_error(
            "tenant read token id must be non-empty".to_string(),
        ));
    }
    if tenant_id.trim() != tenant_id {
        return Err(CliError::cli_other_error(
            "tenant read token id must not contain surrounding whitespace".to_string(),
        ));
    }
    if tenant_id.chars().any(char::is_control) {
        return Err(CliError::cli_other_error(
            "tenant read token id must not contain control characters".to_string(),
        ));
    }
    let token_label = format!("tenant read token for `{tenant_id}`");
    validate_control_secret(token, &token_label)?;
    if token == &self.service_token {
        return Err(CliError::cli_other_error(
            "control tenant read token must not equal service token".to_string(),
        ));
    }
}

validate_control_secret is the same gate the admin token passes: non-empty after trimming, no surrounding whitespace, no control characters. Padding is refused rather than trimmed, which matters because the wire comparison is byte equality against the configured string, so silently trimming would make a token that works in one place fail in another. serve_async calls validate before it binds a listener, so a service configured this way never starts.

2. Per request, before any store opens

config
    .validate()
    .map_err(|error| plain_http_error(StatusCode::INTERNAL_SERVER_ERROR, &error.to_string()))?;
let Some(provided) = control_bearer_token(headers) else {
    return Err(missing_or_invalid_control_token());
};
if bool::from(provided.as_bytes().ct_eq(config.service_token.as_bytes())) {
    return Ok(ResolvedControlReadPrincipal::AdminService);
}
for (tenant_id, token) in &config.tenant_read_tokens {
    if bool::from(provided.as_bytes().ct_eq(token.as_bytes())) {
        return Ok(ResolvedControlReadPrincipal::TenantRead {
            tenant_id: tenant_id.clone(),
        });
    }
}
Err(missing_or_invalid_control_token())

Four things follow from those lines. The admin token is compared before the map is walked, and nothing turns on that order: the only configuration where it would matter, a tenant token equal to the admin token, is refused by the validation directly above. Both comparisons use subtle::ConstantTimeEq, the same primitive the peer digest uses. An unmatched token is a plain 401 missing or invalid control bearer token with WWW-Authenticate: Bearer, which does not distinguish a wrong tenant token from a wrong admin one; control_bearer_token strips the literal prefix "Bearer ", so a lowercase scheme or a second space lands in the same 401.

The fourth is weaker than it looks. The configuration is revalidated on every read and a failure is a 500, but serve_async already validated the same struct before binding and TrustServiceState holds it by value with no mutation path, so a served process cannot reach that branch. It is a guard for embedders that build a TrustServiceConfig by hand, which is exactly what the two tests that cover it do: trust_service_config_boundary_rejects_invalid_auth_material_and_cluster_timing drives a blank tenant id and a whitespace-only tenant token to 500, and auth_helpers_and_metered_billing_validation_cover_error_paths drives the token-equals-service-token collision to it.

Ordering is the part to hold on to. Every handler that reads receipts resolves its principal as its first statement and calls open_receipt_store only after, so no unauthenticated request opens a database file. An authenticated request that will still be refused sometimes does: both the point-load 403 and the evidence-export authorization run after the store is open. The same discipline holds outside the control plane. The SIEM exporter validates its configured read context in ExporterManager::new and refuses anything that is not an AdminAll boundary with SIEM receipt polling requires explicit admin receipt read authority, under a test named for the property (manager_new_rejects_non_admin_read_context_before_opening_db).

3. The store refuses to guess

A read context is not advice to the store; it is the only thing that authorizes a row. ReceiptQuery::effective_read_scope turns the boundary into a scope and returns an error when there is no context at all, which is the property tenant_filter_without_read_context_fails_closed pins against a live SQLite store.

ReceiptReadBoundary::TenantScoped { tenant } => {
    let tenant = tenant.trim();
    if tenant.is_empty() {
        return Err(
            "tenant-scoped receipt query requires a non-empty tenant".to_string()
        );
    }
    if self
        .tenant_filter
        .as_deref()
        .is_some_and(|filter| filter != tenant)
    {
        return Err(
            "receipt query tenant filter cannot widen authenticated tenant scope"
                .to_string(),
        );
    }
    Ok(EffectiveReceiptReadScope {
        tenant: Some(tenant.to_string()),
        include_null_tenant: context.include_null_tenant,
        is_admin_all: false,
    })
}

The store does not rely on the handler having refused first. Every report builder runs require_admin_receipt_read_context and every admin or replication listing runs require_admin_list_context; both turn a tenant-scoped context into a ReadBoundary error, the first with a message that names the reason the restriction exists (... requires admin receipt read authority until tenant-scoped report filtering is implemented). The HTTP 403 and the store error are two independent gates on the same property.

tenant_filter is narrowing only, and only under an admin boundary. Under a tenant boundary it must either match or be absent. The scope then picks one of three SQL predicates, and on a cost-bounded page that choice is also an index choice: the tenant-bound shape plans onto idx_chio_tool_receipts_cost and the admin shape onto idx_chio_tool_receipts_cost_global, asserted by bounded_cost_page_and_count_queries_use_scope_appropriate_indexes through EXPLAIN QUERY PLAN.

let tenant_fragment = match (
    read_scope.tenant.as_deref(),
    read_scope.include_null_tenant && !self.strict_tenant_isolation_enabled(),
) {
    (None, _) => "(?12 IS NULL)",
    (Some(_), true) => "(r.tenant_id = ?12 OR r.tenant_id IS NULL)",
    (Some(_), false) => "(r.tenant_id = ?12)",
};

The middle arm is the compatibility mode for databases written before receipts carried a tenant tag, and it is unreachable in a shipped deployment. Both store open paths set strict_tenant_isolation true, no CLI flag or service option flips it, and the only constructor that sets include_null_tenant on a tenant-scoped context (local_operator_tenant) has no non-test caller anywhere in crates/. Untagged rows are therefore invisible to every tenant-scoped read that a running system can issue.

The same refusal on the CLI. The local reader hits the boundary before it hits the store. Against a receipt database holding one allow and one deny, an unscoped listing is refused outright, a tenant-scoped listing returns nothing because the rows carry no tenant tag, and the admin listing returns both.

chio --receipt-db ./.chio/receipts.db receipt list --limit 5
chio --receipt-db ./.chio/receipts.db receipt list --tenant acme --limit 5
chio --receipt-db ./.chio/receipts.db receipt list --admin-all --limit 5 \
  | jq -c '{id, tool_name, decision}'

$ chio --receipt-db ./.chio/receipts.db receipt list --limit 5
error [urn:chio:error:cli:other]: --tenant <id> or --admin-all is required for local receipt reads
context: {"domain":"cli","severity":"error","stability":"deprecated","string_code":"CHIO-CLI-OTHER"}
suggested fix: Preserve the original message and migrate the call site to a specific registry code when touched.
exit=1

$ chio --receipt-db ./.chio/receipts.db receipt list --tenant acme --limit 5
exit=0

$ chio --receipt-db ./.chio/receipts.db receipt list --admin-all --limit 5 | jq -c '{id, tool_name, decision}'
{"id":"6bccfe0c3a76f0596f0ac166c8f170fea836455fad4bdd7dadb5371a801690cc","tool_name":"hello_world","decision":{"verdict":"allow"}}
{"id":"7dccc54ca07cb9edb27acbeac1524885e03f7a59f34ada796291e8f43c4174f2","tool_name":"read_file","decision":{"verdict":"deny","reason":"requested tool read_file on server hello is not in capability scope","guard":"kernel"}}
exit=0

The middle read exits 0 with no rows, which is the behavior the last paragraph describes: a tenant-scoped scope compiles to (r.tenant_id = ?12) under strict isolation, and untagged rows do not match. An empty result and a refusal are different answers, and confusing them is how a caller concludes a tenant has no receipts when it has no authority.

4. Evidence export authorizes twice

handle_evidence_export calls authorize_evidence_export_query on the caller’s query, runs prepare_evidence_export, and then calls the same authorization again on the prepared query. The second call is not defensive duplication. Preparation merges a signed federation policy document into the query, and merge_read_boundary resolves a conflict-free merge in the policy’s favour: a policy carrying AdminAll replaces the tenant-scoped boundary the first authorization just installed. The second call catches exactly that and answers 403 tenant read token cannot request admin-all evidence export. A policy naming a different tenant fails earlier, inside the merge, with requested export tenant falls outside the signed federation policy.

The same authorization has two more refusals the first call already covers: a request that names another tenant in read_boundary is tenant read token cannot export evidence for another tenant, and one that names another tenant in the tenant field is tenant read token cannot narrow evidence export to another tenant. A tenant token that asks for nothing in particular has its boundary and its tenant field filled in for it. Separately, merge_read_boundary refuses any attached policy that does not bind a boundary of its own: signed federation policy must bind an explicit receipt read boundary.

Where the tag comes from

A read token selects rows by tenant_id. Nothing about the read decides what that column holds. Two writers set it, and both derive it from an authenticated identity rather than from anything the caller put in a request body.

The kernel is the first. resolve_tenant_id_for_session reads the session’s auth context and returns the tenant claim from the OAuth bearer’s enterprise_identity, falling back to federated_claims.tenant_id when the identity provider only supplies the minimal claim set. The resolved value is installed as a request-keyed scope for the duration of the evaluation, so every receipt signed under that request carries it, and the doc comment says why the value cannot come from the request: caller choice would defeat the isolation guarantee. chio-kernel/src/kernel/tests/multi_tenant_receipt.rs holds the line, including blocking_evaluate_without_session_leaves_tenant_id_none, which asserts that a sessionless evaluation writes an untagged receipt regardless of any thread-local residue.

The SCIM lifecycle is the second writer. DELETE /scim/v2/Users/{user_id} revokes every capability the registry tracked for that identity, appends a deprovisioning receipt, and deactivates the record. build_scim_deprovision_receipt takes the tag from record.enterprise_identity.tenant_id and refuses to build the receipt without one: scim deprovision receipts require an enterprise tenant_id, returned as a SCIM-shaped 409. A provider that maps no tenant claim can create users and cannot deprovision them, and the refusal lands late: the capability revocations are already durable by the time the receipt fails to build, so the identity is disarmed while its registry record stays active and unevidenced.

SCIM is the path for human and service-account identity

A SCIM provider is one kind of EnterpriseProviderRecord alongside oidc_jwks, oauth_introspection, and saml. Records validate themselves on every registry load and save rather than at use: validate returns a vector of errors that is stored on the record, and is_validated_enabled requires the record to be enabled with that vector empty. Every kind needs provider_id, provenance.configured_from, provenance.trust_material_ref and subject_mapping.principal_source; the SCIM kind adds scim_base_url. A configured tenant or organization must fall inside trust_boundary.allowed_tenants or allowed_organizations, where an empty allowlist means no constraint rather than no access.

Creating a user requires the urn:chio:params:scim:schemas:extension:chio:2.0:User extension carrying a providerId, and the tenant tag that later scopes reads is the extension’s tenantId copied into the enterprise identity context. The principal is projected from the provider’s configured principal_source: six sources (userName, externalId, id, email, clientId, objectId) spelled ten ways, with anything else refused by name. The stable subject_key is a double SHA-256: derive_enterprise_subject_key digests b"chio.identity_federation.v1", 0x01, the provider id, 0x00 and the canonical principal, then hands the raw digest to sha256_hex, which hashes it again. Both SCIM routes are admin-token only, and both forward to the cluster leader through helpers that do not check the authority lease; see Leader & Failover for what that costs. The registry itself is a JSON file at --scim-lifecycle-file, not a replicated store.

SPIFFE is the only workload identity scheme

WorkloadIdentityScheme has exactly one variant, and the match inside WorkloadIdentity::validate is exhaustive over it. There is no second scheme to add a branch for.

#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum WorkloadIdentityScheme {
    Spiffe,
}

Parsing is strict and every refusal has its own error: a non-spiffe scheme, userinfo or a port, a query or fragment, a missing host, and a path that is empty, unrooted, or contains a doubled slash. validate re-parses the stored URI and reports a Conflict when the recorded trust_domain or path disagrees with what the URI derives, which stops a hand-edited record from claiming one trust domain while naming another. The credential kind (uri, x509_svid, jwt_svid) is carried through parsing untouched: the type records how the identity was said to be presented, not that the claim is true. Full shape and delivery paths are Workload Identity.

The passport split

Hosted passport issuance and lifecycle run on this service, backed by two files: --passport-issuance-offers-file and --passport-statuses-file. Three auth regimes run across those routes and none is tenancy. Operator routes gate on the admin token through validate_service_auth. The OID4VCI token and credential endpoints authenticate the holder flow instead, by pre-authorized code and then by the access token redemption issued. Discovery and public-resolve take no credential. No passport route resolves a control read principal, so a tenant read token reaches none of them.

What crosses an authority boundary is the other half. handle_federated_issue, itself admin-gated, accepts a passport presented by a holder whose credentials were issued under someone else’s key. It binds the presentation to a challenge whose verifier must equal this service’s advertised URL, and for a multi-hop chain it requires a signed delegation policy bound to the exact upstream capability id. Both verifier checks are wrapped in if let Some(advertise_url), so a node started without --advertise-url skips them entirely and accepts a challenge naming any verifier. That mechanism, the signed federation policy documents that constrain an evidence export, and the status reference a foreign verifier resolves against this issuer all belong to Federation. Issuing and revoking under your own key is Cluster; what happens to the credential where your key does not govern is not.

Guarantees and limits

Next Steps

• Cluster Peer Auth · the other credential this service accepts, and why the internal routes do not take a bearer token at all
• Workload Identity · the SPIFFE record in full, its delivery paths, and the policy match that consumes it
• Federation · what happens to a passport, and to exported evidence, once it leaves this authority
• Replication & Convergence · how the receipt rows a tenant token reads reach every node
• Receipt Query API · the filters, cursors, and page limits these routes accept
• Secrets & Signing Keys · custody for the admin token every tenant token is checked against