Credential custody

A credential broker running as a dedicated Linux service, with its keys delivered through sealed descriptors, four private stores, and a client socket that opens only after every restored attempt is reconciled with the authority.

Custody covers more than encryption at rest. The service protects the keys that decrypt the material, authenticates the process that asks to use it, constrains the provider request, and keeps enough history to refuse a replay after a crash. Chio's production broker owns those duties in one supervised process; the agent and the tool receive references, and the broker owns the provider connection and the raw material.

1. Provision the complete custody set

The broker keeps four durable SQLite databases, for encrypted credentials, attempt state, administrative replay state, and signed receipts. They travel with the tenant master key and the broker signing seed. The enterprise migration ledger is anchored independently, so restoring a valid old database prefix cannot quietly bring back an obsolete security posture.

Provision the sealed supervisor handoff and the service identities first, and take installed assets from the reference runtime. Keep provider credentials and service seeds out of command-line arguments and environment variables, which leak into process listings, crash reports, shell history, and child processes; the configuration names custody and policy and holds no plaintext secret.

2. Write the closed runtime configuration

The runtime configuration is one closed, versioned document with the schema chio.secret-brokerd.runtime-config.v5 and camelCase JSON names. Unknown fields are refused, so a misspelled security setting is a startup error.

• Identities. trustedServiceUid is the identity the daemon runs under, and authorizedClientUid constrains the client process; trusted issuer and authority keys constrain signed statements. Unix identity and cryptographic authority are separate checks, and each needs its own setting.
• Provider integration. providerAdapterId, providerAdapterVersion, and providerPlacement (bearer_authorization or api_key_header) select the reviewed integration. Placement is service configuration; a caller never picks a header for the secret.
• Stores. Four absolute, distinct, service-private paths, declared inside the complete configuration.
• Freshness ceilings. Authority timeout, clock skew, and snapshot ages each have a hard ceiling. Choose tighter values from observed behavior, and diagnose stale state at the authority connection before raising one.

Every field, bound, and the audit identity rule are in the runtime configuration reference below.

3. Hand the daemon its two sealed keys

The trusted supervisor provisions two distinct sealed inherited descriptors and the private configuration, then launches the daemon. The variables hold descriptor numbers and a configuration path, never key bytes.

: "${BROKER_CONFIG:?Set the private runtime configuration path}"
: "${BROKER_MASTER_KEY_FD:?Supervisor must supply the sealed master-key descriptor}"
: "${BROKER_SIGNING_KEY_FD:?Supervisor must supply the distinct sealed signing descriptor}"
exec chio-secret-brokerd \
  --config "$BROKER_CONFIG" \
  --master-key-fd "$BROKER_MASTER_KEY_FD" \
  --signing-key-fd "$BROKER_SIGNING_KEY_FD"

The daemon refuses repeated or unknown flags, missing values, malformed descriptor numbers, and one descriptor offered for both key roles. It adopts each descriptor once, marks it close-on-exec, consumes the sealed key, and closes the descriptor before IPC readiness, with no reopen through procfs. An ordinary file descriptor without the sealed-key properties is refused. Daemon credentials shows how the supervisor seals the two descriptors.

4. Start with the retained set and verify readiness

On start, the daemon connects to the authoritative hold service and drains recoverable attempts before it binds the client socket. Held, captured, denied, and reversed states converge under their original deterministic identities, and permanently ambiguous outcomes stay non-dispatchable. A restored broker therefore continues the original issue attempt without restoring obsolete authority or consuming new quota.

systemctl show chio-secret-broker.service \
  -p ActiveState -p SubState -p MainPID
journalctl -u chio-secret-broker.service -n 30 --no-pager

You should see the unit active with a main PID. A build error returns without a ready client listener, so a bound client socket means the configured runtime and the original-store reconciliation completed. Confirm startup from the service log, the socket peer, and the original operation record.

use chio_secret_broker::daemon_runtime::{
    secure_inherited_key_file, BrokerDaemonConfig, BrokerDaemonRuntime,
};
use chio_secret_broker::{BrokerError, Result};
use std::{fs::File, path::Path, sync::atomic::AtomicBool};

fn require_linux_process_custody() -> Result<()> {
    #[cfg(target_os = "linux")]
    {
        chio_secret_broker::daemon_runtime::harden_broker_process_custody()
    }
    #[cfg(not(target_os = "linux"))]
    {
        Err(BrokerError::Custody(
            "broker process custody requires Linux".into(),
        ))
    }
}

/// Linux supervisor integration. Both distinct files are inherited sealed
/// descriptors; never reopen a seed pathname in the daemon.
pub fn serve_with_inherited_custody(
    private_config: &Path,
    inherited_master: File,
    inherited_signer: File,
    shutdown: &AtomicBool,
) -> Result<()> {
    require_linux_process_custody()?;
    let master = secure_inherited_key_file(inherited_master, "master key")?;
    let signer = secure_inherited_key_file(inherited_signer, "signing key")?;
    let config = BrokerDaemonConfig::load(private_config)?;
    // Build checks distinct underlying files, identities, durable stores and
    // authoritative recovery before the client socket becomes ready.
    let runtime = BrokerDaemonRuntime::build(config, master, signer)?;
    runtime.serve_until_stopped(shutdown)
}

5. Remove every direct path to the credential

After the brokered composition validates, remove the matching credential from the tool's environment and filesystem grants, and confirm the tool has no other network route to the provider. A copied environment variable holding the same secret would be a path around the broker. The operating-system administrator stays trusted for broker memory and key custody, and the provider receives the injected credential, so scope the provider's own permissions to the task as well.

The socket lifecycle survives a crash without deleting another service's endpoint

Startup takes an exclusive, owner-checked lifecycle lock before binding the private endpoint. Shutdown rechecks identity and unlinks only the socket device and inode the daemon created, and an unknown pre-existing path stays where it is.

After an uncatchable termination, the supervisor recovers the stale socket while the daemon is stopped: it takes the same lock, verifies the socket's type, owner, and permissions, removes that exact identity, and restarts. The reference unit leaves this to systemd, which removes the service's runtime directory, socket and lock included, whenever the service stops. A missing final response stays tied to its original attempt after readiness returns.

A restore brings back the attempt history with the encrypted material

The broker remembers how each credential has been used. Its attempt history tells an original request from another use, and its administrative replay state records consumed control authority. Restore last night's encrypted credential after a request was accepted this morning, and the credential decrypts while the attempt store has no record of the request: a fresh submission could repeat the effect. The custody set and its rollback checks keep a readable backup from becoming a more permissive service.

Restore the four databases together with the stores that hold authority. The broker's databases hold no revocation, liveness, or budget state; the broker reads all three from the authority over its RPC socket, and startup moves each restored attempt whose hold the authority denied, reversed, or captured to match that outcome. A restored broker database cannot re-enable a revoked grant or restore captured budget. Protect durable state gives the restore order, and the client endpoint opens only after the restore is tested.

A restored database opened under a different tenant key or signing identity fails custody or receipt verification. Treat that as a restore mismatch and keep the old attempt ledger; starting from empty state discards the history that prevents a repeat.

A restored configuration is checked before any store opens

Run this read-only comparison as the intended service UID against the retained configuration. It validates the original, then confirms that validation refuses an aliased attempt and receipt path, a broker key standing in for the independent auditor, and a revocation-age ceiling above sixty seconds. It changes only in-memory copies.

use chio_secret_broker::daemon_runtime::BrokerDaemonConfig;
use chio_secret_broker::{BrokerError, Result};
use std::path::Path;

/// Run as the intended service UID using its retained private runtime config.
/// This validates configuration only; opening/restoring the databases is the
/// daemon's separate custody and reconciliation work.
pub fn compare_restore_configuration(path: &Path) -> Result<()> {
    let original = BrokerDaemonConfig::load(path)?;
    original.validate()?;
    let mut aliased_store = original.clone();
    aliased_store.databases.attempt_database_path =
        original.databases.receipt_database_path.clone();
    if aliased_store.validate().is_ok() {
        return Err(BrokerError::Invariant(
            "aliased stores unexpectedly accepted".into(),
        ));
    }
    let mut aliased_auditor = original.clone();
    aliased_auditor.trusted_audit_runner = original.broker_identity.clone();
    if aliased_auditor.validate().is_ok() {
        return Err(BrokerError::Invariant(
            "broker trusted as independent auditor".into(),
        ));
    }
    let mut stale_authority = original;
    stale_authority.maximum_revocation_snapshot_age_seconds = 61;
    if stale_authority.validate().is_ok() {
        return Err(BrokerError::Invariant(
            "oversized revocation age accepted".into(),
        ));
    }
    Ok(())
}

Download broker_restore_checks.rs. A passing original means the closed contract is coherent under this service UID; readiness still needs store custody and authority reconciliation.

Two database paths alias

Separate history roles no longer have separate paths. Restore the correct retained stores, never empty replacements.

The auditor key equals the broker identity

Independent audit authority has collapsed into the broker. Restore the independently provisioned audit identity.

Master key, signing key, or file identity fails custody

The opened custody set differs from the daemon's retained contract. With the service stopped, inspect the exact keys, the owner, mode, and inode, and the anchors, and preserve the original history.

Runtime configuration

The complete field contract, the nested store declaration, and the bounds the daemon validates at startup.

pub struct BrokerDaemonConfig {
    pub schema: String,
    pub deployment_id: String,
    pub broker_instance_id: String,
    pub tenant_scope: String,
    pub audit_runner_id: String,
    pub trusted_audit_runner: PublicKey,
    pub ipc_socket_path: PathBuf,
    pub authority_socket_path: PathBuf,
    pub trusted_capability_issuer: PublicKey,
    pub trusted_authority: PublicKey,
    pub broker_identity: PublicKey,
    pub broker_audience: String,
    pub parent_audience: String,
    pub provider_adapter_id: String,
    pub provider_adapter_version: u32,
    pub provider_placement: ProviderPlacementConfig,
    pub trusted_service_uid: u32,
    pub authorized_client_uid: u32,
    pub ipc_read_timeout_ms: u64,
    pub ipc_write_timeout_ms: u64,
    pub authority_timeout_ms: u64,
    pub maximum_clock_skew_seconds: u64,
    pub maximum_liveness_snapshot_age_seconds: u64,
    pub maximum_revocation_snapshot_age_seconds: u64,
    pub databases: BrokerDaemonDatabaseConfig,
    pub enterprise_migration: BrokerDaemonMigrationConfig,
    pub admin: BrokerDaemonAdminConfig,
    pub privileged_audit: BrokerDaemonPrivilegedAuditConfig,
}

pub struct BrokerDaemonDatabaseConfig {
    pub secret_database_path: PathBuf,
    pub attempt_database_path: PathBuf,
    pub admin_replay_database_path: PathBuf,
    pub receipt_database_path: PathBuf,
}

Next: harden the host around the broker