Key log witnesses
Run the three witnesses and two audit monitors that sign key-log checkpoints, catch split views, and gate rotation.
On this page
Three key-log witnesses and two audit monitors run beside trust control, each with its own key, store, and socket, so no single service can vouch for a key change.
Two witness signatures make a checkpoint witnessed, both monitors confirm before a rotation activates, and trust control opens only after all five answer a fresh readiness challenge. The steps use the reference runtime's systemd templates on one Linux host.
StartTwo of three witnesses make a checkpoint witnessed
The operator signs a checkpoint: one log position and one tree root. Each witness checks it against its own retained history before signing, and signatures from w₁ and w₂ make it witnessed. The audit monitors take no part in this quorum. Key transparency.
1. Author a policy with a separate root for every role
Install the key-log policy as /etc/chio/keylog/policy.json. It names these roots:
- Bootstrap. One public key that authorizes genesis.
- Operator. One public key, with a mode-0600 Ed25519 seed file, that signs checkpoints, activations, and enterprise receipts. Losing that seed stops new checkpoints and activations.
- Witnesses. Exactly three public keys, each paired with its own socket and SQLite database.
- Auditors. Exactly two public keys, committed into the configuration binding through
chio.key-log.auditor-policy-binding.v1. - Artifact time. A nonempty set of public keys.
- Recovery. Optional public keys and a threshold, when recovery authorization is enabled.
Validation refuses duplicate roots, a key aliased across roles, two services sharing a durable storage identity, stale readiness challenges, and a topology map that differs from the policy-owned roots. A witness signs only a checkpoint that extends the history it already holds, and a monitor compares what the witnesses saw. Separate keys and stores let one service catch another's divergence, and separate custody keeps one compromised account from rewriting every history.
2. Give each service its own seed and configuration
As root, from the directory where you extracted the release archive, write a 32-byte seed and a configuration for each instance: witnesses a, b, c and monitors a, b.
for instance in a b c; do
head -c 32 /dev/urandom > /etc/chio/credentials/keylog-witness-$instance.seed
sed "s/witness-a/witness-$instance/g; s/witness@a/witness@$instance/g; \
s/\"witness_id\": \"a\"/\"witness_id\": \"$instance\"/" \
reference-runtime/keylog/witness-a.json.example \
> /etc/chio/keylog/witness-$instance.json
done
for instance in a b; do
head -c 32 /dev/urandom > /etc/chio/credentials/keylog-audit-$instance.seed
sed "s/audit-a/audit-$instance/g; s/audit@a/audit@$instance/g; \
s/\"monitor_id\": \"a\"/\"monitor_id\": \"$instance\"/" \
reference-runtime/keylog/audit-a.json.example \
> /etc/chio/keylog/audit-$instance.json
done
chgrp chio-keylog /etc/chio/keylog/*.json && chmod 0640 /etc/chio/keylog/*.jsonEach configuration's seed_file_path is the credentials path systemd mounts for that unit, and its socket_path sits in the unit's runtime directory, which systemd removes on stop. provision: true lets the first start create the database; set it to false afterwards so a missing database fails startup. A monitor opens the operator database read-only.
3. Start the five services and confirm each is ready
systemctl enable --now \
chio-keylog-witness@{a,b,c}.service chio-keylog-audit@{a,b}.service
systemctl is-active chio-keylog-witness@{a,b,c}.service chio-keylog-audit@{a,b}.serviceactive active active active active
You should see five active lines. Each unit is Type=notify and starts through chio security supervise, which reports ready only once the service's socket answers, within a 30-second timeout. The units run with a private network and Unix sockets only.
Then start trust control with --authority-keyring-config. It sends every service a fresh readiness challenge and opens its HTTP listener only after five signed answers; the rotation command lists the required flags. The templates share the chio-keylog account, so distinct sockets and stores do not isolate the processes from each other: run separate accounts or hosts where custody must be independent, as service account isolation describes.
Witnesses accept a longer log only with a consistency proof
When a witness or monitor learns of a checkpoint that extends its retained log, it requires an RFC 6962 consistency proof between the old tree and the new one. verify_consistency_proof checks the proof against both roots and both tree sizes; a growing synchronization without one is refused with InvalidCheckpoint.
The keyring runtime reaches each witness over its own socket. synchronize_witness_to_candidate pages through the store's synchronization response and asks the witness to sign each candidate in turn, as a SignCandidate IPC operation. POST /v1/authority/key-log/sync is a separate, read-only export of the witnessed history for holders of the service token.
Artifact time is anchored to a checkpoint as well. A signing epoch alone leaves open when an artifact was produced, so SignedArtifactTimeAnchor binds the artifact's hash to a signed anchored_at time, and ArtifactTimeAnchorKind ties that anchor to a receipt checkpoint, a key-log checkpoint, or an external commitment. A verifier checks the checkpoint the artifact was produced against, in place of the artifact's own timestamp.
A split view leaves two signed checkpoints as evidence
A monitor relays signed checkpoints between witnesses on its own polling schedule. The receiving witness checks the relayed signature and compares the checkpoint with its retained views at the same sequence number or tree size. Different roots at one position are a conflict: the store keeps both signed views and returns KeyringError::EquivocationDetected.
pub struct CheckpointEquivocationEvidence {
pub schema: String,
pub kind: CheckpointConflictKind,
pub first: SignedKeyLogCheckpoint,
pub conflicting: SignedKeyLogCheckpoint,
pub detected_at: u64,
}The evidence pairs the two signed checkpoints with the kind of conflict and the detection time, under schema chio.key-log.equivocation.v1. Its only wire form is this struct, so a consumer checks the same signed checkpoints a witness already verifies. Preserve the pair for investigation; overwriting the first row destroys the proof.
Two signatures count only when two witnesses sign one checkpoint
The portable comparison configures three distinct policy-owned witness keys, refuses one signature and a duplicate of that same signer, accepts two distinct signatures on the same checkpoint, and refuses to move them to another root. Call compare_witness_quorum() from a host that depends on chio-keyring and chio-core-types, or download witness_quorum_cases.rs.
use chio_core_types::{sha256, Ed25519Backend, Keypair, SigningBackend};
use chio_keyring::{
KeyLogCheckpointBody, LogId, SignedKeyLogCheckpoint, WitnessId, WitnessSignature,
KEY_LOG_CHECKPOINT_SCHEMA,
};
use std::collections::BTreeMap;
/// Independent keys in one portable fixture are not independent deployed
/// services. This tests exact checkpoint agreement and unique quorum counting.
pub fn compare_witness_quorum() -> Result<(), Box<dyn std::error::Error>> {
let operator = Ed25519Backend::new(Keypair::generate());
let mut checkpoint = SignedKeyLogCheckpoint::sign(
KeyLogCheckpointBody {
schema: KEY_LOG_CHECKPOINT_SCHEMA.into(),
log_id: LogId::new("larkfield-authority")?,
checkpoint_sequence: 2,
tree_size: 2,
root_hash: sha256(b"fixture: verified two-event root"),
previous_checkpoint_hash: None,
issued_at: 1_800_000_001,
},
&operator,
)?;
let witnesses = [
Ed25519Backend::new(Keypair::generate()),
Ed25519Backend::new(Keypair::generate()),
Ed25519Backend::new(Keypair::generate()),
];
let ids = [
WitnessId::new("east")?,
WitnessId::new("north")?,
WitnessId::new("west")?,
];
let pins = ids
.iter()
.cloned()
.zip(witnesses.iter().map(|w| w.public_key()))
.collect::<BTreeMap<_, _>>();
checkpoint.witness_signatures.push(WitnessSignature::sign(
&checkpoint,
ids[0].clone(),
&witnesses[0],
)?);
assert!(checkpoint.verify_witnesses(&pins).is_err());
let first = checkpoint.witness_signatures[0].clone();
checkpoint.witness_signatures.push(first);
assert!(checkpoint.verify_witnesses(&pins).is_err());
println!("REFUSE one-of-three and two copies of the same witness");
checkpoint.witness_signatures.pop();
checkpoint.witness_signatures.push(WitnessSignature::sign(
&checkpoint,
ids[1].clone(),
&witnesses[1],
)?);
assert_eq!(checkpoint.verify_witnesses(&pins)?.len(), 2);
println!("VERIFY two distinct witnesses on the same checkpoint");
checkpoint.body.root_hash = sha256(b"fixture: a different tree");
assert!(checkpoint.verify_witnesses(&pins).is_err());
println!("REFUSE signatures transplanted onto another root");
Ok(())
}REFUSE one-of-three and two copies of the same witness VERIFY two distinct witnesses on the same checkpoint REFUSE signatures transplanted onto another root
You should see one refusal, one acceptance, and a second refusal. The two accepted signatures cover one operator checkpoint statement, so changing the root invalidates both, and a duplicate identity fails before it can inflate a majority. All three keys live in one process here: the test establishes signature and quorum semantics, and three keys in one executable are a fixture, never the production topology.
An outage and a split view stop progress for different reasons
Readiness reflects the complete configured topology, and a refusal names the check that failed:
| Refusal | Condition |
|---|---|
Keyring | Two signed checkpoints disagree about the same log; a monitor also reports any gossip import a witness refuses |
Invalid "growing synchronization omitted consistency proof" | A synchronization extended the log without proving the extension |
Invalid | Untrusted anchor root, bad anchor signature, future-dated anchor, or artifact hash or key mismatch |
Before maintenance, identify which services and stores go offline, schedule a rotation apart from a storage replacement, and run fresh readiness challenges before trust control serves authority operations again.
A witness is down
With two witnesses running, checkpoints still reach witnessed status and a rotation can still activate, but trust control cannot restart: startup needs signed readiness from all five services. With two witnesses down, no checkpoint reaches a majority and activation stops. Restore the service with its own retained history and identity; history it signed before the outage stays valid evidence. The operator never counts itself as a missing witness, and policy roots never change in place.
An audit monitor is down
Checkpoint signing and the other monitor's polling continue. Activation and the production readiness report wait for both monitors, so a rotation stays pending with its checkpoint and witness signatures retained until the monitor returns. The rotation then passes its final activation guard; retain its activation commit, signing epoch, and audit pin.
EquivocationDetected is reported
Treat it as a conflicting history, separate from an outage. Record the last accepted checkpoint, the pending operator head, the activated epoch, each witness response, and any monitor readiness failure, and keep both signed views for the investigation.
Service units, configuration, and test cases
Search for socket, provision, poll, or split_view.
Service units
| Service | Unit | Role |
|---|---|---|
chio- | chio- | Signs checkpoints it has verified as consistent |
chio- | chio- | Polls the three witnesses and reports divergent views |
Both templates run as the chio-keylog system account and start through chio security supervise, which wraps the chio-keylog-witness and chio-keylog-audit binaries with --ready-unix-socket, --ready-timeout 30, and --stop-grace 10. Instances load their seed with LoadCredential=seed:/etc/chio/credentials/keylog-witness-%i.seed (or keylog-audit-%i.seed) and keep state under /var/lib/chio-keylog/witness-%i or audit-%i. Cryptographic independence comes from the distinct policy-owned keys and histories, and the unit names add none.
Witness configuration
{
"schema": "chio.key-log.witness-service-config.v1",
"policy_path": "/etc/chio/keylog/policy.json",
"database_path": "/var/lib/chio-keylog/witness-a/witness.sqlite3",
"socket_path": "/run/chio-keylog/witness-a/witness.sock",
"witness_id": "a",
"seed_file_path": "/run/credentials/chio-keylog-witness@a.service/seed",
"provision": true
}Audit monitor configuration
A monitor polls the three witness sockets every poll_interval_millis and keeps its own database of what it observed, beside a read-only view of the operator database. The operator store holds a writer lock for its single writer; observers open it without that lock.
{
"schema": "chio.key-log.audit-service-config.v1",
"policy_path": "/etc/chio/keylog/policy.json",
"database_path": "/var/lib/chio-keylog/audit-a/audit.sqlite3",
"operator_database_path": "/var/lib/chio-keylog/audit-a/operator.sqlite3",
"socket_path": "/run/chio-keylog/audit-a/audit.sock",
"monitor_id": "a",
"seed_file_path": "/run/credentials/chio-keylog-audit@a.service/seed",
"witness_sockets": {
"a": "/run/chio-keylog/witness-a/witness.sock",
"b": "/run/chio-keylog/witness-b/witness.sock",
"c": "/run/chio-keylog/witness-c/witness.sock"
},
"poll_interval_millis": 5000,
"provision": true
}Source locations
verify_consistency_proof is defined at chio-core-types/src/merkle.rs:332. The sync route is AUTHORITY_KEY_LOG_SYNC_PATH in chio-control-plane/src/trust_control/service_types/paths.rs:11. SignedArtifactTimeAnchor is at chio-keyring/src/time.rs:59 and ArtifactTimeAnchorKind at line 33 of the same file.
Witness test cases
The adversarial suite in crates/core/chio-adversarial-suite/cases/ carries four cases built against this topology: key_log_split_view, key_log_omission, key_log_noncontiguous_sync, and key_log_inconsistent_growth. Each is a chio.adversarial-mutation-evidence.v1 record pairing a control test from the keyring suite with a mutant, a specific change to the function under test, that the control test must catch and deny for a divergent or incomplete key-log history.
On the host that runs the trust control plane, the keyring suite runs twelve test groups against the installed binaries: the RFC 6962 Merkle vectors and eleven keyring groups, covering checkpoints, witnesses, and audit.
chio security selftest keyring --evidence ./evidencesuite result keyring pass RFC 6962 vectors, checkpoint, witness, and audit cases 1 suite passed; evidence in ./evidence