Shadow migration

Your existing application ends this procedure fully inventoried, with every converted manifest re-signed by your authority and its cohort carried by signed transitions from Disabled through Shadow to Enforced.

Migration starts from what the application already holds: live grants, direct credentials, unresolved operations, and sessions that read data before flow tracking began. History with no evidence takes its most restrictive meaning, and each posture change is a signed transition linked to the retained head, which configuration or restart cannot skip.

1. Inventory every tool, credential, and history record

The input follows the closed chio.active-defense.shadow-migration-input.v1 schema. Build it from existing signed artifacts and deployment records, with one deployment record for every tool in each signed manifest, collected from the component that owns each path:

For each tool, inventory runtime egress, policy-owned clearances and purposes, adapter field preservation, direct credential grants, managed or native execution, and the independent cage ceiling for native tools. A partial per-tool inventory is rejected. Nested objects are closed, so review notes go in a companion record that references the input digest.

Resolve every disagreement between a signed declaration and the running adapter before you generate the report. A transport that accepts a field and drops it before dispatch cannot carry that declared contract: repair the adapter, or keep the tool out of the cohort. A successful discovery response is no evidence of field preservation.

2. Generate the migration report

Run chio security shadow-migrate on the complete inventory, then read the report's unknown-history assignments, unsigned conversions, and direct credential grants before you change posture. A fixer session that ran before enforcement may hold Harbor Goods' contract terms even when its current prompt holds only issue #417, and the report is where that shows.

chio security shadow-migrate \
  --input active-defense-inventory.json \
  --output active-defense-migration-report.json
jq '{input_digest,
     unsigned_conversions: (.unsigned_v2_artifacts | length),
     direct_credential_grants: (.direct_credential_grants | length),
     unknown_principal_history: .backfill.principal_records_assigned_top,
     unknown_lineage_history: .backfill.lineage_records_assigned_top,
     all_require_resigning: all(.unsigned_v2_artifacts[]; .operator_resigning_required)}' \
  active-defense-migration-report.json

{
  "input_digest": "sha256:9f2c…41ab",
  "unsigned_conversions": 2,
  "direct_credential_grants": 1,
  "unknown_principal_history": 3,
  "unknown_lineage_history": 1,
  "all_require_resigning": true
}

You should see the input digest, two unsigned conversions (the workspace and shell manifests written before v2), one direct credential grant, and the principal and lineage records assigned unknown history. The command verifies the complete input before it replaces the output file, rejecting duplicate JSON members, unknown fields, unsafe numbers, duplicate key or registry identities, invalid manifests, and unverifiable backfill evidence. It writes the canonical report, syncs it, and renames it into place atomically.

Keep the report with the input inventory and the trusted key registrations: it is the verification result for that exact evidence set. Removing a direct credential grant takes a broker transition, since editing its inventory entry leaves the running credential path in place.

3. Re-sign every converted manifest

A signed v1 body is verified in its original canonical form before conversion. The converted v2 artifact is unsigned and records its source and converted digests with operator_resigning_required=true. The migration command takes no signing input, so it never authorizes the converted contract on your behalf.

Review the resulting tool declarations and runtime permissions, then re-sign through the intended authority. Only a verified, signed v2 manifest opts into the flow contract and supports backfill of past labels; an unsigned conversion stays out of every enforced registry.

4. Backfill labels only from verified receipts

Backfill reads the signed active_defense_backfill_v1 receipt metadata, bound to a registered receipt key and to the exact digest of a verified, signed v2 manifest that contains the tool. The receipt signature and parameter hash are checked along with the metadata schema.

Without that evidence the history is Top, and when any inventoried session lacks evidence, its principal or lineage is Top as well. A session that read an internal document before flow tracking stays Top after it closes: a signed session-closed record says nothing about what the agent learned, and a new declaration of what the tool should have returned is no evidence of what the session received.

A new isolated workflow needs an explicit transition that imports none of the old session's unclassified memory or outputs, and a new session identifier carrying the same context is no isolation. The host's isolation and state contract decides what can begin independently.

5. Commit each posture change as a signed transition

A transition binds its predecessor, generation, stages, posture, evidence, authorization, intent, trusted time, and signer:

pub struct EnterpriseMigrationTransitionBody {
    pub signature_domain: String,
    pub schema_version: u8,
    pub key: EnterpriseMigrationKey,
    pub generation: u64,
    pub from_stage: Option<EnterpriseMigrationStage>,
    pub to_stage: EnterpriseMigrationStage,
    pub prior_head_digest: Option<Digest32>,
    pub posture_digest: Digest32,
    pub evidence_digest: Digest32,
    pub authorization_digest: Digest32,
    pub intent_digest: Digest32,
    pub trusted_at_unix_ms: u64,
    pub signer_public_key: String,
}

Before accepting restored history, compare prior_head_digest with the independently retained minimum head. Before serving, compare posture_digest with the running runtime and declaration set, and compare evidence_digest and authorization_digest with the reviewed promotion record. Each stage's gates and signed release record are the ones every cohort shares in promote controls in order.

A signature over one transition authorizes only its own posture, even when another posture carries the same stage name Enforced. Once Enforced commits, a runtime binding that names Shadow, or Enforced with a different posture_digest, fails.

The portable fixture exercises the signed transition and SQLite store APIs for larkfield-deployment. It moves the cohort from Disabled to Shadow to Enforced, checks the runtime binding at each boundary, and reopens the store with an independently retained minimum head. Call compare_migration_history(path) only with a fresh private database; its evidence and authorization hashes are synthetic and qualify no traffic.

use chio_core_types::{sha256, Keypair};
use chio_security_types::ports::{Digest32, RecordId};
use chio_security_types::{
    EnterpriseMigrationCasOutcome, EnterpriseMigrationControl, EnterpriseMigrationKey,
    EnterpriseMigrationRegisterOutcome, EnterpriseMigrationScopeKind, EnterpriseMigrationStage,
    EnterpriseMigrationState, EnterpriseMigrationStateStore, EnterpriseMigrationTransition,
    EnterpriseMigrationTransitionBody,
};
use chio_store_sqlite::{
    sign_enterprise_migration_transition, SqliteEnterpriseMigrationOpenPolicy,
    SqliteEnterpriseMigrationStateStore,
};
use std::path::Path;

fn digest(label: &[u8]) -> Digest32 {
    Digest32::new(*sha256(label).as_bytes())
}

/// Construct once from the reviewed predecessor. The production owner durably
/// retains this signed value before calling promote_original or retrying it.
pub fn prepare_original_transition(
    signer: &Keypair,
    prior: &EnterpriseMigrationState,
    posture: Digest32,
    evidence: Digest32,
    authorization: Digest32,
    intent: Digest32,
    trusted_at_unix_ms: u64,
) -> Result<EnterpriseMigrationTransition, Box<dyn std::error::Error>> {
    let body = EnterpriseMigrationTransitionBody::promotion(
        prior,
        posture,
        evidence,
        authorization,
        intent,
        trusted_at_unix_ms,
        signer.public_key().to_hex(),
    )?;
    Ok(sign_enterprise_migration_transition(body, signer)?)
}

/// Keep the borrowed original even on a lost acknowledgment or port error.
/// Conflict returns retained state and never silently rebases the signed intent.
pub fn promote_original(
    store: &SqliteEnterpriseMigrationStateStore,
    retained_original: &EnterpriseMigrationTransition,
) -> Result<EnterpriseMigrationCasOutcome, Box<dyn std::error::Error>> {
    Ok(store.compare_and_promote(retained_original)?)
}

/// Signed transition/store fixture, never a qualified promotion of a real cohort.
/// The named digests stand for fixture evidence; production first meets rollout gates.
pub fn compare_migration_history(path: &Path) -> Result<(), Box<dyn std::error::Error>> {
    if path.exists() {
        return Err("migration database must be fresh".into());
    }
    let signer = Keypair::generate();
    let trusted_pin = signer.public_key();
    let key = EnterpriseMigrationKey {
        deployment_id: RecordId::new("larkfield-deployment")?,
        scope_kind: EnterpriseMigrationScopeKind::Provider,
        scope_id: RecordId::new("code-host")?,
        control: EnterpriseMigrationControl::BrokerCredentialCustody,
    };
    let store = SqliteEnterpriseMigrationStateStore::open(
        path,
        SqliteEnterpriseMigrationOpenPolicy::new(vec![trusted_pin.clone()], vec![])?,
    )?;
    let genesis = sign_enterprise_migration_transition(
        EnterpriseMigrationTransitionBody::genesis(
            key.clone(),
            digest(b"disabled fixture posture"),
            digest(b"genesis fixture evidence"),
            digest(b"genesis fixture authorization"),
            digest(b"genesis fixture intent"),
            1_800_000_000_000,
            signer.public_key().to_hex(),
        )?,
        &signer,
    )?;
    let disabled = match store.register(&genesis)? {
        EnterpriseMigrationRegisterOutcome::Registered(state) => state,
        _ => return Err("unexpected existing cohort".into()),
    };
    let shadow_transition = prepare_original_transition(
        &signer,
        &disabled,
        digest(b"shadow fixture posture"),
        digest(b"shadow fixture evidence"),
        digest(b"shadow fixture authorization"),
        digest(b"shadow fixture intent"),
        1_800_000_001_000,
    )?;
    let shadow = match promote_original(&store, &shadow_transition)? {
        EnterpriseMigrationCasOutcome::Promoted(state) => state,
        _ => return Err("shadow predecessor conflict".into()),
    };
    assert_eq!(shadow.stage, EnterpriseMigrationStage::Shadow);
    // Configuration alone cannot skip the next signed transition.
    assert!(shadow
        .runtime_binding(
            EnterpriseMigrationStage::Enforced,
            digest(b"enforced fixture posture")
        )
        .is_err());
    let enforced_transition = prepare_original_transition(
        &signer,
        &shadow,
        digest(b"enforced fixture posture"),
        digest(b"enforced fixture evidence"),
        digest(b"enforced fixture authorization"),
        digest(b"enforced fixture intent"),
        1_800_000_002_000,
    )?;
    let enforced = match promote_original(&store, &enforced_transition)? {
        EnterpriseMigrationCasOutcome::Promoted(state) => state,
        _ => return Err("enforced predecessor conflict".into()),
    };
    assert!(enforced
        .runtime_binding(EnterpriseMigrationStage::Shadow, shadow.posture_digest)
        .is_err());
    assert!(enforced
        .runtime_binding(EnterpriseMigrationStage::Enforced, shadow.posture_digest)
        .is_err());
    let minimum = enforced.minimum_head(); // retain independently outside the restored database
    drop(store);
    let restored = SqliteEnterpriseMigrationStateStore::open(
        path,
        SqliteEnterpriseMigrationOpenPolicy::new(vec![trusted_pin], vec![minimum])?,
    )?;
    let original = restored.load(&key)?.ok_or("missing original cohort")?;
    assert_eq!(original, enforced);
    original
        .runtime_binding(EnterpriseMigrationStage::Enforced, enforced.posture_digest)
        .map_err(|error| format!("restored runtime binding refused: {error:?}"))?;
    println!("RESTORE original signed cohort: Disabled -> Shadow -> Enforced, same retained head");
    println!("REFUSE uncommitted advance, stage downgrade and substituted posture");
    Ok(())
}

RESTORE original signed cohort: Disabled -> Shadow -> Enforced, same retained head
REFUSE uncommitted advance, stage downgrade and substituted posture

You should see two lines: RESTORE original signed cohort through all three stages with the same retained head, then REFUSE uncommitted advance, stage downgrade and substituted posture. For a negative test of your own, substitute the runtime posture or predecessor in a copy of a valid transition and confirm that verification rejects it; for restart, confirm that the new serving owner loads its latest permitted state from the unchanged chain.

A failed transition keeps the last valid stage

Preserve the original input, the canonical report, the converted unsigned manifests, and newly reviewed signatures as separate artifacts; a report failure names the inconsistent evidence set before the output is replaced. A cohort that fails promotion stays at its last valid stage while you repair the declaration, adapter, authority, or recovery issue, and enforced work keeps its protected failure behavior. Disabling the mechanism while the same tools stay reachable is a change of authority.

In an embedding, durably retain the value from prepare_original_transition before passing it to promote_original, which borrows it so an error cannot discard it. A compare-and-promote conflict returns retained state: inspect its predecessor before deciding whether a new review is needed, and keep the signed intent unchanged after a lost acknowledgment. For backup and restore, keep the signer registration and minimum head outside the restored database, so reopening a valid but stale prefix cannot erase that floor.

pub enum EnterpriseMigrationStage {
    Disabled,
    Shadow,
    Enforced,
    LegacyRemoved,
}

Stages are durable state. Changing an enforced runtime's manifest, signer, permissions, resource grants, broker binding, or ledger configuration needs a new transition bound to the changed contract. After each change, compare the running manifest set, policy, broker binding, resource grants, and durable posture record with the reviewed inputs, then run the hardening checks a newly enforced cohort passes.

Next: measure the cohort before enforcement