Chio/Docs
LOGIN · JOIN

PlatformFleet Operations

Cluster

Kubernetes Admission

Use a ChioPolicy CRD and fail-closed admission webhooks to check signed capabilities before Kubernetes admits a pod.

Two Go modules back this page. github.com/backbay-labs/chio-k8s-webhooks (sdks/k8s/webhooks/) is the admission server: it holds the handlers, the capability verifier, and the two admission-configuration manifests. github.com/backbay-labs/chio-k8s-controller (sdks/k8s/controller/) is the Job reconciler. The CRD itself sits beside them at sdks/k8s/crds/.

Policy and admission verification

ChioPolicy is the declarative record an operator authors and kubectl surfaces. The webhook is the enforcement point, and it reads nothing from the CRD: its trust set and required-scope list come from its own environment, and a namespace opts into enforcement with a label. The two use the same canonical scope grammar, so what a policy declares and what the webhook enforces are written the same way.
rendering
The route a pod CREATE takes in a namespace labelled for enforcement. VerifyCapability decides, and a denial carries its reason back through the API server to the client.
sourcesdks/k8s/webhooks/validating-webhook.yaml:20-31sdks/k8s/webhooks/server.go:53-136at fe56570

The ChioPolicy CRD

ChioPolicy is a namespaced custom resource in the chio.world API group, served at v1alpha1. It declares the scopes pods in a namespace must present, an enforcement mode, an optional selector, and default sidecar wiring for matching pods.

chiopolicy.yamlyaml
apiVersion: chio.world/v1alpha1
kind: ChioPolicy
metadata:
  name: payments-guardrail
  namespace: payments
spec:
  # Every admitted pod must present a capability covering all of these.
  requiredScopes:
    - "ledger:invoke"
    - "resource:receipts/*:read"
  # Optional: narrow the policy to matching pods. Omit to cover the namespace.
  selector:
    matchLabels:
      chio.world/governed: "true"
  # enforce (default) | audit | disabled
  enforcement: enforce
  sidecarConfig:
    image: ghcr.io/backbay-labs/chio-sidecar:latest
    upstream: http://localhost:8080
    specPath: /etc/chio/openapi.yaml
    receiptStore: /var/lib/chio/receipts.db
    autoInject: false

receiptStore is a path, not a URI

The CRD types the field as a free string described as a “receipt storage backend URI”, and nothing validates it. The flag it feeds is not a URI: --receipt-store is an Option<PathBuf> documented as an “optional SQLite receipt store path”, and the sidecar opens the file it names. A value with a scheme in front of it would be taken as a relative path. The ECS and Cloud Run manifests both pass a filesystem path here.

Install the definition before applying any policy. The CRD ships in the Chio repository at sdks/k8s/crds/chiopolicy-crd.yaml:

bash
kubectl apply -f sdks/k8s/crds/chiopolicy-crd.yaml
kubectl get chiopolicy -n payments

The resource is chiopolicies.chio.world, plural chiopolicies, singular chiopolicy, short name chiop. Three additional printer columns put the policy on one line of kubectl get output: Scopes from .spec.requiredScopes, Enforcement from .spec.enforcement, and Age from .metadata.creationTimestamp. The status subresource carries observedGeneration and a standard conditions array.

FieldTypeMeaning
spec.requiredScopesstring[] (required, min 1)Canonical Chio scopes every governed pod's capability must cover.
spec.selectorlabel selectormatchLabels / matchExpressions (In, NotIn, Exists, DoesNotExist). Omitted means the whole namespace.
spec.enforcementenum, default enforceenforce rejects non-compliant pods; audit admits but logs violations; disabled skips enforcement.
spec.sidecarConfig.imagestringContainer image for the Chio sidecar governing matching pods.
spec.sidecarConfig.upstreamstringDefault upstream URL for the sidecar proxy.
spec.sidecarConfig.specPathstringPath to the OpenAPI spec the sidecar enforces.
spec.sidecarConfig.receiptStorestringTyped in the CRD as a “receipt storage backend URI”. The sidecar flag it feeds takes a SQLite file path.
spec.sidecarConfig.autoInjectbool, default falseInject the sidecar into matching pods without a per-pod annotation.

Install the webhooks

Enforcement lives in two admission configurations, both shipped under sdks/k8s/webhooks. The validating webhook decides admission; the mutating webhook applies the same decision at CREATE so a mutation pass can never become a bypass.

ConfigurationWebhookPathOperations
ValidatingWebhookConfigurationvalidate.chio.world/validateCREATE, UPDATE
MutatingWebhookConfigurationmutate.chio.world/mutateCREATE

Both target pods at Namespaced scope, register only v1 in admissionReviewVersions, declare sideEffects: None, and set a 10-second timeout. Both use failurePolicy: Fail: if the webhook is unreachable, admission is denied rather than waved through. The mutating configuration adds reinvocationPolicy: IfNeeded, so the API server may call it again after another mutating webhook has changed the object.

sdks/k8s/webhooks/validating-webhook.yamlyaml
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  name: chio-validating-webhook
  labels:
    app.kubernetes.io/name: chio-admission-controller
    app.kubernetes.io/component: validating-webhook
webhooks:
  - name: validate.chio.world
    admissionReviewVersions:
      - v1
    clientConfig:
      service:
        name: chio-admission-controller
        namespace: chio-system
        path: /validate
        port: 443
      # caBundle should be injected by cert-manager or manually set.
      # caBundle: <base64-encoded-ca-cert>
    rules:
      - apiGroups: [""]
        apiVersions: ["v1"]
        operations: ["CREATE", "UPDATE"]
        resources: ["pods"]
        scope: Namespaced
    failurePolicy: Fail
    sideEffects: None
    timeoutSeconds: 10
    namespaceSelector:
      matchLabels:
        chio.world/enforce: "true"

The webhook serves TLS on port 8443 behind a chio-admission-controller Service in the chio-system namespace, fronted by the Service's port 443. Provision the serving certificate the way you provision any admission-webhook cert. The manifests leave the caBundle for cert-manager to inject, or set it by hand. See Secrets & Signing Keys for cert and key handling.

The mutating webhook emits no patch

In the shipped binary the mutating handler runs the identical verification and returns an allow with no patch. It never relaxes the check: if the validating webhook would deny, the mutating webhook denies too. No component under sdks/k8s/ reads spec.sidecarConfig, so a deployment that wants the sidecar attached wires it in the pod template or in its own injector rather than expecting this webhook to do it.

Opt in with namespace labels

A namespace only reaches the webhooks when it carries the matching label. The validating webhook fires on chio.world/enforce: "true"; the mutating webhook fires on chio.world/inject: "true". Unlabeled namespaces are untouched, so rollout is namespace by namespace.

bash
$ kubectl label namespace payments chio.world/enforce=true
namespace/payments labeled

Configure the trust set

The webhook reads its verification policy from two environment variables, and only these two. There is no default: an unset or empty value fails closed, so a misconfigured deployment denies every pod rather than admitting on a permissive fallback.

VariableValueEmpty behavior
CHIO_WEBHOOK_TRUSTED_KERNEL_KEYSComma-separated Ed25519 issuer public keys, hex-encoded.Fail closed.
CHIO_WEBHOOK_REQUIRED_SCOPESComma-separated canonical scopes every capability must cover.Fail closed.

Each trusted key is validated at load time as a well-formed 32-byte Ed25519 public key; a malformed entry stops the process from starting. The required-scope strings share the grammar used in spec.requiredScopes, so a namespace's policy and the webhook's enforced list read the same.

deployment.envyaml
env:
  - name: CHIO_WEBHOOK_TRUSTED_KERNEL_KEYS
    value: "25403c1e...f0,9a7b18d2...c4"   # kernel issuer public keys (hex)
  - name: CHIO_WEBHOOK_REQUIRED_SCOPES
    value: "ledger:invoke,resource:receipts/*:read"
  - name: TLS_CERT_FILE
    value: /etc/chio/tls/tls.crt
  - name: TLS_KEY_FILE
    value: /etc/chio/tls/tls.key

With TLS_CERT_FILE and TLS_KEY_FILE both set the server serves HTTPS with a TLS 1.2 floor; with neither set it logs a warning and serves plaintext for local development only. Override the listen port with PORT; the default is 8443. A GET /healthz endpoint returns ok for liveness and readiness probes.


What the pod carries

A governed pod presents its capability as canonical JSON in the chio.world/capability-token annotation. The token is the standard Chio capability shape: an issuer, a subject, a scope envelope, a validity window, an optional parameter binding, and an Ed25519 signature.

pod.yamlyaml
apiVersion: v1
kind: Pod
metadata:
  name: ledger-writer
  namespace: payments
  labels:
    chio.world/governed: "true"
  annotations:
    chio.world/capability-token: |
      {"id":"cap-019dbbf8-...","issuer":"25403c1e...f0",
       "subject":"25403c1e...f0",
       "scope":{"grants":[{"server_id":"*","tool_name":"ledger",
         "operations":["invoke"]}],
         "resource_grants":[{"uri_pattern":"receipts/*",
         "operations":["read"]}]},
       "issued_at":1776975000,"expires_at":1776978600,
       "parameter_binding":{"operations":["CREATE"],
         "namespaces":["payments"],"resources":["pods"]},
       "signature":"7be63cdb..."}
spec:
  containers:
    - name: app
      image: myapp:latest

The signature covers a canonical serialization of every field except signature itself: keys sorted, no insignificant whitespace. The webhook reconstructs that byte string from the raw annotation, strips the signature field, and verifies against the issuer key, so re-ordering or reformatting the JSON does not change the verified bytes. For the capability model in full, see Capabilities.

Self-asserted policy is never trusted

A pod that sets chio.world/required-scopes or chio.world/exempt on itself is denied. Required scopes come from the webhook's configuration only, and the webhook honors no self-declared exemption: a workload cannot rewrite the operator's policy or opt itself out.

The verification flow

On every admission the webhook decodes the AdmissionReview, extracts the candidate token, and builds an admission context from the request: operation (upper-cased), namespace, and resource. It then runs the full flow, denying on the first failure with a typed reason that Kubernetes surfaces to the caller.

  1. Deny if no trusted kernel keys are configured.
  2. Deny if no required scopes are configured.
  3. Deny if the capability token is missing or malformed.
  4. Deny if the issuer is not in the trusted-key set.
  5. Deny if the Ed25519 signature does not verify.
  6. Deny if the token is not yet valid or has expired.
  7. Deny if the parameter binding does not cover the admission context.
  8. Deny if any required scope is not covered by the token's grants.

The validity window is compared against the request time in Unix seconds: a token is rejected while now < issued_at and once now >= expires_at. A token with no parameter_binding is permitted, since scope coverage carries the authorization, but a binding that is present must match: its operations compare case-insensitively, its namespaces and resources exactly. Empty fields in a binding act as wildcards.

A denial is an AdmissionResponse with allowed: false, the reason under status.message, and code: 403. The handler copies the request's uid onto the response and re-encodes the whole review, so the body on the wire also carries the request it answers. The response half:

json
{
  "apiVersion": "admission.k8s.io/v1",
  "kind": "AdmissionReview",
  "response": {
    "uid": "<echoed from request.uid>",
    "allowed": false,
    "status": {
      "message": "webhook denies admission: capability token does not cover required scope \"ledger:invoke\"",
      "code": 403
    }
  }
}

Scope grammar

Required scopes and the grants inside a capability are written in the same colon-delimited grammar. The webhook accepts three forms and normalizes operation aliases before matching.

FormExpands toExample
<name>:<op>tool scope, any serverledger:invoke
<kind>:<name>:<op>resource or prompt scoperesource:receipts/*:read
tool:<server>:<name>:<op>fully-qualified tool scopetool:srv-files:read_file:invoke

A grant covers a required scope when the server and name patterns match and the operation is present. Patterns are exact, or a trailing * matches by prefix, or a bare * matches anything, so a grant on receipts/* covers a requirement on receipts/2026-07. Operation aliases fold together: call, exec, and execute normalize to invoke; watch to subscribe; result to read_result. The two-part shorthand folds harder than the longer forms: read and write both become invoke there, while in a three- or four-part scope read stays read and write is rejected as an unsupported operation.


Governing batch jobs

Admission gating covers pods at the door. For batch Job workloads that need a grant minted at creation and receipts aggregated across their lifecycle, the Chio Kubernetes controller pairs with the webhook. It watches batch/v1 Jobs labeled chio.world/governed: "true", mints a capability through the sidecar at creation, harvests per-pod receipts, and releases the grant with a signed JobReceipt on completion or failure.

bash
# Controller manifests create the chio-system namespace, RBAC, and Deployment.
$ kubectl apply -f sdks/k8s/controller/config/manager/manager.yaml
$ kubectl apply -f sdks/k8s/controller/config/rbac/role.yaml

The controller ships as ghcr.io/backbay-labs/chio-k8s-controller and reaches the sidecar at CHIO_SIDECAR_URL. Its fail-closed configuration reinforces the webhook: if the sidecar is unreachable at mint time it requeues with backoff and persists no placeholder capability, so the Job's pods stay ungoverned and are rejected at admission rather than starting unsigned.

Where receipts land

The controller aggregates per-pod receipts into a JobReceipt and posts it to the sidecar's receipt store. Those receipts are the same cryptographic artifacts the rest of Chio emits. See Receipts for the format and verification path.

Next steps

  • Capabilities for the token model the annotation carries: scope, delegation, and parameter binding.
  • Trust Model for what a trusted kernel key attests and how issuers are rotated.
  • Sidecar HTTP Service for the sidecar image referenced by spec.sidecarConfig.
  • Secrets & Signing Keys for provisioning the webhook's serving certificate.
Kubernetes Admission · Chio Docs