Chio/Docs
LOGIN · JOIN

BuildOperations

Istio ext_authz

Use Chio as an Istio ext_authz provider to allow or deny matched mesh traffic before Envoy forwards a request.

Where the code lives

examples/istio-ext-authz/. The reference manifests target Istio 1.22+ on Kubernetes 1.28+. See ci-validation.md in the example for static-validation recipes (kubeconform, istioctl analyze) you can run without a live cluster.

What It Shows

  • The chio Envoy ext_authz adapter (chio-envoy-ext-authz) deployed as a cluster-wide Deployment in the chio-system namespace.
  • Chio registered under MeshConfig.extensionProviders with the gRPC backend at chio-sidecar.chio-system.svc.cluster.local:9091.
  • Three AuthorizationPolicy objects: a CUSTOM policy that opts matched workloads into chio, a DENY backstop for unauthenticated requests, and an ALLOW rule that lets kubelet probes through.
  • A demo workload in the agent-tools namespace that opts in via chio.world/secured=true.
  • A test harness that requires an authenticated POST to return HTTP 200 with an x-chio-receipt-id header, and an unauthenticated POST to return 403 from the backstop. The header is a contract on the adapter image you build, not something the ext_authz crate in this repository emits; see what the adapter returns on allow.

Architecture

Envoy in the demo workload pod is configured with the envoyExtAuthzGrpc backend pointing at the Chio sidecar Service. For each matched request, Envoy issues a gRPC CheckRequest against chio. Chio runs the kernel pipeline, signs a receipt, and responds with allow plus header mutations or deny. Envoy then forwards or rejects.

text
Client
   |
   v  POST /tools/hello (with x-chio-capability-token)
+---------------------+
| Envoy sidecar       |   <-- AuthorizationPolicy CUSTOM matches
| (demo-tool pod)     |       and routes ext_authz to chio
+---------------------+
   |  gRPC Authorization/Check (port 9091)
   v
+----------------------------+
| chio-sidecar Deployment    |   chio-system namespace
| chio-envoy-ext-authz       |   Service: chio-sidecar:9091/gRPC, 9090/HTTP
| (kernel pipeline + signer) |   /health on 9090
+----------------------------+
   |  CheckResponse: OK, no response headers,
   |  dynamic metadata chio.verdict = "allow"
   v
+---------------------+
| Envoy sidecar       |   forwards upstream
+---------------------+
   |
   v
demo-tool app (go-httpbin)

Prerequisites

  • Istio 1.22+ (the typed-header forwarding used here stabilized in 1.22).
  • Kubernetes 1.28+ (required for the security.istio.io/v1 and networking.istio.io/v1 GA APIs the policies use).
  • kubectl with cluster access; istioctl 1.22+; curl and awk on the workstation that runs the harness.
  • A dedicated Chio Envoy ext_authz adapter image. The reference manifest uses ghcr.io/backbay-labs/chio-ext-authz:latest as a placeholder. Replace it with the adapter image you built and published. Do not use the generic ghcr.io/backbay-labs/chio-sidecar image: that is the HTTP sidecar and does not expose the gRPC Authorization/Check service on :9091.
  • A capability token for the demo workload (or the demo token chio accepts in shadow mode). Export it as CHIO_DEMO_CAPABILITY_TOKEN before running the harness.

Files

text
examples/istio-ext-authz/
  00-chio-sidecar-deployment.yaml   Namespace, ServiceAccount, Deployment,
                                    ConfigMap, Secret, Service
  01-meshconfig-patch.yaml          IstioOperator overlay; registers chio-ext-authz
  02-authorization-policy.yaml      CUSTOM (opt-in), DENY (backstop), ALLOW (probes)
  03-demo-workload.yaml             Namespace, ServiceAccount, Deployment,
                                    Service, VirtualService (go-httpbin demo)
  README.md
  ci-validation.md                  kubeconform + istioctl analyze recipes
  test-harness.sh                   port-forward + curl harness

Step 1: Deploy the Chio Sidecar

bash
kubectl apply -f examples/istio-ext-authz/00-chio-sidecar-deployment.yaml
kubectl -n chio-system rollout status deploy/chio-sidecar --timeout=120s
kubectl -n chio-system get svc chio-sidecar

The Service is a ClusterIP with two named ports: grpc-ext-authz on 9091 with appProtocol: grpc, and http-health on 9090 with appProtocol: http (00-chio-sidecar-deployment.yaml:206-230). Istio reads only the first; the second is what the three probes use.

Smoke-test the adapter from inside the cluster before wiring Istio at it:

bash
kubectl run chio-smoke --rm -it --restart=Never \
  --image=curlimages/curl:8.9.1 -- \
  curl -sS http://chio-sidecar.chio-system.svc.cluster.local:9090/health

A 200 is what the startup, readiness and liveness probes require on /health (00-chio-sidecar-deployment.yaml:122-146), so a pod that never answers there never becomes ready. That endpoint is part of the contract on the adapter image, not something this repository ships: chio-envoy-ext-authz declares a [lib] and nothing else, with no binary and no HTTP server. The body it returns is yours to choose.

Fail-closed by design

The manifest expects an adapter that exits non-zero when it cannot load its kernel config or policy bundle (00-chio-sidecar-deployment.yaml:11-14). Kubernetes restarts it, the readiness probe on /health keeps failing, and the provider config sends the refusal back as a 403 through statusOnError: "403" (01-meshconfig-patch.yaml:65). An unavailable Chio adapter results in denial.

Step 2: Register Chio in MeshConfig

Pick the install path that matches how your mesh was provisioned.

Fresh install:

bash
istioctl install -y \
  -f examples/istio-ext-authz/01-meshconfig-patch.yaml

Existing mesh managed via IstioOperator:

bash
kubectl -n istio-system apply \
  -f examples/istio-ext-authz/01-meshconfig-patch.yaml
kubectl -n istio-system rollout restart deploy/istiod

The patch registers chio under meshConfig.extensionProviders:

examples/istio-ext-authz/01-meshconfig-patch.yaml34-68yaml
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
  name: chio-ext-authz-meshconfig
  namespace: istio-system
spec:
  # Minimum Istio version. Earlier 1.20/1.21 builds also work for gRPC
  # ext_authz but 1.22 stabilised the typed header forwarding knobs below.
  meshConfig:
    extensionProviders:
      - name: chio-ext-authz
        envoyExtAuthzGrpc:
          # Fully-qualified cluster DNS name of the Service created in
          # 00-chio-sidecar-deployment.yaml.
          service: chio-sidecar.chio-system.svc.cluster.local
          port: 9091
          # Chio evaluation budget. Increase if your guard pipeline is slow.
          timeout: 0.25s
          # Forward identity + capability headers into the CheckRequest.
          # `chio-envoy-ext-authz::translate` pulls identity from these names.
          includeRequestHeadersInCheck:
            - authorization
            - x-chio-capability-token
            - x-chio-session-id
            - x-request-id
          # Chio never stores or forwards raw request bodies, but small
          # bodies must be available for guards that inspect content.
          includeAdditionalHeadersInCheck:
            x-chio-source: istio-mesh
          # Status propagated back to the client on Chio failure. See
          # docs/protocols/ENVOY-EXT-AUTHZ-INTEGRATION.md section 11.3.
          statusOnError: "403"
          # Inject the Envoy downstream peer certificate so Chio can derive a
          # SPIFFE-style CallerIdentity from the mTLS handshake.
          includePeerCertificate: true

Verify the provider was picked up:

bash
kubectl -n istio-system logs deploy/istiod | grep -i extensionprovider
istioctl proxy-config bootstrap -n istio-system deploy/istiod \
  | grep -A3 chio-ext-authz

Step 3: Deploy the Demo Workload and Policy

bash
kubectl apply -f examples/istio-ext-authz/03-demo-workload.yaml
kubectl -n agent-tools rollout status deploy/demo-tool --timeout=120s
kubectl apply -f examples/istio-ext-authz/02-authorization-policy.yaml
kubectl -n agent-tools get authorizationpolicies

Three policies land in agent-tools: chio-tool-authorization (CUSTOM), chio-deny-unauthenticated (DENY), and chio-allow-health-probes (ALLOW). Istio evaluates CUSTOM first, then DENY, then ALLOW. An unauthenticated request matches none of the three CUSTOM rules, so it is never sent to Chio at all; the DENY backstop is what refuses it.

All three select on the same label, chio.world/secured=true, so a workload opts in once and gets all of them. The CUSTOM policy's three rules are ORed: POSTs to /tools/* or /invoke/* with a non-empty x-chio-capability-token, the same POSTs with a Bearer Authorization header, and GETs on /tools/* with one. The whole file, comments included:

examples/istio-ext-authz/02-authorization-policy.yamlyaml
# Route mesh traffic through Chio via Istio's AuthorizationPolicy CUSTOM action.
#
# The CUSTOM action delegates the allow/deny decision to the named provider
# registered in MeshConfig (see 01-meshconfig-patch.yaml). This file contains
# three policies:
#
#   1. chio-tool-authorization     -- opts workloads labelled
#                                    `chio.world/secured=true` in the
#                                    `agent-tools` namespace into Chio
#                                    evaluation, with a header-driven filter
#                                    so only capability-bearing requests pay
#                                    the ext_authz latency.
#   2. chio-deny-unauthenticated   -- fail-closed backstop: anonymous requests
#                                    that slip past the CUSTOM match above
#                                    are denied outright. Ensures there is
#                                    no bypass path when the capability
#                                    header is missing.
#   3. chio-allow-health-probes    -- excludes kubelet probe traffic so the
#                                    demo workload's `/healthz` path stays
#                                    reachable without a capability token.
#
# All policies use `security.istio.io/v1` (GA in Istio 1.22+). The CUSTOM
# action uses the provider:name field -- the deprecated
# `.external.httpAuthorizationService` style is explicitly NOT used.
---
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: chio-tool-authorization
  namespace: agent-tools
  labels:
    app.kubernetes.io/part-of: chio-protocol
spec:
  # Apply only to workloads that opted in by labelling themselves.
  selector:
    matchLabels:
      chio.world/secured: "true"
  action: CUSTOM
  provider:
    name: chio-ext-authz
  rules:
    # Allow-rule #1: route POSTs to /tools/* through Chio when either an
    # Chio capability token header or a Bearer Authorization header is
    # present. This is the hot path for agent tool invocation.
    - to:
        - operation:
            methods: ["POST"]
            paths: ["/tools/*", "/invoke/*"]
      when:
        - key: request.headers[x-chio-capability-token]
          notValues: [""]
    # Allow-rule #2: route POSTs through Chio when the request authenticates
    # with a Bearer token instead of a capability header.
    - to:
        - operation:
            methods: ["POST"]
            paths: ["/tools/*", "/invoke/*"]
      when:
        - key: request.headers[authorization]
          values: ["Bearer *"]
    # Allow-rule #3: route GETs on /tools/* (listings, schema fetches)
    # through Chio only when an Authorization header is present. This
    # illustrates header-based routing -- unauthenticated GETs fall
    # through to the deny policy below.
    - to:
        - operation:
            methods: ["GET"]
            paths: ["/tools/*"]
      when:
        - key: request.headers[authorization]
          values: ["Bearer *"]
---
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: chio-deny-unauthenticated
  namespace: agent-tools
  labels:
    app.kubernetes.io/part-of: chio-protocol
spec:
  selector:
    matchLabels:
      chio.world/secured: "true"
  action: DENY
  rules:
    # Anything hitting /tools/* without either header is denied before it
    # ever reaches Chio. Saves the ext_authz RTT and keeps receipts focused
    # on real authenticated traffic.
    - to:
        - operation:
            paths: ["/tools/*", "/invoke/*"]
      when:
        - key: request.headers[x-chio-capability-token]
          values: [""]
        - key: request.headers[authorization]
          notValues: ["Bearer *"]
---
apiVersion: security.istio.io/v1
kind: AuthorizationPolicy
metadata:
  name: chio-allow-health-probes
  namespace: agent-tools
  labels:
    app.kubernetes.io/part-of: chio-protocol
spec:
  selector:
    matchLabels:
      chio.world/secured: "true"
  action: ALLOW
  rules:
    - to:
        - operation:
            methods: ["GET"]
            paths: ["/healthz", "/readyz", "/metrics"]

Confirm Envoy on the demo pod picked up the provider reference:

bash
POD="$(kubectl -n agent-tools get pod \
  -l app.kubernetes.io/name=demo-tool \
  -o jsonpath='{.items[0].metadata.name}')"
istioctl proxy-config listener -n agent-tools "${POD}" \
  --port 8080 -o json | grep chio-ext-authz

Step 4: Run the Harness

bash
export CHIO_DEMO_CAPABILITY_TOKEN="$(cat ~/.chio/demo.token)"
./examples/istio-ext-authz/test-harness.sh

The harness:

  1. Opens a kubectl port-forward to svc/demo-tool:80.
  2. Sends POST /tools/hello with x-chio-capability-token and an authorization: Bearer header, and requires HTTP 200 with a non-empty x-chio-receipt-id response header.
  3. Sends the same request without credentials and asserts HTTP 403.

On success it prints a four-line summary built from ARTIFACT_DIR, RECEIPT_ID and DENY_STATUS (test-harness.sh:142-148). The receipt id it echoes is whatever the adapter put on the header; a Chio receipt id is 64 lowercase hex characters (crates/core/chio-core-types/src/receipt/body.rs:240-243).


Manual curl Verification

bash
kubectl -n agent-tools port-forward svc/demo-tool 18080:80 &

# Allow path: the CUSTOM rule matches, so chio evaluates
curl -i -X POST \
  -H "x-chio-capability-token: ${CHIO_DEMO_CAPABILITY_TOKEN}" \
  --data '{"hello":"world"}' \
  http://127.0.0.1:18080/tools/hello

# Deny path: no header, so no CUSTOM rule matches and the DENY policy fires
curl -i -X POST --data '{}' http://127.0.0.1:18080/tools/hello

What the adapter returns on allow

On an allow the ext_authz crate in this repository returns an OkHttpResponse with headers, headers_to_remove and response_headers_to_add all empty (crates/protocol/chio-envoy-ext-authz/src/response.rs:24-37). Its only allow-side output is one Envoy dynamic-metadata field, chio.verdict = "allow" (src/metadata.rs:11-13), which reaches access logs and downstream filters and is not an HTTP header. So a 200 through this adapter carries no x-chio-* header of its own.

The harness asserts a header this crate does not set

test-harness.sh fails the run when the allow response has no x-chio-receipt-id header. If you want the receipt id on the wire, the adapter image you build has to add it on the allow path, or a downstream Envoy filter has to lift chio.verdict and the receipt id out of dynamic metadata. Either way it is your adapter's behaviour, not this crate's.

The deny path does write headers. The Verdict::Deny arm builds x-chio-denial-reason and x-chio-denial-guard onto the DeniedHttpResponse, with a body of {"verdict":"deny","reason":...,"guard":...} (response.rs:38-65). The guard name is whatever guard refused; the crate's own tests use ScopeGuard (response.rs:253-280). A fail-closed refusal, meaning translation or kernel evaluation itself errored, is different again: HTTP 500, x-chio-denial-reason: ext_authz request denied fail closed as the only header, and a body whose guard reads fail_closed (response.rs:73-85).

The deny in the curl above is not one of those. No CUSTOM rule matches a request with no capability header, so Chio never runs and the 403 comes from Envoy enforcing the chio-deny-unauthenticated policy. It carries no x-chio-* header for the same reason: nothing in Chio was asked.


Smoke Assertions

The test harness checks the allow and deny cases. From test-harness.sh:

examples/istio-ext-authz/test-harness.sh91-141bash
log "issuing allow request (capability token present)..."
ALLOW_STATUS="$(
  "${CURL}" -sS \
    -o "${ALLOW_BODY}" \
    -D "${ALLOW_HEADERS}" \
    -w '%{http_code}' \
    -H "x-chio-capability-token: ${CAPABILITY_TOKEN}" \
    -H "authorization: Bearer ${CAPABILITY_TOKEN}" \
    -X POST \
    --data '{"hello":"world"}' \
    "http://127.0.0.1:${LOCAL_PORT}/tools/hello"
)"

if [[ "${ALLOW_STATUS}" != "200" ]]; then
  log "allow response headers:"
  cat "${ALLOW_HEADERS}" >&2 || true
  fail "expected HTTP 200 on allow, got ${ALLOW_STATUS}"
fi

# Header names are case-insensitive per RFC 7230. Match loosely.
RECEIPT_ID="$(
  awk 'BEGIN{IGNORECASE=1} /^x-chio-receipt-id:/ { sub(/\r$/, ""); print $2; exit }' \
    "${ALLOW_HEADERS}"
)"

if [[ -z "${RECEIPT_ID}" ]]; then
  log "allow response headers:"
  cat "${ALLOW_HEADERS}" >&2 || true
  fail "allow response missing x-chio-receipt-id header"
fi

log "allow OK: status=200 receipt=${RECEIPT_ID}"

log "issuing deny request (no credentials)..."
DENY_STATUS="$(
  "${CURL}" -sS \
    -o "${DENY_BODY}" \
    -D "${DENY_HEADERS}" \
    -w '%{http_code}' \
    -X POST \
    --data '{"hello":"denied"}' \
    "http://127.0.0.1:${LOCAL_PORT}/tools/hello"
)"

if [[ "${DENY_STATUS}" != "403" ]]; then
  log "deny response headers:"
  cat "${DENY_HEADERS}" >&2 || true
  fail "expected HTTP 403 on deny, got ${DENY_STATUS}"
fi

log "deny OK: status=403"

Decision rule

Use this example when Chio must govern requests through an existing service mesh with Envoy sidecars. Use hello-tool with HTTP middleware for a single Node service that does not need mesh-wide policy. The mesh adds a per-request gRPC round trip to the Chio adapter.

Troubleshooting

SymptomLikely causeFix
All requests 403, including authenticatedChio pod not ready or MeshConfig not reloadedkubectl -n chio-system get pods; istioctl proxy-config bootstrap ... | grep chio-ext-authz
Allow missing x-chio-receipt-idExpected. The allow path sets no response header; see aboveRead chio.verdict from Envoy dynamic metadata, or have your adapter image add the header
AuthorizationPolicy rejected at applyAPI version mismatch (v1beta1 vs v1)Cluster on Istio 1.22+ and serving security.istio.io/v1
Authenticated POST reaches the app but Chio never ranPod not labelled chio.world/secured=true, so the CUSTOM policy selector does not match itkubectl -n agent-tools get pod -l app.kubernetes.io/name=demo-tool --show-labels
503 from demo podIstio sidecar injection disabled on agent-toolskubectl label ns agent-tools istio-injection=enabled --overwrite

Teardown

bash
kubectl delete -f examples/istio-ext-authz/02-authorization-policy.yaml
kubectl delete -f examples/istio-ext-authz/03-demo-workload.yaml
kubectl delete -f examples/istio-ext-authz/00-chio-sidecar-deployment.yaml
# Optional: remove the extension provider from MeshConfig by editing the
# istio ConfigMap (or reinstall without 01-meshconfig-patch.yaml).

Where to read more

Envoy ext_authz for the full filter contract, gRPC vs HTTP modes, and the check protocol. Protect an API for the broader API-protection playbook.
Istio ext_authz · Chio Docs