Chio/Docs
LOGIN · JOIN

Computer Use Guards

Gate clicks, keystrokes, navigation, screenshots, and remote-desktop channels before a computer-use agent acts, and score embeddings against known threats.

Concept chio@5f63e6e

On this page

Five guards stand between a computer-use agent and the screen it drives. ComputerUseGuard gates remote-session and input action types, navigation, and screenshot rate. InputInjectionCapabilityGuard checks injected keyboard, mouse, and touch input, RemoteDesktopSideChannelGuard switches each remote-session channel, BrowserAutomationGuard holds a browser to read-only verbs and scans typed text for credentials, and EmbeddingAnomalyGuard refuses embeddings close to known threats.

Each reads the action and arguments the host submits, so the integration classifies the action and supplies the fields each guard reads. Every guard returns Verdict::Allow or Verdict::Deny. None returns Verdict::PendingApproval; approval uses a separate hosted workflow, described in Approval & HITL.

An action-type match decides the coarse gate before any browser check

ComputerUseGuard reads an action type first and a browser action second, and returns at the first rule that names the call.

The computer-use guard, in check orderChoose an input.
r = remote.webrtcractiontypeg1well-formedg2screenshotbucketg3domainsg4vallowrremote.webrtccomputer-useg1action typeallowed, with a warningg2well-formedg3screenshot bucketg4domainsvallow

Guardrail warns on the action-type list

remote.webrtc is outside the default allowed action types. The default Guardrail mode allows it with a warning, and RemoteDesktopSideChannelGuard refuses the unknown channel. Enforcement modes.

ComputerUseGuard's evaluate drawn as a rail; an action-type match returns at the first check. Actions are illustrative: the navigation example assumes a blocked_domains entry and the screenshot example a configured rate. The other four computer-use guards are left out.

The action type is the tool name when it begins with remote. or input., or an action_type, actionType, custom_type, or customType argument with that prefix. A match is decided against allowed_action_types under the configured mode, and the guard returns. Otherwise extract_action_checked parses the call and refuses a malformed one. On a ToolAction::BrowserAction, a screenshot verb takes a token from the bucket, and a navigation verb (navigate, goto, open) has its host checked against the domain lists. Every other action passes.

Host parsing folds mixed-case schemes, scheme-relative URLs (//host), userinfo (user@host), and bracketed IPv6 to the bare host, so *.suffix wildcards match. Opaque targets such as CSS selectors, data URIs, about:, and javascript: pass here, and BrowserAutomationGuard owns the finer domain checks.

Guardrail, the default mode, warns on the action-type list

The mode maps the allowlist outcome to a verdict.

ModeIn allowlistOut of allowlist
ObserveAllowAllow (logged)
GuardrailAllowAllow (warning)
FailClosedAllowDeny

Set mode: fail_closed for hard enforcement on action types. The navigation and screenshot paths are stricter: Guardrail and FailClosed both refuse a blocked domain or an empty bucket, and a non-empty allowed_domains admits only its hosts under FailClosed.

A warning at the coarse gate still meets the channel guard. Under Guardrail, remote.webrtc clears the action-type list with a warning, and RemoteDesktopSideChannelGuard then refuses it as an unknown channel.

Remote-session channels and injected input are allowed one by one

RemoteDesktopSideChannelGuard has a switch for each of six channels (clipboard, file transfer, session share, audio, drive mapping, and printing), each read from its remote.* action type and on by default. Its default branch refuses any other remote.* type, so a new channel needs a configuration update before use. With max_transfer_size_bytes set, a remote.file_transfer call must carry an integer transfer_size. Session connect, disconnect, and reconnect belong to ComputerUseGuard.

InputInjectionCapabilityGuard fires on input.inject and on keyboard, mouse, touch, or input tools that carry an input_type. It refuses an input_type outside allowed_input_types (keyboard, mouse, and touch by default) and, with strict on, an injection-shaped call that lacks one.

With require_postcondition_probe set, it refuses a call that carries no probe hash. The guard checks that the hash is present. The host establishes where the probe came from and verifies the resulting screen before treating the hash as evidence that the action succeeded.

Browser sessions start read-only, and typed text is scanned for credentials

BrowserAutomationGuard reads ToolAction::BrowserAction on three axes: a verb allowlist, navigation domains with separate blocked and allowed lists, and credential detection on type and input verbs. Its fifteen default verbs are read-only, among them navigate, screenshot, get_content, and back. type, click, fill, and submit_form are refused until you add them to allowed_verbs.

A writeable session pays for credential detection on every type call. The built-in patterns match AWS access keys, GitHub tokens, Slack tokens, JWT shapes, PEM private-key headers, password = and token = assignments, and OpenAI and Stripe secret keys, and extra_credential_patterns adds your own. One policy configures this guard and the three before it:

policy.yamlyaml
hushspec: "0.1.0"
rules:
  computer_use:
    enabled: true
    mode: fail_closed
    allowed_actions:
      - "remote.session.connect"
      - "input.inject"
  input_injection:
    enabled: true
    allowed_types: ["keyboard", "mouse"]
    require_postcondition_probe: true
  remote_desktop_channels:
    enabled: true
    clipboard: false
    file_transfer: true
    audio: false
    drive_mapping: false
  browser_automation:
    enabled: true
    allowed_domains: ["*.larkfield.example"]
    blocked_domains: ["169.254.169.254"]
    allowed_verbs: ["navigate", "screenshot", "get_content", "type"]
    credential_detection: true

HushSpec carries only the fields its compiler consumes, a narrower set than each guard's Rust config.

Embeddings close to a known threat are refused, and you set the ambiguous band

EmbeddingAnomalyGuard scores the request's embedding by cosine similarity against every entry in a pattern database loaded at construction. A top score at or above similarity_threshold + ambiguity_band (0.85 and 0.10 by default) is refused, a score at or below the threshold minus the band is allowed, and an in-band score takes ambiguous_policy, Allow by default. A request with no embedding passes; one with the wrong dimension or a non-finite value is refused.

Receipts carry the top-scoring entry alongside the threshold and band. The guard has no HushSpec rule, so build the pattern database and register it in code:

src/guards.rs · embedded hostrust
let guard = EmbeddingAnomalyGuard::from_json_file(
    "/etc/chio/embedding_anomaly_patterns.json",
)?;
kernel.add_guard(Box::new(guard));

Computer-use guard reference

ComputerUseGuard

Source: crates/guards/chio-guards/src/computer_use.rs. A coarse gate for three action groups:

  • Remote-session and side-channel actions whose tool name or action_type or custom_type argument starts with remote. or input..
  • Browser navigation verbs (navigate, goto, open) checked against a blocked-domain list and an optional allowed-domain list.
  • Screenshot-family verbs rate-limited by a token bucket (screenshot, screen_capture, screen_shot, capture, capture_screen, browser_screenshot).

Config

ComputerUseConfig fields:

KnobTypeDefaultPurpose
enabled
bool
true
Master switch. A disabled guard returns Allow.
allowed_action_types
Vec<String>
10 entries
Allowlist for remote.* and input.* action types.
mode
EnforcementMode
Guardrail
Observe, Guardrail, or FailClosed.
blocked_domains
Vec<String>
empty
Hosts denied for navigation. Supports *.suffix.
allowed_domains
Vec<String>
empty
When non-empty, only listed hosts pass under FailClosed.
screenshot_rate_per_second
Option<f64>
None
Refill rate. None disables rate limiting.
screenshot_burst
Option<u32>
Some(5) when rate is set
Token-bucket burst capacity.

Default allowed action types

crates/guards/chio-guards/src/computer_use.rs46-59rust
pub fn default_allowed_action_types() -> Vec<String> {
    vec![
        "remote.session.connect".to_string(),
        "remote.session.disconnect".to_string(),
        "remote.session.reconnect".to_string(),
        "input.inject".to_string(),
        "remote.clipboard".to_string(),
        "remote.file_transfer".to_string(),
        "remote.audio".to_string(),
        "remote.drive_mapping".to_string(),
        "remote.printing".to_string(),
        "remote.session_share".to_string(),
    ]
}

Detection algorithm

  1. Extract a computer-use action type: the tool name itself if it begins with remote. or input., otherwise an action_type, actionType, custom_type, or customType argument with the same prefix. If found, apply the mode mapping against the action-type allowlist and return.
  2. Extract the action through extract_action_checked. A type-mismatched or unparseable call is a hard Err, and the guard denies it (Err(_) => GuardDecision::deny(Vec::new())), so a malformed call never reaches the step 3 allow. On a parsed ToolAction::BrowserAction, match the screenshot verb set first and consume a token on a screenshot; on a navigation verb, parse the host and check the blocked and allowed lists.
  3. Once extract_action_checked succeeds and the action is not a BrowserAction, the call passes with Verdict::Allow.

Failure modes

  • A type-mismatched or unparseable action call (extract_action_checked returns Err) denies. Guard boundaries use the checked extractor, not the best-effort extract_action, so a malformed call fails closed.
  • Token-bucket mutex poisoning collapses to no tokens, so the rate limiter denies under Guardrail and FailClosed.
  • Opaque navigation targets (CSS selectors, data URIs, about:, javascript:) pass through. Fine-grained domain checks belong to BrowserAutomationGuard.
  • Mixed-case schemes, scheme-relative URLs (//host), userinfo segments (user@host), and bracketed IPv6 hosts all parse to the bare host, so wildcards match correctly.

InputInjectionCapabilityGuard

Source: crates/guards/chio-guards/src/input_injection.rs. A fine-grained gate that fires when tool_name is "input.inject" or "input_inject", when an action_type or custom_type argument equals input.inject, or when a recognized tool name (keyboard, mouse, touch, input) carries an input_type field.

Config

KnobTypeDefaultPurpose
enabled
bool
true
Master switch.
allowed_input_types
Vec<String>
["keyboard", "mouse", "touch"]
Permitted input_type values.
require_postcondition_probe
bool
false
When true, requires a non-empty postcondition_probe_hash.
strict
bool
true
Deny calls that look like injection but lack input_type.

Failure modes

  • An input_type outside the allowlist denies.
  • A missing input_type on an injection-flagged call denies with strict = true and allows otherwise.
  • With require_postcondition_probe set, a call missing both postcondition_probe_hash and postconditionProbeHash denies. Empty strings count as missing. The guard checks presence, never the digest or the action's result.

RemoteDesktopSideChannelGuard

Source: crates/guards/chio-guards/src/remote_desktop.rs. Per-channel toggles for six named side channels, plus a transfer-size ceiling. Session-lifecycle actions (remote.session.connect, remote.session.disconnect, remote.session.reconnect) belong to ComputerUseGuard.

ChannelAction typeConfig field
Clipboard
remote.clipboard
clipboard_enabled
File transfer
remote.file_transfer
file_transfer_enabled
Session share
remote.session_share
session_share_enabled
Audio
remote.audio
audio_enabled
Drive mapping
remote.drive_mapping
drive_mapping_enabled
Printing
remote.printing
printing_enabled

Every channel toggle defaults to true. max_transfer_size_bytes is Option<u64> and defaults to None. When set, remote.file_transfer calls must include a transfer_size or transferSize u64 argument; a missing or non-integer transfer size with a configured ceiling denies. The default branch is fail-closed: an unknown remote.* type such as remote.webrtc denies.

BrowserAutomationGuard

Source: crates/guards/chio-guards/src/browser_automation.rs. Three policy axes on ToolAction::BrowserAction:

  • Verb allowlist. Empty means any verb; defaults to a read-only set.
  • Navigation domain gating with separate blocked and allowed lists.
  • Credential detection on type and input verbs through built-in regex patterns plus operator-supplied extras.

Default allowed verbs

crates/guards/chio-guards/src/browser_automation.rs44-62rust
pub fn default_allowed_verbs() -> Vec<String> {
    vec![
        "navigate".to_string(),
        "goto".to_string(),
        "open".to_string(),
        "screenshot".to_string(),
        "screen_capture".to_string(),
        "capture".to_string(),
        "browser_screenshot".to_string(),
        "get_url".to_string(),
        "get_title".to_string(),
        "read".to_string(),
        "get_content".to_string(),
        "close".to_string(),
        "back".to_string(),
        "forward".to_string(),
        "reload".to_string(),
    ]
}

Verbs like type, click, fill, and submit_form deny unless added explicitly. A writeable session widens allowed_verbs and accepts the credential-detection cost on every type call.

Config

KnobTypeDefaultPurpose
enabled
bool
true
Master switch.
allowed_domains
Vec<String>
empty
When set, the only hosts the agent may navigate to.
blocked_domains
Vec<String>
empty
Always denied. Evaluated before the allowlist.
allowed_verbs
Vec<String>
15-entry read-only set
Permitted action verbs. Empty means any verb.
credential_detection
bool
true
Run regex detection on type and input action text.
extra_credential_patterns
Vec<String>
empty
Operator regex patterns layered on top of built-ins.

Built-in credential patterns

Compiled once through OnceLock. They cover AWS access keys (AKIA or ASIA plus 16 characters), GitHub PATs (gh[pousr]_*), Slack tokens (xox[abopsr]-*), JWT shapes (eyJ*.*.*), PEM private-key headers, generic password = ... and token = ... assignments, OpenAI sk-* keys, and Stripe sk_live_* and sk_test_* keys.

Selectors skip navigation gating. The guard treats values starting with #, ., [, a single /, or xpath= as CSS selectors. URLs like //host still parse and check.

EmbeddingAnomalyGuard

Source: crates/guards/chio-guards/src/embedding_anomaly.rs. Guard name: embedding-anomaly. A cosine-similarity anomaly detector over a pre-loaded database of known-threat embeddings.

Detection algorithm

  1. Extract a query embedding from the arguments. Preferred shapes: embedding (array of numbers), vector, or embeddings (list of vectors, mean-pooled).
  2. If no embedding is present, allow.
  3. If the embedding has the wrong dimension or non-finite values, deny.
  4. Score the request by cosine similarity against every pattern in the database, with hand-rolled f64 accumulation and no BLAS dependency.
  5. Apply the threshold band. A top score ≥ threshold + ambiguity_band denies. A top score ≤ threshold - ambiguity_band allows. An in-band score dispatches on the configured ambiguous_policy.

Config

KnobTypeDefaultPurpose
similarity_threshold
f64
0.85
Center of the deny and allow band.
ambiguity_band
f64
0.10
Half-width of the band.
top_k
usize
5
Top-K matches retained per query.
ambiguous_policy
AmbiguousPolicy
Allow
Verdict for in-band scores.

The pattern database is loaded at construction (EmbeddingAnomalyPatternDb::from_json or EmbeddingAnomalyGuard::from_json_file; EmbeddingAnomalyGuard::new(db, config) takes a pre-built database). It must be non-empty, have a uniform non-zero dimensionality, and carry only finite values. Construction returns EmbeddingAnomalyError::Parse, EmbeddingAnomalyError::Invalid, EmbeddingAnomalyError::Config, or EmbeddingAnomalyError::Io.

Evidence

Each PatternEntry carries an id, a category (such as prompt_injection), a stage (perception, cognition, action, or feedback), a human-readable label, and the embedding vector. The guard scans the full database for the maximum cosine; receipts emit the top-scoring entry alongside the threshold and band.

Evaluation cost drivers

Evaluation cost depends on the input and the configured patterns. The grid names the work each guard performs; benchmark your own action mix for latency.

GuardHot pathScales with
RemoteDesktopSideChannelGuard
Set lookup + optional u64 compare
Action-field extraction and channel lookup
InputInjectionCapabilityGuard
HashSet lookup on input type
Input-field extraction and type lookup
ComputerUseGuard
Action-type set + URL parse + token bucket
URL length, domain patterns, and bucket contention
BrowserAutomationGuard
Verb set, URL parsing, and configured credential regexes
Domain patterns, typed text, and detector patterns
EmbeddingAnomalyGuard
Cosine over the full pattern database
Pattern count and vector dimensions

HushSpec configuration

The computer-use rule blocks sit directly under the flat top-level rules: key, as in the policy above. HushSpec models the coarse allowlist, injection, side-channel toggles, and browser-automation gate, with only the fields the compiler consumes. There is no rules.embedding_anomaly block: like the jailbreak and prompt-injection guards, EmbeddingAnomalyGuard is operator- or code-wired.

Next: decide which actions need a person