Computer Use Guards
Gate clicks, keystrokes, navigation, screenshots, and remote-desktop channels before a computer-use agent acts, and score embeddings against known threats.
On this page
Five guards stand between a computer-use agent and the screen it drives. ComputerUseGuard gates remote-session and input action types, navigation, and screenshot rate. InputInjectionCapabilityGuard checks injected keyboard, mouse, and touch input, RemoteDesktopSideChannelGuard switches each remote-session channel, BrowserAutomationGuard holds a browser to read-only verbs and scans typed text for credentials, and EmbeddingAnomalyGuard refuses embeddings close to known threats.
Each reads the action and arguments the host submits, so the integration classifies the action and supplies the fields each guard reads. Every guard returns Verdict::Allow or Verdict::Deny. None returns Verdict::PendingApproval; approval uses a separate hosted workflow, described in Approval & HITL.
An action-type match decides the coarse gate before any browser check
ComputerUseGuard reads an action type first and a browser action second, and returns at the first rule that names the call.
Guardrail warns on the action-type list
remote.webrtc is outside the default allowed action types. The default Guardrail mode allows it with a warning, and RemoteDesktopSideChannelGuard refuses the unknown channel. Enforcement modes.
The action type is the tool name when it begins with remote. or input., or an action_type, actionType, custom_type, or customType argument with that prefix. A match is decided against allowed_action_types under the configured mode, and the guard returns. Otherwise extract_action_checked parses the call and refuses a malformed one. On a ToolAction::BrowserAction, a screenshot verb takes a token from the bucket, and a navigation verb (navigate, goto, open) has its host checked against the domain lists. Every other action passes.
Host parsing folds mixed-case schemes, scheme-relative URLs (//host), userinfo (user@host), and bracketed IPv6 to the bare host, so *.suffix wildcards match. Opaque targets such as CSS selectors, data URIs, about:, and javascript: pass here, and BrowserAutomationGuard owns the finer domain checks.
Guardrail, the default mode, warns on the action-type list
The mode maps the allowlist outcome to a verdict.
| Mode | In allowlist | Out of allowlist |
|---|---|---|
Observe | Allow | Allow (logged) |
Guardrail | Allow | Allow (warning) |
FailClosed | Allow | Deny |
Set mode: fail_closed for hard enforcement on action types. The navigation and screenshot paths are stricter: Guardrail and FailClosed both refuse a blocked domain or an empty bucket, and a non-empty allowed_domains admits only its hosts under FailClosed.
A warning at the coarse gate still meets the channel guard. Under Guardrail, remote.webrtc clears the action-type list with a warning, and RemoteDesktopSideChannelGuard then refuses it as an unknown channel.
Remote-session channels and injected input are allowed one by one
RemoteDesktopSideChannelGuard has a switch for each of six channels (clipboard, file transfer, session share, audio, drive mapping, and printing), each read from its remote.* action type and on by default. Its default branch refuses any other remote.* type, so a new channel needs a configuration update before use. With max_transfer_size_bytes set, a remote.file_transfer call must carry an integer transfer_size. Session connect, disconnect, and reconnect belong to ComputerUseGuard.
InputInjectionCapabilityGuard fires on input.inject and on keyboard, mouse, touch, or input tools that carry an input_type. It refuses an input_type outside allowed_input_types (keyboard, mouse, and touch by default) and, with strict on, an injection-shaped call that lacks one.
With require_postcondition_probe set, it refuses a call that carries no probe hash. The guard checks that the hash is present. The host establishes where the probe came from and verifies the resulting screen before treating the hash as evidence that the action succeeded.
Browser sessions start read-only, and typed text is scanned for credentials
BrowserAutomationGuard reads ToolAction::BrowserAction on three axes: a verb allowlist, navigation domains with separate blocked and allowed lists, and credential detection on type and input verbs. Its fifteen default verbs are read-only, among them navigate, screenshot, get_content, and back. type, click, fill, and submit_form are refused until you add them to allowed_verbs.
A writeable session pays for credential detection on every type call. The built-in patterns match AWS access keys, GitHub tokens, Slack tokens, JWT shapes, PEM private-key headers, password = and token = assignments, and OpenAI and Stripe secret keys, and extra_credential_patterns adds your own. One policy configures this guard and the three before it:
hushspec: "0.1.0"
rules:
computer_use:
enabled: true
mode: fail_closed
allowed_actions:
- "remote.session.connect"
- "input.inject"
input_injection:
enabled: true
allowed_types: ["keyboard", "mouse"]
require_postcondition_probe: true
remote_desktop_channels:
enabled: true
clipboard: false
file_transfer: true
audio: false
drive_mapping: false
browser_automation:
enabled: true
allowed_domains: ["*.larkfield.example"]
blocked_domains: ["169.254.169.254"]
allowed_verbs: ["navigate", "screenshot", "get_content", "type"]
credential_detection: trueHushSpec carries only the fields its compiler consumes, a narrower set than each guard's Rust config.
Embeddings close to a known threat are refused, and you set the ambiguous band
EmbeddingAnomalyGuard scores the request's embedding by cosine similarity against every entry in a pattern database loaded at construction. A top score at or above similarity_threshold + ambiguity_band (0.85 and 0.10 by default) is refused, a score at or below the threshold minus the band is allowed, and an in-band score takes ambiguous_policy, Allow by default. A request with no embedding passes; one with the wrong dimension or a non-finite value is refused.
Receipts carry the top-scoring entry alongside the threshold and band. The guard has no HushSpec rule, so build the pattern database and register it in code:
let guard = EmbeddingAnomalyGuard::from_json_file(
"/etc/chio/embedding_anomaly_patterns.json",
)?;
kernel.add_guard(Box::new(guard));Computer-use guard reference
Try screenshot_burst, strict, or remote.clipboard.
ComputerUseGuard
Source: crates/guards/chio-guards/src/computer_use.rs. A coarse gate for three action groups:
- Remote-session and side-channel actions whose tool name or
action_typeorcustom_typeargument starts withremote.orinput.. - Browser navigation verbs (
navigate,goto,open) checked against a blocked-domain list and an optional allowed-domain list. - Screenshot-family verbs rate-limited by a token bucket (
screenshot,screen_capture,screen_shot,capture,capture_screen,browser_screenshot).
Config
ComputerUseConfig fields:
| Knob | Type | Default | Purpose |
|---|---|---|---|
enabled | bool | true | Master switch. A disabled guard returns Allow. |
allowed_ | Vec<String> | 10 entries | Allowlist for remote.* and input.* action types. |
mode | EnforcementMode | Guardrail | Observe, Guardrail, or FailClosed. |
blocked_domains | Vec<String> | empty | Hosts denied for navigation. Supports *.suffix. |
allowed_domains | Vec<String> | empty | When non-empty, only listed hosts pass under FailClosed. |
screenshot_ | Option<f64> | None | Refill rate. None disables rate limiting. |
screenshot_ | Option<u32> | Some(5) when rate is set | Token-bucket burst capacity. |
Default allowed action types
pub fn default_allowed_action_types() -> Vec<String> {
vec![
"remote.session.connect".to_string(),
"remote.session.disconnect".to_string(),
"remote.session.reconnect".to_string(),
"input.inject".to_string(),
"remote.clipboard".to_string(),
"remote.file_transfer".to_string(),
"remote.audio".to_string(),
"remote.drive_mapping".to_string(),
"remote.printing".to_string(),
"remote.session_share".to_string(),
]
}Detection algorithm
- Extract a computer-use action type: the tool name itself if it begins with
remote.orinput., otherwise anaction_type,actionType,custom_type, orcustomTypeargument with the same prefix. If found, apply the mode mapping against the action-type allowlist and return. - Extract the action through
extract_action_checked. A type-mismatched or unparseable call is a hardErr, and the guard denies it (Err(_) => GuardDecision::deny(Vec::new())), so a malformed call never reaches the step 3 allow. On a parsedToolAction::BrowserAction, match the screenshot verb set first and consume a token on a screenshot; on a navigation verb, parse the host and check the blocked and allowed lists. - Once
extract_action_checkedsucceeds and the action is not aBrowserAction, the call passes withVerdict::Allow.
Failure modes
- A type-mismatched or unparseable action call (
extract_action_checkedreturnsErr) denies. Guard boundaries use the checked extractor, not the best-effortextract_action, so a malformed call fails closed. - Token-bucket mutex poisoning collapses to no tokens, so the rate limiter denies under Guardrail and FailClosed.
- Opaque navigation targets (CSS selectors, data URIs,
about:,javascript:) pass through. Fine-grained domain checks belong toBrowserAutomationGuard. - Mixed-case schemes, scheme-relative URLs (
//host), userinfo segments (user@host), and bracketed IPv6 hosts all parse to the bare host, so wildcards match correctly.
InputInjectionCapabilityGuard
Source: crates/guards/chio-guards/src/input_injection.rs. A fine-grained gate that fires when tool_name is "input.inject" or "input_inject", when an action_type or custom_type argument equals input.inject, or when a recognized tool name (keyboard, mouse, touch, input) carries an input_type field.
Config
| Knob | Type | Default | Purpose |
|---|---|---|---|
enabled | bool | true | Master switch. |
allowed_ | Vec<String> | ["keyboard", "mouse", "touch"] | Permitted input_type values. |
require_ | bool | false | When true, requires a non-empty postcondition_. |
strict | bool | true | Deny calls that look like injection but lack input_type. |
Failure modes
- An
input_typeoutside the allowlist denies. - A missing
input_typeon an injection-flagged call denies withstrict = trueand allows otherwise. - With
require_postcondition_probeset, a call missing bothpostcondition_probe_hashandpostconditionProbeHashdenies. Empty strings count as missing. The guard checks presence, never the digest or the action's result.
RemoteDesktopSideChannelGuard
Source: crates/guards/chio-guards/src/remote_desktop.rs. Per-channel toggles for six named side channels, plus a transfer-size ceiling. Session-lifecycle actions (remote.session.connect, remote.session.disconnect, remote.session.reconnect) belong to ComputerUseGuard.
| Channel | Action type | Config field |
|---|---|---|
Clipboard | remote. | clipboard_ |
File transfer | remote. | file_ |
Session share | remote. | session_ |
Audio | remote.audio | audio_enabled |
Drive mapping | remote. | drive_ |
Printing | remote.printing | printing_ |
Every channel toggle defaults to true. max_transfer_size_bytes is Option<u64> and defaults to None. When set, remote.file_transfer calls must include a transfer_size or transferSize u64 argument; a missing or non-integer transfer size with a configured ceiling denies. The default branch is fail-closed: an unknown remote.* type such as remote.webrtc denies.
BrowserAutomationGuard
Source: crates/guards/chio-guards/src/browser_automation.rs. Three policy axes on ToolAction::BrowserAction:
- Verb allowlist. Empty means any verb; defaults to a read-only set.
- Navigation domain gating with separate blocked and allowed lists.
- Credential detection on type and input verbs through built-in regex patterns plus operator-supplied extras.
Default allowed verbs
pub fn default_allowed_verbs() -> Vec<String> {
vec![
"navigate".to_string(),
"goto".to_string(),
"open".to_string(),
"screenshot".to_string(),
"screen_capture".to_string(),
"capture".to_string(),
"browser_screenshot".to_string(),
"get_url".to_string(),
"get_title".to_string(),
"read".to_string(),
"get_content".to_string(),
"close".to_string(),
"back".to_string(),
"forward".to_string(),
"reload".to_string(),
]
}Verbs like type, click, fill, and submit_form deny unless added explicitly. A writeable session widens allowed_verbs and accepts the credential-detection cost on every type call.
Config
| Knob | Type | Default | Purpose |
|---|---|---|---|
enabled | bool | true | Master switch. |
allowed_domains | Vec<String> | empty | When set, the only hosts the agent may navigate to. |
blocked_domains | Vec<String> | empty | Always denied. Evaluated before the allowlist. |
allowed_verbs | Vec<String> | 15-entry read-only set | Permitted action verbs. Empty means any verb. |
credential_ | bool | true | Run regex detection on type and input action text. |
extra_ | Vec<String> | empty | Operator regex patterns layered on top of built-ins. |
Built-in credential patterns
Compiled once through OnceLock. They cover AWS access keys (AKIA or ASIA plus 16 characters), GitHub PATs (gh[pousr]_*), Slack tokens (xox[abopsr]-*), JWT shapes (eyJ*.*.*), PEM private-key headers, generic password = ... and token = ... assignments, OpenAI sk-* keys, and Stripe sk_live_* and sk_test_* keys.
Selectors skip navigation gating. The guard treats values starting with #, ., [, a single /, or xpath= as CSS selectors. URLs like //host still parse and check.
EmbeddingAnomalyGuard
Source: crates/guards/chio-guards/src/embedding_anomaly.rs. Guard name: embedding-anomaly. A cosine-similarity anomaly detector over a pre-loaded database of known-threat embeddings.
Detection algorithm
- Extract a query embedding from the arguments. Preferred shapes:
embedding(array of numbers),vector, orembeddings(list of vectors, mean-pooled). - If no embedding is present, allow.
- If the embedding has the wrong dimension or non-finite values, deny.
- Score the request by cosine similarity against every pattern in the database, with hand-rolled f64 accumulation and no BLAS dependency.
- Apply the threshold band. A top score ≥
threshold + ambiguity_banddenies. A top score ≤threshold - ambiguity_bandallows. An in-band score dispatches on the configuredambiguous_policy.
Config
| Knob | Type | Default | Purpose |
|---|---|---|---|
similarity_ | f64 | 0.85 | Center of the deny and allow band. |
ambiguity_band | f64 | 0.10 | Half-width of the band. |
top_k | usize | 5 | Top-K matches retained per query. |
ambiguous_ | AmbiguousPolicy | Allow | Verdict for in-band scores. |
The pattern database is loaded at construction (EmbeddingAnomalyPatternDb::from_json or EmbeddingAnomalyGuard::from_json_file; EmbeddingAnomalyGuard::new(db, config) takes a pre-built database). It must be non-empty, have a uniform non-zero dimensionality, and carry only finite values. Construction returns EmbeddingAnomalyError::Parse, EmbeddingAnomalyError::Invalid, EmbeddingAnomalyError::Config, or EmbeddingAnomalyError::Io.
Evidence
Each PatternEntry carries an id, a category (such as prompt_injection), a stage (perception, cognition, action, or feedback), a human-readable label, and the embedding vector. The guard scans the full database for the maximum cosine; receipts emit the top-scoring entry alongside the threshold and band.
Evaluation cost drivers
Evaluation cost depends on the input and the configured patterns. The grid names the work each guard performs; benchmark your own action mix for latency.
| Guard | Hot path | Scales with |
|---|---|---|
Remote | Set lookup + optional u64 compare | Action-field extraction and channel lookup |
Input | HashSet lookup on input type | Input-field extraction and type lookup |
Computer | Action-type set + URL parse + token bucket | URL length, domain patterns, and bucket contention |
Browser | Verb set, URL parsing, and configured credential regexes | Domain patterns, typed text, and detector patterns |
Embedding | Cosine over the full pattern database | Pattern count and vector dimensions |
HushSpec configuration
The computer-use rule blocks sit directly under the flat top-level rules: key, as in the policy above. HushSpec models the coarse allowlist, injection, side-channel toggles, and browser-automation gate, with only the fields the compiler consumes. There is no rules.embedding_anomaly block: like the jailbreak and prompt-injection guards, EmbeddingAnomalyGuard is operator- or code-wired.