Handle tool outcomes: refusal, failure, and cancellation

Run a repricing job and know, after every outcome, whether the tool ran, what it left on disk, and whether retrying is safe.

On this page

A repricing job that a client can read five ways: published, refused, cancelled, cut short by a failed database, and published with its acknowledgement lost. Only one of those stops before the tool runs.

The job applies one discount to sku-118, sku-204, and sku-301 in turn, writes a price-sheet row for each, then publishes the sheet. One job ID, which is also the kernel request ID, stays attached to the job from submission to its result and receipt. Every job draws on a shared allowance of local credits.

Start

Repricing outcomesChoose an outcome.

The kernel admits the job and the sheet publishes

reprice writes one row for each of sku-118, sku-204, and sku-301 and publishes price-sheets/<job-id>.csv. The kernel signs an allow receipt that carries the job ID, so the client follows one job from submission to receipt. The mediated call.

One repricing job through the pricing desk: the kernel K, the reprice tool d, the price sheet f, and the receipt ρ. Verdicts, reasons, and sheet states come from the captured runs; the queued cancellation, which stops before the kernel, is described in the panel.

Only the refusal stops at the kernel. The other outcomes happen after dispatch, so each leaves rows and a receipt behind, and the client reads both before it decides what to do next.

1. Start the repricing service

Starts from chio new pricing-desk --example pricing-desk, or the same project as a download. The desk is Larkfield's pricing functions behind small hosts that embed the kernel, so it needs no separate Chio process. Cargo selects Rust 1.94.1 from the project's toolchain file and fetches the Chio crates its manifest pins; the first build takes a few minutes.

Terminal · pricing-deskbash

chio new pricing-desk --example pricing-desk && cd pricing-desk
cargo run --locked --bin repricing-service

The host keeps its state in state/repricing and signs receipts into state/repricing/receipts.db. It serves its interface at 127.0.0.1:4312; keep the terminal running.

Before you begin

Use a Unix-compatible system with durable file locking, Rust through rustup, and curl and jq for the client commands. Keep the selected port free. The project README lists every endpoint, credential file, and supported persistence setting.

Open the interface with the credential from state/repricing/client-token.txt, choose Reprice all three SKUs, and watch the rows land one by one. In a second terminal in the same directory, define two helpers the steps below use: one submits a job, the other waits until the job matches a condition and keeps its snapshot in job.json.

Terminal 2 · pricing-deskbash

TOKEN="$(cat state/repricing/client-token.txt)"
reprice() {
  curl -sS http://127.0.0.1:4312/jobs -H "Authorization: Bearer $TOKEN" \
    -H 'Content-Type: application/json' -d "$1" | jq -r .job_id
}
job() {  # wait until the job matches a condition, then keep it in job.json
  until curl -sS "http://127.0.0.1:4312/jobs/$1" -H "Authorization: Bearer $TOKEN" -o job.json &&
    jq -e "${2:-.result != null}" job.json > /dev/null; do sleep 0.5; done
}

2. Submit a job and follow its ID

Reprice the catalog at 10 percent in complete mode and read the finished job.

Terminal 2 · pricing-deskbash

JOB_ID="$(reprice '{"percent":10,"mode":"complete"}')"
job "$JOB_ID"
jq '{job_id, published, partial_sheet, result: (.result | {verdict, terminal_state, receipt_id})}' job.json

Output

{
  "job_id": "7f8a964c-8ca9-44e3-a21b-10252c3f112c",
  "published": true,
  "partial_sheet": "sku,list,percent,price\nsku-118,24.00,10,21.60\nsku-204,58.00,10,52.20\nsku-301,140.00,10,126.00\n",
  "result": {
    "verdict": "Allow",
    "terminal_state": { "state": "completed" },
    "receipt_id": "cab91c6383dcaee2b976b23d53dbdb64e4e620901b34631e117af311bc7d6a1e"
  }
}

You should see three rows, published: true, and an Allow verdict. The sheet is state/repricing/price-sheets/<job-id>.csv and its exact result is state/repricing/jobs/<job-id>.result.json. Read terminal_state and the receipt's decision together with the verdict: after dispatch, the verdict alone cannot say whether the tool changed anything.

3. Cancel a job in flight

Start another job, wait for the sku-118 row, and request cancellation.

Terminal 2 · pricing-deskbash

JOB_ID="$(reprice '{"percent":10,"mode":"complete"}')"
job "$JOB_ID" '.partial_sheet // "" | test("sku-118")'
curl -sS "http://127.0.0.1:4312/jobs/$JOB_ID/cancel" -H "Authorization: Bearer $TOKEN" \
  -H 'Content-Type: application/json' -d '{}' | jq -r .message
job "$JOB_ID"
jq '{partial_sheet, result: (.result | {verdict, terminal_state, receipt_id})}' job.json

Output

Wait for the final outcome; cancellation does not undo rows already written
{
  "partial_sheet": "sku,list,percent,price\nsku-118,24.00,10,21.60\n",
  "result": {
    "verdict": "Deny",
    "terminal_state": {
      "reason": "The operator stopped this repricing job",
      "state": "cancelled"
    },
    "receipt_id": "ea2c6de12eb7e8907d91ab28ecc201c076687e810315553be03ad3d9afa3071f"
  }
}

You should see the sku-118 row kept and the job ended cancelled with its own receipt. A cancellation that lands in the 500 ms queue window before admission ends differently: status: cancelled_before_dispatch with no receipt, because the host stopped the job before the kernel saw it. The interface's Request cancellation button sends the same request.

How the tool cooperates with cancellation

src/repricing.rs · in RepricingTool::invokerust

if let Some(bridge) = bridge.as_deref_mut() {
    bridge.poll_parent_cancellation()?;
}
tokio::time::sleep(std::time::Duration::from_millis(700)).await;
if let Some(bridge) = bridge.as_deref_mut() {
    bridge.poll_parent_cancellation()?;
}

The tool polls Chio's nested-flow cancellation bridge around each timed row and persists every row it finishes, so a cancellation stops future rows and leaves written ones in place. A remote tool needs its own cancellation mechanism; dropping a browser request or killing a worker proves nothing about the operation behind it.

4. Run each failure mode

Change mode to exercise the other outcomes. Give each experiment its own state directory and restart the service with it, because a job that fails after dispatch keeps its reservation exposed. Here is the lost acknowledgement:

Terminal 2 · pricing-deskbash

export CHIO_APP_STATE=state/repricing-uncertain   # restart the service with this set
TOKEN="$(cat state/repricing-uncertain/client-token.txt)"
JOB_ID="$(reprice '{"percent":10,"mode":"uncertain"}')"
job "$JOB_ID"
jq '{job_id, published, result: (.result | {verdict, reason, terminal_state, receipt_id})}' job.json

Output

{
  "job_id": "3170f82b-07ef-48a7-9eac-301ed25041cc",
  "published": true,
  "result": {
    "verdict": "Deny",
    "reason": "internal error: Price sheet saved, but its delivery acknowledgement was lost",
    "terminal_state": { "state": "completed" },
    "receipt_id": "ce0ec15be2e3fb97b80673c65c1c761a98c180af6cca3a3ab29c6299d47265a0"
  }
}

You should see published: true beside a Deny verdict: the sheet exists although the job reports failure. Reconcile the existing job ID and never publish a second sheet for the same change.

mode	The tool	Inspect
`complete`	Writes three rows, publishes the sheet	Allow receipt, final CSV
`refuse`	Never runs; the job asks for `refund`, outside the grant	Deny receipt, no rows
`fail`	Writes the `sku-118` row, then the orders database stops responding	Deny receipt, one row kept
`uncertain`	Publishes, then loses the acknowledgement	Deny receipt, `published: true`

A refusal names its cause: “requested tool refund on server pricing is not in capability scope”. Repeating that request cannot widen the grant, so request suitable authority instead. A failed database leaves the sku-118 row on disk with the reason “The orders database stopped responding after sku-118”; repair the dependency, then decide deliberately about a new attempt. A discount outside 0 to 100 percent is rejected with HTTP 400 before it becomes a job, so it leaves no receipt.

5. Recover an interrupted job by its ID

Ctrl+C drains the service: it cancels active jobs and signs their receipts. A crash leaves no such record. On fresh state, start a job, kill the service after the first row lands, start it again with the same state, and read the original job.

Terminal 2 · pricing-deskbash

export CHIO_APP_STATE=state/repricing-crash   # restart the service with this set
TOKEN="$(cat state/repricing-crash/client-token.txt)"
JOB_ID="$(reprice '{"percent":10,"mode":"complete"}')"
job "$JOB_ID" '.partial_sheet // "" | test("sku-118")'
pkill -9 -f repricing-service   # a crash, not a clean shutdown
# Start the service again in terminal 1 with the same state, then:
job "$JOB_ID" '.active == false'
jq '{job_id, active, needs_reconciliation, started, published, partial_sheet, result}' job.json

Output

{
  "job_id": "d3a2320f-3890-4a15-be12-6f4dd46811ee",
  "active": false,
  "needs_reconciliation": true,
  "started": true,
  "published": false,
  "partial_sheet": "sku,list,percent,price\nsku-118,24.00,10,21.60\n",
  "result": null
}

You should see needs_reconciliation: true: the saved request has no terminal result and no live worker. Keep its original ID, inspect the receipt store and the allowance, and resolve the existing effect before any replacement work. Here started comes from the partial sheet, which is an observation by the application, not proof about the orders database.

Terminal 2 · pricing-deskbash

ls state/repricing-crash/jobs/"$JOB_ID".* state/repricing-crash/price-sheets/"$JOB_ID".*
curl -sS http://127.0.0.1:4312/budget -H "Authorization: Bearer $TOKEN" | jq -c '{exposed, spent, available}'

Output

state/repricing-crash/jobs/d3a2320f-3890-4a15-be12-6f4dd46811ee.request.json
state/repricing-crash/price-sheets/d3a2320f-3890-4a15-be12-6f4dd46811ee.partial.csv
{"exposed":80,"spent":0,"available":120}

The request and the partial sheet exist; the result does not. The reservation stays exposed (80 cents held, 120 available) until you resolve the job. Whether a partial sheet is reused, replaced, or discarded is your application's contract. The host never retries a job on its own, and a new state directory creates new authority and leaves spent credits spent.

6. Read the receipt behind the result

Read the signed receipt inside the lost-acknowledgement job from step 4 and check that it belongs to this job.

Terminal 2 · pricing-deskbash

jq '{receipt_id: .result.receipt.id, tool: .result.receipt.tool_name, kind: .result.receipt.receipt_kind,
     job_in_receipt: .result.receipt.metadata.job_id, verdict: .result.receipt.decision.verdict, published}' job.json

Output

{
  "receipt_id": "ce0ec15be2e3fb97b80673c65c1c761a98c180af6cca3a3ab29c6299d47265a0",
  "tool": "reprice",
  "kind": "mediated_decision",
  "job_in_receipt": "3170f82b-07ef-48a7-9eac-301ed25041cc",
  "verdict": "deny",
  "published": true
}

You should see the receipt ID from the result, reprice, and the job ID in the receipt's signed metadata. A deny beside published: true is the lost acknowledgement, recorded exactly. The captured result and the captured interruption keep the signed records and their accounting exposure; use the IDs your own jobs return.

Complete source: src/repricing.rs, src/client.rs, repricing.html, all in the project download.

Next: share the allowance across workers

Share a worker budgetRun three concurrent workers against the same USD 2.00 grant.Trace a repricing jobFollow one job into an exported trace and its receipt.

PreviousRequire Human Approval NextTenant-Scoped Service