Skip to content

The authoring model

How durable workflows execute, and the rules that follow from it. Read this once before writing anything longer than a demo - the model is small, and every guideline below falls out of it.

A workflow script is a normal program that the engine can run more than once. Each durable call - step, sleep, wait-signal, wait-event, activity-collect - is a checkpoint: its result is recorded in the run’s event log the first time it completes. When the run resumes (after a crash, a sleep, a signal, an agent restart), the script executes again from the top. Durable calls whose results are already in the log return them instantly, without re-running anything; execution races through the recorded prefix and picks up where it left off.

That’s the whole model. There is no bytecode capture, no special runtime, no requirement that your code be deterministic in any formal sense. The engine replays recorded results; it never re-executes recorded code.

Two consequences shape everything else:

  • Between checkpoints, your code runs again on every pass. A log line between two steps prints on every resume. A counter incremented between steps counts passes, not events.
  • The suspension points are real process exits. When a sleep or a wait has to park, the script process is terminated and the agent slot is freed - a run sleeping for a month costs nothing. The call you wrote never returns on that pass; it returns on a later pass, from the log.

Wrap side effects and nondeterministic reads in a step. Anything that touches the outside world (deploys, API calls, file writes) and anything a later decision depends on that could change between passes - time, randomness, environment lookups, network reads - belongs inside a step. The recorded result is then the same on every pass, so decisions made from it are stable. Passes are not tied to a machine: a run that suspends is dispatched again to whichever agent is free, and the recorded result travels with the run, so a step’s value is the same one no matter where the resume lands or how long it took.

Make steps idempotent where you can. A step that dies after acting but before its result is recorded will run again. “Create or update”, “push if absent”, version-stamped writes - the usual idempotency toolkit applies.

When the remote system dedupes for you - most payment, messaging and ticketing APIs take an idempotency key or a transaction id - hand it a token that stays the same across every way one call can happen twice. On a graph node that’s node.idempotency_key:

{ "id": "charge", "type": "http",
"config": { "method": "POST", "url": "https://api.example.com/charges",
"headers": { "Idempotency-Key": { "$expr": "node.idempotency_key" } } } }

It’s stable across a retry:, across a replay after a suspend, and across a job re-dispatched because its agent died - all of those are the same call site of the same run. Inside a fan_out it also carries the item’s index, so ten notifications send ten keys rather than one. A rerun of the run gets a new token, which is right: a rerun is a deliberate new attempt.

In a script, the same token reaches a cryo step body as $CRYO_IDEMPOTENCY_KEY:

Terminal window
cryo step charge -- ./charge.sh "$account"
charge.sh
curl -fsS https://api.example.com/charges \
-H "Idempotency-Key: $CRYO_IDEMPOTENCY_KEY" \
-d "account=$1"

It is scoped to that step of that job of that run, so it holds across a retry:, a replay after a suspend and a re-dispatch to another agent, and two steps get two keys. A fan-out item is its own job, so ten items running the same charge step send ten different keys.

A step body deliberately sees none of the other CRYO_* variables - it has no agent socket of its own, and a nested cryo step should fail loudly rather than half-work. This one is put back afterwards because it is a value to send onwards rather than a handle to call back with, and it can’t be shadowed by a project secret of the same name.

Don’t reach for $CRYO_ACTIVITY_ID instead. It is visible in the script body and it is one constant for the whole job, so three retries of a dunning ladder would send one key, the provider would dedupe all three into the first charge, and nothing would tell you the customer was never charged again.

A YAML steps: job has no token of its own - its steps are one atomic script with no durable boundaries between them, so the whole job re-runs together. Put a call that must not double up in a script: job behind a cryo step, or on a graph node.

For “has anyone already handled this record?” - the same webhook delivered twice, two schedules picking up one row - the answer isn’t a key you send anywhere, it’s a claim you take. See Durable state.

Keep between-step code cheap and repeatable. It re-runs on every pass. Pure formatting, branching on recorded results, and building argument lists are all fine; sending an email is not. The engine enforces this: if a replay takes a different path and reaches a different durable call than the first pass recorded at that point, it fails with a determinism error instead of silently re-running. Branch on values you captured in a step, not on the wall clock or fresh randomness. A deployment that has to allow a divergence can set CRYO_REPLAY_MODE=lenient on the agent, which removes the check for everything that agent runs. When replay goes wrong walks through a real determinism error, how to find the branch that caused it, and what lenient mode actually does to a run.

Names are identity where something refers back to them. These are unique per run, and a second use is rejected rather than silently returning the first call’s result:

  • activity <name>, because activity-collect <name> retrieves that child’s result. Two submits under one name would make the second uncollectable.
  • lock <group> with no command, because the lease is held to the end of the run - there is one of it, not one per call site.
  • state cas / state incr, keyed by --as <name> (defaulting to the cell), because each attempt in a read-modify-write loop is its own question.

Everything else repeats. A step or stepFn name is its program point, so a loop reaches one name as many times as it iterates and each iteration gets its own memo; pass --label if you want the run view to say which host or which attempt an iteration was for. A wait-signal or wait-event name is a queue, so waiting on one twice takes the first signal and then the second. And the primitives that carry no name at all - sleep, emit, spawn - key off call order, so two sleeps are two waits.

What all of them need is stability across passes: the same call reached in the same order every time. That is what replay is built on, and it is why a branch on wall-clock or randomness is a determinism error rather than a slightly different run.

A handler run started by an event is an ordinary workflow: it replays under the same rules as any run you submit yourself, so nothing here changes when a run is triggered off the bus rather than by hand.

Mind result sizes. Step results are stored in the event log. Record a path, an id, or a summary - not a build artifact.

  • Code changes don’t strand in-flight runs. Each run replays against the definition it started with, so deploying new workflow code leaves in-flight runs untouched - they finish on their pinned version, and new runs pick up the change. No migration step, no version-pinning ceremony.
  • Any language works today. The durable calls are an exec away (cryo step …) or a socket write away (the SDKs); there’s no runtime to port.
  • Nothing to unlearn: a workflow reads exactly like the script you would have written anyway, with checkpoints at the moments that matter.

The trade: the engine cannot catch a rule violation for you. Code between steps that misbehaves - an unwrapped random() steering a branch, a side effect outside a step - produces a run that is plausible but wrong rather than a loud error. The rules above are short enough to actually follow.

A polyglot script executes on an agent advertising the polyglot capability, with the interpreter chosen by file extension or --interpreter (see polyglot scripts). The agent needs that runtime installed:

Language Runs as Agent needs
bash inline script (default) nothing extra
Python inline script, python3 python3
TypeScript inline script, bun run bun
JavaScript inline script, node node
Go go run (stdlib-only single file), or from a checkout / prebuilt binary when importing the SDK go toolchain
Rust compiled binary, or cargo run from a checkout rust toolchain

The Python, TypeScript, Go, and Rust SDKs speak the agent’s socket directly; the bash prelude and plain cryo verbs do the same thing one subprocess at a time. See workflows in your language for the same workflow in several languages, and note that a non-bash workflow needs its runtime (and, for the import path, its SDK) present wherever the agent runs it.