Skip to content

Coming from Temporal or Restate

You already know why durable execution is worth having. The two questions you arrive with are what the rules are and what happens when you change your code while runs are in flight. Both answers are short.

A workflow is a normal program the engine can run more than once. Each durable call - step, sleep, wait-signal, wait-event, activity-collect - is a checkpoint whose result is recorded the first time it completes. When the run resumes, the script executes again from the top; calls already in the log return their recorded results instantly and execution races through to where it left off.

There is no bytecode capture, no special runtime, and no sandbox. A workflow is a bash, Python, TypeScript, Go or Rust program, and the durable calls are a subprocess away (cryo step …) or a socket write away (the SDKs).

Suspension is a real process exit. When a sleep or a wait parks, the script process is terminated and the agent slot is freed, so a run waiting a month holds no worker and no memory.

Four of them, and the authoring model is the whole treatment:

  • Wrap side effects and nondeterministic reads in a step. Anything touching the outside world, and anything a later decision depends on that could change between passes, belongs inside a step.
  • Make steps idempotent where you can. A step that dies after acting but before its result is recorded will run again.
  • Keep between-step code cheap and repeatable. It re-runs on every pass, so branch on values you captured in a step.
  • A step name is a program point. Reaching it again is an ordinary loop, and each pass records its own memo, so nothing needs a per-iteration name. The names that stay unique per run are the ones something refers back to: an activity (because activity-collect retrieves it), a bare lock <group>, and a state cas / state incr --as name.

If a replay reaches a different durable call than the first pass recorded at that position, the run fails with a determinism error instead of quietly re-running work. CRYO_REPLAY_MODE=lenient removes that check, per agent rather than per workflow. When replay goes wrong is the page to read next: a real determinism error, how to trace the reported op position back to the branch that caused it, and what the engine does not catch.

A run replays against the definition it started with. Shipping new workflow code leaves in-flight runs alone: they finish on their pinned version, and new runs pick up the change.

The pin is a property of the run, because the definition is stored data rather than something compiled into the agent. Nothing about your agent fleet has to change when you edit a workflow. There is no marker to leave in the code until the last old run drains, no combinatorial branch set to test, and no second fleet to keep alive and then remember to retire.

The cost of that choice is that a long-lived run finishes on the logic it started with, so a bug fix reaches it only if you cancel and resubmit.

Your word Here Worth knowing
workflow workflow a script, in whichever language you like
activity step cryo step name -- cmd, or step() in an SDK
worker agent claims work matching its advertised capabilities
task queue requires: a job claims only agents carrying those capabilities
signal signal cryo wait-signal <name>; the approval job kind uses one
child workflow child run cryo call graph, or the workflow: job kind
event history run event log what the run page renders
retry policy retry: per job or per activity-backed node
workflow id run id <org>:<project>:run-<random>
side effect API a step the recorded result is the same on every pass
patching / versioning nothing to do a run is pinned to its definition
continue-as-new a schedule, or a graph that names itself each pass is a fresh run with no accumulated log
query state cell cryo state get, entity-scoped for cross-run reads
idempotency key cryo state add claims the key; tells you whether you were first

The engine cannot catch a rule violation for you. Code between steps that misbehaves - an unwrapped random() steering a branch, a side effect outside a step - produces a run that is plausible and wrong rather than a loud error. The divergence check fires when the durable call sequence changes, and that is the only guard rail. The rules above are short enough to follow, and there is no static analysis here that will follow them for you.

Two more things to size up front:

  • Step results live in the event log. Record a path, an id or a summary, and put the build artifact in cryo artifact put.
  • Pick the loop shape deliberately. A while loop around cryo sleep re-executes every unwrapped command above it on each pass. Cron-shape work is its own primitive where each tick is a fresh run, and a graph that has to iterate until a condition calls itself. See Repeating work.
  • Queries and updates as a protocol. There is no request-response handler you register on the workflow, so there is no asking a run a question it did not prepare an answer for. Reading what it did record works: a run’s durable state cells are readable from outside it, live or long finished (see reading a cell from outside the run). What that costs you is having to decide in advance what to publish - the run writes the cell, and the reader fetches it.
  • Compensation as a first-class construct. Sagas are yours to write.
  • Anything resembling a replay test harness.