Skip to content
Datalium StrataDatalium Strata
These docs cover Strata 0.9.0-preview.1.It is a preview, and the API can still change before 1.0.

Human-in-the-loop

Some steps shouldn’t run until a person says yes: sending money, deleting data, writing to a customer. Strata lets a graph stop before such a step, save the thread, and continue when a person answers. The answer can take minutes or days. While the thread waits, nothing runs and your process holds nothing of it.

Interrupt boundaries

Section titled “Interrupt boundaries”

You mark a node as an interrupt boundary with AddInterruptBoundary in C# or add_interrupt_boundary in Rust. The pause comes after that node. The node runs and its state is committed. Then, instead of going on, the run stops. Strata works out the next node from the edges at that moment and saves it in the checkpoint.

So the usual shape is: one node prepares the risky action, the boundary is on that node, and the node after it carries the action out. The person reviews what the first node prepared.

The run returns Interrupted. It carries the thread ID, the node the run stopped at (SuspendedNode / suspended_node), the node it continues at (NextNode / next_node), the frozen state, and where the thread is waiting (WaitingAt / waiting_at, see Interrupts inside a subgraph).

This graph prepares a wire transfer and pauses for approval. A rejection goes to its own node. Later, the thread is approved with a corrected amount:

var graph = new StateGraph<WireTransferState>()
    .AddNode("PrepareWireTransfer", state => state with { InvoiceGenerated = true, Status = "awaiting approval" })
    .AddInterruptBoundary("PrepareWireTransfer", onReject: "CancelWireTransfer")
    .AddNode("ExecuteWireTransfer", (state, ctx, _) =>
    {
        var decision = ctx.Decision ?? throw new InvalidOperationException("a transfer must not run without a decision");
        var reference = bank.Send(state.Beneficiary, state.Amount, state.Currency);
        return ValueTask.FromResult(state with { Status = $"sent ({reference})", ApprovedBy = decision.Reviewer });
    })
    .AddNode("CancelWireTransfer", (state, ctx, _) =>
        ValueTask.FromResult(state with { Status = $"cancelled by {ctx.Decision!.Reviewer}" }))
    .AddEdge("PrepareWireTransfer", "ExecuteWireTransfer")
    .Compile();


// Runs PrepareWireTransfer, saves the thread, and returns Interrupted.
var result = await graph.RunAsync("wire-acme-corp", WireTransferState.New("Acme Corp", 4200.50m), store);


// Later, in this process or another one: approve, and correct the amount.
var approval = HumanSignal.Approve("jdoe@acme.io").WithOverride("Amount", 4150.25m);
var resumed = await graph.ResumeAsync("wire-acme-corp", approval, store);

Freeze, Flush, Evict, Rehydrate, Resume

Section titled “Freeze, Flush, Evict, Rehydrate, Resume”

A pause and its resume go through five steps:

Step What happens
Freeze The boundary node has committed its state. The run stops, and no later node starts.
Flush Strata writes an interrupted checkpoint to the store: the state, the node the run stopped at, and the node it continues at. A run always writes a checkpoint where it stops, whichever checkpoint policy you chose.
Evict The run returns Interrupted. Nothing of the run is still running or awaited, and the process keeps nothing of the thread.
Rehydrate When an answer arrives, Strata loads the latest checkpoint, applies the answer’s changes to the saved state, and turns the JSON back into your state type.
Resume The run continues at the saved next node, or at the reject route, with that state.

Between Evict and Rehydrate the thread exists only in the store. In the benchmark results, ten thousand threads frozen at a boundary against SQLite grew the heap by 0 bytes each in Rust and by nothing measurable in .NET, and took 1.6 KB each on disk. The in-memory store is different, because it is the store: it holds its copy of each checkpoint, 1.7 KB (.NET) or 3.6 KB (Rust) per thread for a 1.25 KB checkpoint.

Because the checkpoint holds the whole thread, any process with the same compiled graph and the same store can answer it, and a restart in between loses nothing. Both engines write the same checkpoint format, so a thread that a .NET service paused can be resumed by a Rust service. What happens when a process crashes in the middle of a node, rather than at a boundary, is on Durable execution.

Answering with a human signal

Section titled “Answering with a human signal”

You answer a paused thread with ResumeAsync (C#) or resume (Rust): the thread ID, a HumanSignal and the store. A signal has these parts:

Part C# Rust What it does
The answer IsApproved is_approved Approve or reject.
Who answered ReviewerToken reviewer_token An identity, such as a user ID or an email address. Strata saves it and shows it to every node, so never put a credential here.
Typed overrides Overrides, WithOverride overrides, with_override Replace top-level fields of the saved state, each with a JSON value of its own type: a number, a boolean, an object.
String overrides FieldOverrides field_overrides The same, but each value is written as a JSON string.

HumanSignal.Approve(reviewer) and HumanSignal.Reject(reviewer) in C#, and HumanSignal::approve and HumanSignal::reject in Rust, build the common cases.

Overrides are how a reviewer corrects the state before the run goes on. Each one replaces a whole top-level field. Strata applies them before any node runs, so the node after the boundary already sees the corrected state.

The recorded decision

Section titled “The recorded decision”

A resume that answers a boundary records the answer as a HumanDecision: the boundary, whether it was approved, the reviewer, the overrides applied and when. Every node after the boundary can read it, with ctx.Decision in C# or ctx.decision() in Rust. Every checkpoint the thread writes afterwards carries it. So if a later node fails and someone retries it, the node still knows who approved.

Rejecting

Section titled “Rejecting”

Without a reject route, a rejection continues at the next node, like an approval. That node reads IsApproved from the decision and decides what to do.

With a reject route, a rejection continues at its own node instead. You give it when you add the boundary: AddInterruptBoundary("Approve", onReject: "Cancel") in C#, add_interrupt_boundary_with_reject("Approve", "Cancel") in Rust. Overrides still apply on a rejection, so the reviewer can put a reason in the state. The route counts as an edge when the graph compiles, so a route to a node that doesn’t exist fails with STRATA_ERR_DANGLING_EDGE.

Checked against your state type

Section titled “Checked against your state type”

An override must fit your state. A string override for a number field doesn’t fit, which is why typed overrides exist. Before Strata touches the thread, it applies the overrides to a copy of the saved state and turns that copy into your state type. If that fails, or a key is set in both override maps, the resume returns Failed and the checkpoint is left as it was. The thread still waits, and you can answer it again with a corrected signal. The signal itself isn’t generic: your state type is the type that is checked.

What a resume refuses

Section titled “What a resume refuses”

A waiting checkpoint can be resumed once. A resume returns Failed and writes nothing when:

  • the thread has no checkpoint;
  • the checkpoint was already resumed. When two resumes race, only one of them wins;
  • the thread completed, was ended, or is running;
  • the thread waits for an event or a timer, not a person (see Waits that aren’t a person);
  • an override doesn’t fit the state.

A resume also accepts a thread that failed or was cancelled. It re-enters the node that stopped, and the signal’s overrides act as a correction. That kind of resume records no decision and never takes a reject route.

Strata doesn’t re-check the state when a thread resumes after a long wait. Prices, stock levels or permissions may have changed while the person decided. Check what matters in the first node after the boundary. See Known limitations.

Interrupts inside a subgraph

Section titled “Interrupts inside a subgraph”

A subgraph is a compiled graph used as one node of another (see Graphs). A boundary inside a subgraph pauses the whole thread. The outer checkpoint stops at the subgraph node and carries the subgraph’s own checkpoint inside it. SuspendedNode is then the subgraph node, and WaitingAt gives the full path, such as ["Work", "Approve"] for the boundary Approve inside the subgraph node Work. Subgraphs can nest to any depth.

A resume goes back into the subgraph and continues after its boundary. The nodes before the boundary don’t run again. The signal’s overrides go to the innermost state, the one the person reviewed. To change an outer level’s state in the same resume, pass a LevelSignals instead of a single signal. A subgraph doesn’t inherit its parent’s decision: it sees only the answers to its own boundaries.

The branches of a parallel group and the child graph of a map can’t have interrupt boundaries. See Known limitations.

Waits that aren’t a person

Section titled “Waits that aren’t a person”

A thread can also wait for an event, such as a payment arriving, or for a time. These waits use the same Freeze to Resume steps and hold nothing while they wait:

  • An event wait: AddWaitForEvent / add_wait_for_event, optionally with a timeout.
  • A durable timer: AddTimer / add_timer.
  • A human boundary with a timeout: AddInterruptBoundary with timeout and onTimeout in C#, add_interrupt_boundary_with and BoundaryOptions in Rust.

Each kind of wait has one answer, and the others are refused:

The thread waits for ResumeAsync / resume SignalAsync / signal WakeDueAsync / wake_due, once due
A person, no timeout Resumes Refused Never due
A person, with a timeout Resumes Refused Times out
An event Refused Resumes, for that event type only Times out, if it has a timeout
A timer Refused Refused Wakes

An event signal (EventSignal) carries its payload as typed overrides, like a person’s, but records no decision. A timeout continues at its timeout route. With no route, the thread is ended with the reason timed_out. A signal that arrives before the thread reaches its wait is refused, not kept for later (see Known limitations).

WakeDueAsync / wake_due wakes every due thread in a store, one at a time. To wake one thread through your own resume, for example to stream it, pass LevelSignals.Due (C#) or LevelSignals::due() (Rust) as the signal. LevelSignals.Event(signal) / LevelSignals::event(signal) answers an event wait the same way. Running wakes on a schedule is covered on Durable execution.

Ending a waiting thread

Section titled “Ending a waiting thread”

When an answer will never come, EndAsync (C#) or end (Rust) ends the thread where it stands. It returns Cancelled with the reason Ended, and the thread can’t be resumed after that.

Finding paused threads

Section titled “Finding paused threads”

Your application can ask the store for threads in the interrupted state, with ListThreadsAsync (C#) or list_threads (Rust). From a terminal, strata thread list --storage checkpoints.db --status interrupted lists them, and strata thread inspect shows where a thread waits and the decision it carries.

The CLI can’t resume a thread. It doesn’t have your graph’s code, so resuming is up to your application. See Known limitations.

See it run

Section titled “See it run”
Demo What it shows C# Rust
05-hitl-interruption A boundary with a reject route, a refused string override, an approval with a typed override, all after a simulated restart C# Rust
13-subgraph-composition A boundary inside a subgraph that pauses the whole thread C# Rust
19-waits-and-timers An event wait, a human boundary with a timeout, a timer, and ending a thread C# Rust

Next

Section titled “Next”