Contents

The payout saga

A payout reaches an external rail that answers in its own time, so it runs as a stored state machine: reserve, submit, settle — every transition a compare-and-set that posts its money in the same transaction.

A “state machine” is anything that can only be in one of a few named states at a time and can only move between them along set paths — think of a package that goes “shipped → out for delivery → delivered,” never the other way. A payout runs exactly like that, so an interruption can only ever leave it parked on a known step.

Source src/ports.ts#L902-L909SAGA_STATESsrc/operations/requestPayout.tssrc/worker/payouts.ts

A topUp or a spend finishes the instant you call it. A payout can’t: it reaches an external rail that answers in its own time.

So economy-lab models a payout as an explicit state machine. The operation becomes a stored record (a Saga) that the background worker advances one safe step at a time.[1] If a crash interrupts the work, the record resumes from its last recorded state. A step that runs twice still counts once.

Why it exists

Picture the alternative: paying a seller synchronously inside requestPayout. You call the provider, wait, post the money, and return. That couples a money posting to a network call the ledger doesn’t control.

The failure modes are bad. If the call hangs, the request hangs. If the process dies after the provider pays but before the ledger records it, the books and the provider disagree, with no record of why.

A long-lived record fixes the disagreement by making every intermediate state durable and named. The worker can read where a payout got to and drive it forward. An operator can see a stuck one rather than a silent gap.

The same shape covers anything that outlives one request. Subscriptions reuse it to renew period after period, and a promo grant’s expiry is likewise a row the worker sweeps, not a thread it holds open.

The states

requestPayout does not pay. It reserves the credits (debit earned → credit payout_reserve, two CREDIT legs that cancel out) and opens a Saga, a record whose state walks through SagaState (see SAGA_STATES).

The full enum declares REQUESTED, RESERVED, SUBMITTED, SETTLED, and FAILED. A live payout starts at RESERVED, because the credits were set aside in the same database transaction that opened the saga.

StateMeaningWhat advances it
RESERVEDCredits locked in payout_reserve; nothing sent to the rail yet.The worker’s payouts sweep.
SUBMITTEDThe sweep converted the reserve to USD and called the payout rail.The provider’s settlement webhook.
SETTLEDThe rail confirmed; the reserve clears to revenue and an equal sum of USD leaves trust_cash for the seller.Terminal.
FAILEDThe payout was given up on; the reserve returns to the seller’s earned.Terminal.
RESERVEDreserve heldSUBMITTEDsent to railSETTLEDterminalFAILEDterminalpayouts sweepreserve → USDsettlementwebhooktimeout, max attempts, reverse→ reserve returns to earnedSagaState also declares REQUESTED; a live payout opens already RESERVED, in the same transaction as the reservation.
The payout saga. Each transition is a compare-and-set that posts its money in the same transaction, so a re-driven step pays at most once. SETTLED and FAILED are terminal; the reserve is released exactly once.

The two triggers

The two forward steps run on different triggers. The worker’s payouts sweep handles only the first. It claims due RESERVED sagas, converts the reserve to USD at the locked rate, calls processor.submitPayout, records the providerRef, and advances RESERVED → SUBMITTED (see submitToProvider).

It does not settle itself. Settlement arrives on the provider’s verified “payout settled” webhook. That webhook maps to a settlePayout operation, which the worker’s inbox drain applies off the request path. That operation runs the SUBMITTED → SETTLED step.

What makes it safe to re-drive

Three guards let the worker retry a step without paying twice.

  • The call to the rail is idempotent, keyed by the saga id (key: saga.id), so a retried submit still pays once. The settlement webhook is deduped on the provider’s event id, so a redelivered settlement settles once.
  • Each state change is a compare-and-set.[2] sagas.advance(id, from, to, patch) moves a saga only if it is still in from, and returns false (changing nothing) otherwise, so two sweeps racing the same saga can’t both advance it (see SagaStore.advance).
  • A stuck payout is given up on. Past maxPayoutAttempts failed attempts, or after maxPayoutAgeMs in SUBMITTED with no settlement webhook, the sweep force-fails it. It flips the saga to FAILED and, in the same transaction, posts the exact reverse of the reservation (debit payout_reserve → credit earned), returning the credits to the seller (see deadLetter).

That force-fail reversal and the operator’s reversePayout share one compare-and-set guard on the saga. A reversal posts the compensating entry only if its compare-and-set wins. A lost compare-and-set means a concurrent settle or reverse already accounted for the reserve, so posting again would return the credits a second time. An operator can force the same early reversal with reversePayout, but only while the money hasn’t already left.

The invariant

A payout moves through its states once, in order, no matter how many times the worker re-drives it or how its triggers race.

Every transition is a compare-and-set against the current state, so a step is applied by exactly one writer. The money posting that accompanies a transition commits in the same database transaction as the state flip, so the books and the record never diverge.

A RESERVED payout pays at most once, and its reserve is released exactly once: by settlement, by timeout-fail, or by an operator reversal, whichever wins the compare-and-set.

How it’s enforced

The enforcement lives in three places, and you can verify each one in the source.

  • The store’s compare-and-set method. SagaStore.advance is a conditional update that returns whether it changed a row, so the handler can detect and stand down on a lost race (see SagaStore).
  • One transaction per step. Each forward step posts its ledger entry and advances its record inside store.transaction, so a partial step rolls back whole: a flipped state with no posting, or a posting with no flip, can’t persist.
  • Idempotent external calls and dedupe. processor.submitPayout is keyed by the saga id (see Processor), and inbound provider events are deduped by event id before they apply, so a redelivered webhook or a retried submit resolves to the same money move at most once.

Operations and ports that rely on it

requestPayout opens the saga; settlePayout and reversePayout move it. The worker’s payouts sweep and its inbox drain drive transitions off the request path.

The saga records persist through the SagaStore sub-store of the Store port, and a payout’s disbursement leaves through the Processor port. The discipline the saga depends on — a compare-and-set on every transition, one transaction per step — is the same machinery that keeps the ledger’s integrity invariants intact under concurrency.

See also

Notes

  1. The saga pattern — one long-lived transaction run as a sequence of smaller, compensatable steps — is from Garcia-Molina and Salem's Sagas paper (SIGMOD 1987). source
  2. Compare-and-set is the storage-level form of compare-and-swap, which shipped with IBM System/370 in 1970; the instruction is set out in IBM patent US 3,886,525 (Brown and Smith, 1975). source