Ironflow Multi-Node Cluster

Ironflow Multi-Node Cluster An architecture diagram generated by Archify. Load balancer · no session affinity · Architecture component Load balancer no session affinity Pull workers · claim from any node · Architecture component Pull workers claim from any node Node 1 · ironflow serve --node-id node-1 · Ironflow nodes — stateless, interchangeable Node 1 ironflow serve --node-id node-1 Node 2 · ironflow serve --node-id node-2 · Ironflow nodes — stateless, interchangeable Node 2 ironflow serve --node-id node-2 Node 3 · ironflow serve --node-id node-3 · Ironflow nodes — stateless, interchangeable Node 3 ironflow serve --node-id node-3 STEPS stream · WorkQueuePolicy, consumed once · External NATS JetStream cluster STEPS stream WorkQueuePolicy, consumed once EVENTS stream · entity + system events · External NATS JetStream cluster EVENTS stream entity + system events KV SYS_cron_triggers · one key per slot · External NATS JetStream cluster KV SYS_cron_triggers one key per slot PostgreSQL · runs, steps, entities, RBAC · PostgreSQL — shared state and dispatch · required for a cluster PostgreSQL runs, steps, entities, RBAC required for a cluster dispatch_queue · lane counter + fenced leases · PostgreSQL — shared state and dispatch dispatch_queue lane counter + fenced leases HTTP / gRPC, any node long-poll GET /workers/{id}/jobs step work items publish + consume ironflow.*.*.events.> atomic Create, first node wins the slot read + write run state claim FOR UPDATE SKIP LOCKED Ironflow nodes — stateless, interchangeable External NATS JetStream cluster PostgreSQL — shared state and dispatch Legend Backend Database Message bus External

Why the nodes are interchangeable

  • • Every node runs the same binary and holds no run state of its own
  • • Node 2 is drawn wired up for legibility; node 1 and node 3 hold the identical set
  • • A pull worker may take a job from any node, and heartbeats every 30s

Coordination mechanisms

  • • Cron dedup: NATS KV atomic Create on SYS_cron_triggers, first writer wins the slot
  • • Scheduler: PostgreSQL FOR UPDATE SKIP LOCKED, one node per step
  • • Job dispatch: the DB-backed dispatch_queue allocates execution_seq and mints a fenced lease (ADR 0037)

Recovery and limits

  • • Stale-claim sweep runs every 60s and reclaims after IRONFLOW_STALE_CLAIM_THRESHOLD (default 2m)
  • • Keep that threshold above the 30s heartbeat plus margin, or live claims get reclaimed
  • • PostgreSQL is required — SQLite supports single-node crash-resume only (#613)
  • • SYS_config_* and SYS_secrets_* KV buckets also live in this NATS cluster, keyed by environment ID