NATS JetStream Streams
Ironflow keeps two stores side by side and they answer different questions. The database is the record: every row it holds survives for the life of the environment, and it is what an executor reconciles against after a crash. NATS is transport — five JetStream streams, each with a retention window tuned to its job, plus KV buckets that have no table behind them at all. For events emitted through the API and entity appends, the database transaction commits before the outbox publishes notifications. External NATS ingestion reverses that order: an event exists only in NATS until EventRouter consumes and stores it. Losing that message before storage loses the event unless the producer can replay it.
The five streams
Section titled “The five streams”| Stream | Retention | Max age | Max size (prod) | Subject |
|---|---|---|---|---|
EVENTS | Limits | 7 days | 512 MB | ironflow.*.*.events.> |
STEPS | WorkQueue | 1 day | 512 MB | ironflow.*.*.steps.> |
RESULTS | Limits | 1 hour | 128 MB | ironflow.*.*.results.> |
PUBSUB | Limits | 7 days | 512 MB | public.> |
CANCEL | Limits | 1 hour | 64 MB | ironflow.*.*.cancel.> |
The *.* wildcards match the {projectName}.{envName} segments. PUBSUB is the exception: it is keyed by the environment ID and carries no project segment (#1516).
Retention decides when a message disappears. Under LimitsPolicy a message stays until it ages out or the stream hits its size cap — reading it changes nothing, so two subscribers both see it. Under WorkQueuePolicy the message is deleted the moment it is acked, which is what makes a work queue a work queue.
STEPS is still created with the configuration above, but it no longer carries pull dispatch. Since ADR 0037 (#1206) capacity admission is the only dispatch path: runs are admitted to the dispatch_queue table and workers long-poll GET /api/v1/workers/{id}/jobs over HTTP. Nothing in the server publishes to or consumes from ironflow.*.*.steps.> today.
NATS KV buckets
Section titled “NATS KV buckets”KV buckets are purely NATS-native (the JetStream KV API) with no corresponding DB tables. The SYS_ prefix is hidden from users; APP_ buckets are the ones that show up in the dashboard.
| Bucket | Holds |
|---|---|
SYS_cron_triggers | Cron dedup — the first kv.Create() for a slot wins |
SYS_config_* | Environment-scoped JSON config documents |
SYS_secrets_* | Encrypted environment-scoped secrets |
SYS_debounce_state | Pending debounce arms, per function and key |
SYS_policy_epoch | CEL policy cache epoch, per organization |
SYS_agent_tools | Agent tool definitions registered by the SDK |
APP_{bucketName} | Arbitrary user key/value — one NATS KV bucket each |
SYS_* buckets are keyed by the environment ID verbatim (SYS_secrets_env_default), unlike the ironflow.* subject family above (#2024).
What the database owns
Section titled “What the database owns”events, runs, steps, functions, entity streams, consumer_groups and audit_events are all kept for the life of the environment. For API emits and entity appends, the event and its outbox rows commit together before publication. External ingestion instead calls CreateEvent after consuming from EVENTS, then creates matching runs. Cancellation also commits the terminal run state before publishing its best-effort signal.