NATS JetStream Streams

Ironflow keeps two stores side by side and they answer different questions. The database is the record: every row it holds survives until a retention policy or an explicit delete removes it, and it is what an executor reconciles against after a crash. NATS is transport — five JetStream streams, each with retention limits tuned to its job, plus KV buckets, nearly all of which have no table behind them. For events emitted through the API and entity appends, the database transaction commits before the outbox publishes notifications. External NATS ingestion reverses that order: an event exists only in NATS until EventRouter consumes and stores it. Losing that message before storage loses the event unless the producer can replay it.

The five streams

Stream	Retention	Max age	Max size (prod)	Subject
EVENTS	Limits, DiscardNew	No age expiry	512 MiB / 50,000 messages	ironflow.*.*.events.ingest.> and ironflow.*.*.events.resume.>
STEPS	WorkQueue	1 day	512 MB	ironflow.*.*.steps.>
RESULTS	Limits	1 hour	128 MB	ironflow.*.*.results.>
PUBSUB	Limits	7 days	512 MB	public.>
CANCEL	Limits	1 hour	64 MB	ironflow.*.*.cancel.>

Those sizes are the file-storage caps. The Starter, Cluster, HA, and multi-node Compose files set NATS_FILE_STORAGE=true, which selects them. Memory storage uses smaller caps, and a restart of every NATS server drops the messages.

The *.* wildcards match the {projectName}.{envID} segments: the project name and the environment ID (#2864). The ingest router also accepts an environment name from external publishers. PUBSUB is the exception: it puts the environment ID after the public. prefix and carries no project segment (#1516).

Retention decides when a message disappears. Under LimitsPolicy the configured age and discard policy control retention — reading it changes nothing, so two subscribers both see it. Under WorkQueuePolicy the message is deleted the moment it is acked, which is what makes a work queue a work queue.

STEPS is still created with the configuration above, but it no longer carries pull dispatch. Since ADR 0037 (#1206) capacity admission is the only dispatch path: runs are admitted to the dispatch_queue table and workers long-poll GET /api/v1/workers/{id}/jobs over HTTP. Nothing in the server publishes to or consumes from ironflow.*.*.steps.> today.

EVENTS rejects new publications at capacity; ACK does not free space. Payloads are reclaimed only after durable terminal proof and verified absence. See NATS acceptance.

NATS KV buckets

KV buckets use the JetStream KV API. Most have no corresponding DB table. Two are backed by one: SYS_policy_epoch copies organizations.policy_epoch, and pending_debounce_fires backs crash recovery for SYS_debounce_state. The SYS_ prefix is hidden from users; APP_ buckets are the ones that show up in the dashboard.

Bucket	Holds
SYS_cron_triggers	Best-effort fired-slot history; database cursors own admission
SYS_config_*	Environment-scoped JSON config documents
SYS_secrets_*	Encrypted environment-scoped secrets
SYS_debounce_state	Pending debounce arms, per function and key
SYS_debounce_fired	Order time of the last event that fired, per debounce key
SYS_policy_epoch	Policy-set cache epoch, per organization
SYS_agent_tools	Agent tool definitions registered by the SDK
APP_{bucketName}	Arbitrary user key/value — one NATS KV bucket each

SYS_* buckets are keyed by the environment ID verbatim (SYS_secrets_env_default), never by a name derived from it (#2024).

Write durability

Most KV buckets and direct PUBSUB publishes have no database copy, so JetStream’s disk is their only record. The nats-server default acknowledges a write at once and fsyncs every two minutes. An OS crash or power loss in that window rolls back writes that were already acknowledged: a rotated secret comes back, for example. The embedded server therefore fsyncs every write before it acknowledges it, whatever the replica count. Replication is not a substitute, because a correlated crash, such as a power cut to every node, can hit all replicas inside one window. Operators can choose an interval instead, for throughput. External NATS servers need sync_interval: always in their own config. See ADR 0134 and Write durability.

Consumer limits

Ironflow caps EVENTS, STEPS, RESULTS and CANCEL at 1,000 consumers each and PUBSUB at 100,000. Shipped servers set jetstream.limits.default_max_consumers: -1, so KV watches and listings remain unlimited. External NATS 2.15 or later needs the same setting on every server. Limit errors name the stream and the relevant settings. See External NATS configuration.

What the database owns

events, runs, steps, functions, entity streams, consumer_groups and audit_events are all kept until a retention policy or an explicit delete removes them (see ADR 0092). For API emits and entity appends, the event and its outbox rows commit together before publication. External ingestion instead calls CreateEvent after consuming from EVENTS, then creates matching runs. Cancellation also commits the terminal run state before publishing its best-effort signal.