Event Sourcing

Ironflow provides native event sourcing primitives that allow you to derive application state from an append-only log of events.

Why Event Sourcing?

Traditional databases only store the “current state.” Ironflow stores every change that led to that state. This gives you:

• Auditability: A complete history of every event appended to the stream.
• Time Travel: Reconstruct state at any point in the past.
• Analytics: Build new data models (projections) months after events occurred.

Core Concepts

Concept	Description
Entity Stream	An ordered log of events for one ID (e.g., order_123).
Optimistic Concurrency	Ensures writes only happen if the version matches.
Upcasters	SDK-side functions that migrate old event data to the current schema when a function receives its event. Stored events are never rewritten.
Global Event Log	Every event, in nats_seq order, across all entity streams — the source projections fold over.
Projections	Read models built by folding the global event log across many streams.

---

---

Quick Example (Node.js)

import { createClient } from "@ironflow/node";

const ironflow = createClient({ apiKey: process.env.IRONFLOW_API_KEY });

// 1. Append an event (auto-creates the stream)
const { entityVersion } = await ironflow.streams.append("order-123", {
  entityType: "order",
  name: "order.created",
  data: { total: 99.99 },
}, { expectedVersion: 0 }); // 0 = ensure this is the first event

// 2. Read the history
const { events, totalCount } = await ironflow.streams.read("order-123");
console.log(`Stream version is now: ${entityVersion} (${totalCount} events)`);

Removal: what “append-only” actually promises

An entity stream is append-only in the sense that matters for correctness — nothing rewrites an event’s position, version or ordering, and nothing is deleted as a side effect of ordinary use. It is not an unconditional promise that a row lives forever. Three operations remove data on purpose, and every one is irreversible and admin-gated (ADR 0092):

Operation	What goes	What survives
Delete a stream	A $stream.deleted tombstone is appended; later appends are refused and snapshots are dropped. With purge, every event row below the tombstone goes too.	The tombstone row. Reads keep answering.
Retention	IRONFLOW_ENTITY_EVENT_RETENTION_DAYS prunes events below a tombstone, on tombstoned streams only, that nothing still references.	Live streams, always.
Redact an event	The event’s data is replaced with a placeholder, and the stream’s snapshots are dropped.	The row, its id, name, sequence and entity version.

The distinction between the first two and the third is the one to hold onto: delete removes rows, redaction removes bytes. After a redaction the stream still has the same number of events at the same versions — a replay hands the reducer a value at every position it had before. That value is:

{"$redacted": true, "sha256": "<hex of the original bytes>", "redactedAt": "<RFC 3339>"}

so replay ordering and optimistic concurrency are unaffected. The SDK upcaster chains skip a redacted payload rather than migrate it: the placeholder has no fields to migrate, and an upcaster that rebuilt the object around them would strip the very marker the guard below depends on.

A reducer that trusts its own shape will read missing fields rather than crash — and silently fold nothing into state. Guard it with isRedacted (is_redacted in Python, IsRedacted in Go):

import { isRedacted } from "@ironflow/node";

function reduce(state, event) {
  if (isRedacted(event.data)) return state; // keep what we had
  return { ...state, email: event.data.email };
}

What removal does not reach

Ironflow removes what it owns: rows and their offloaded blobs. It does not reach into state you derived from them.

• Projection state is not rewritten. A projection’s read model is yours; the engine never edits it, on a stream delete (Core Concepts above) or on a redaction. A projection that folded the removed payload into its state keeps it until you rebuild the projection or handle the removal in your reducer. Managed projections rebuild with ironflow projection rebuild.
• Runs are separate. A run’s input holds a copy of the event payload that triggered it, and a step’s output holds whatever that step produced. Redacting the event reaches neither — use ironflow run redact and ironflow run redact-step.
• Published copies leave on their own schedule: the PUBSUB stream keeps a message at most 7 days, and the EVENTS stream keeps one until a durable terminal outcome deletes it.
• A failed blob sweep leaves the object behind. Offloaded payloads are removed on a best-effort basis after the row is scrubbed; a failure is logged and nothing reclaims the object automatically.

Full mechanics: Deleting a Stream and Redacting an Event’s Payload.

Guides

• Entity Streams — Appending, reading, and concurrency.
• Projections — Building read-optimized views.
• Event Versioning — Schema evolution via Upcasters.