Skip to content
Ironflow Docs

Projections

Projections build read-optimized materialized views from event streams. Instead of reading an entity’s full event history every time, a projection continuously processes events and maintains a derived state you can query instantly.

How It Works

Projections are a split responsibility between your SDK worker and the Ironflow server:

  • Your app (SDK) — defines the projection handler and runs it in a streaming or poll loop
  • Ironflow server — stores state, manages NATS consumers, serves queries to your frontend

The SDK registers the projection, receives events (via real-time streaming or polling), runs your handler locally, and saves the resulting state back to the server. The server never executes your handler logic — it just feeds events and persists whatever state the SDK sends back.

This means your projection logic lives in your codebase, versioned with your app, while the server handles durability, partitioning, lag tracking, and real-time distribution to connected clients.

Projection Derivation — showing events flowing through a reducer to build read-optimized state

Two Modes

ModeState locationHandler typeUse case
ManagedIronflow DBPure reducer (state, event) => newStateDashboards, aggregations, read models
ExternalYour databaseSide-effect handler (event) => voidSearch indexes, cache warming, third-party sync

Mode is auto-detected: if you provide initialState, it’s managed. Otherwise it’s external. You can also set mode explicitly.

Managed Projections

A managed projection is a pure reducer — it takes the current state and an event, and returns the new state. Ironflow stores the materialized state in its database.

import { createProjection } from "@ironflow/node";


const orderStats = createProjection({
  name: "order-stats",
  events: ["order.created", "order.completed"],
  handler: (state, event) => {
    switch (event.name) {
      case "order.created":
        return {
          ...state,
          totalOrders: state.totalOrders + 1,
          totalAmount: state.totalAmount + event.data.amount,
        };
      case "order.completed":
        return { ...state, completedOrders: state.completedOrders + 1 };
      default:
        return state;
    }
  },
  initialState: () => ({
    totalOrders: 0,
    completedOrders: 0,
    totalAmount: 0,
  }),
});

Querying State

Once a managed projection is running, query its state via the REST API or SDK:

import { ironflow } from "@ironflow/browser";


const result = await ironflow.getProjection<OrderStats>("order-stats");
console.log(result.state); // { totalOrders: 42, completedOrders: 38, totalAmount: 12500 }
console.log(result.mode); // "managed"

External Projections

An external projection runs side effects — writing to your own database, updating a search index, or syncing to a third-party service. Ironflow only tracks the cursor position (which events have been processed).

import { createProjection } from "@ironflow/node";


const searchIndex = createProjection({
  name: "employee-search-index",
  events: ["employee.*"],
  handler: async (event, ctx) => {
    // Side effect — write to your own system
    await elasticsearch.index({
      index: "employees",
      id: event.data.id,
      body: event.data,
    });
    ctx.logger.info(`Indexed employee ${event.data.id}`);
  },
  // No initialState → auto-detected as external mode
});

Registering Projections

Add projections to your worker configuration. Projections need a persistent poll loop, so they run in pull mode (workers), not push mode (serverless).

import { createWorker } from "@ironflow/node";


const worker = createWorker({
  functions: [processOrder, sendEmail],
  projections: [orderStats, searchIndex],
});


await worker.start();
// Projections automatically register and start polling

If you pass projections to serve() (push mode), a warning is logged. Projections require a persistent process — use createWorker() instead.

Streaming vs Polling

The projection runner supports two event delivery strategies:

StrategyHow it worksLatencyAvailability
StreamingOpens a persistent ConnectRPC server-stream that pushes events in real-timeMillisecondsIronflow ≥ 0.15
PollingPeriodically calls PollProjectionEvents with exponential backoff (1 s → 2 s → 4 s → 8 s → 10 s, reset on success)SecondsAll versions

The worker automatically tries streaming first. If the server returns 404 or 501 (older version), it falls back to polling transparently — no code changes required.

Micro-batching (streaming mode)

In streaming mode the runner groups incoming events into micro-batches to amortize state saves:

  • Batch size trigger — flushes when the batch reaches batchSize events (default 100)
  • Time trigger — flushes 100 ms after the first event in the batch, whichever comes first

This means a burst of 50 events results in a single state save instead of 50 individual writes.

Reconnection

If the stream ends cleanly (e.g., server restart), the runner reconnects after a 1-second delay. On stream errors the delay is 2 seconds. Either way it resumes from the last acknowledged position. No events are lost.

Partitioned Projections

Partition projections by a key extracted from event data. Each partition maintains its own independent state — useful for per-customer, per-tenant, or per-entity projections.

const customerStats = createProjection({
  name: "customer-stats",
  events: ["order.created", "order.completed"],
  partitionKey: "$.data.customerId",
  handler: (state, event) => ({
    ...state,
    orderCount: state.orderCount + 1,
    totalSpend: state.totalSpend + (event.data.amount ?? 0),
  }),
  initialState: () => ({ orderCount: 0, totalSpend: 0 }),
});

Query a specific partition:

const result = await ironflow.getProjection("customer-stats", {
  partition: "cust-456",
});

The partitionKey uses dot-notation JSONPath to extract the key from event data. For example, $.data.customerId extracts customerId from { "data": { "customerId": "cust-456" } }.

Real-Time Subscriptions

Subscribe to projection state updates in the browser. Events are pushed via WebSocket whenever the projection state changes.

import { ironflow } from "@ironflow/browser";


// Subscribe to all updates for a projection
const sub = await ironflow.subscribeToProjection<OrderStats>("order-stats", {
  onUpdate: (state, event) => {
    console.log("Projection updated:", state);
  },
  onError: (error) => console.error(error),
});


// Subscribe to a specific partition
const partSub = await ironflow.subscribeToProjection(
  "customer-stats",
  {
    onUpdate: (state) => console.log("Customer updated:", state),
  },
  { partition: "cust-456" },
);


// Clean up
sub.unsubscribe();
partSub.unsubscribe();

Rebuilding Projections

Rebuild a projection to reprocess all events from the beginning. This is useful when you change your handler logic or need to recover from errors.

await ironflow.rebuildProjection("order-stats");

What happens during a rebuild:

  1. Projection status changes to rebuilding
  2. All stored state is deleted (managed mode)
  3. The rebuild manager resolves a start cursor (0 for full rebuild, or the nats_seq of fromEventId for partial) and records progress markers on the projection registry
  4. The SDK continues its normal stream/poll loop; the server replays historical events from the cursor with phase=scan (on spans projection.rebuild.batch and the ironflow_projection_rebuild_events_applied_total counter) while live events keep flowing
  5. Status returns to active once the rebuild catches up to the live cursor

For external projections, you’re responsible for clearing your own data store before triggering a rebuild.

Pause and Resume

Temporarily pause a projection to stop it from processing new events, then resume when ready.

// Pause
resp, err := http.Post("http://localhost:9123/api/v1/projections/order-stats/pause", "application/json", nil)
if err != nil {
    log.Fatal(err)
}
resp.Body.Close()


// Resume
resp, err = http.Post("http://localhost:9123/api/v1/projections/order-stats/resume", "application/json", nil)
if err != nil {
    log.Fatal(err)
}
resp.Body.Close()

A projection can only be paused when it is active or rebuilding. It can only be resumed when paused.

Deleting Projections

Unregister a projection and remove its stored state:

req, _ := http.NewRequest("DELETE", "http://localhost:9123/api/v1/projections/order-stats", nil)
resp, err := http.DefaultClient.Do(req)
if err != nil {
    log.Fatal(err)
}
resp.Body.Close()

Projection Status

Check the status of a projection:

const status = await ironflow.getProjectionStatus("order-stats");
console.log(status.status); // "active" | "rebuilding" | "paused" | "error"
console.log(status.lastEventSeq); // last processed event sequence
console.log(status.errorMessage); // error details if status is "error"

Status Values

StatusDescription
activeProcessing events normally
rebuildingReprocessing from the beginning
pausedTemporarily stopped
errorFailed — check errorMessage for details

Listing Projections

List all registered projections:

const projections = await ironflow.listProjections();
projections.forEach((p) => {
  console.log(`${p.name}: ${p.status} (${p.mode})`);
});

SQL Projections

SQL projections are query-time (non-materialized) projections defined by a SQL query instead of a handler function. They don’t process events — instead, the stored SQL is executed each time the projection is queried.

SQL projections are registered via the ConnectRPC CreateProjection RPC (not the REST API). Once created, query them like any other projection:

Terminal window
# Query a SQL projection (executes the SQL at query time)
curl http://localhost:9123/api/v1/projections/active-orders

SQL projections:

  • Must use read-only SELECT queries
  • Are subject to the same safety limits as the SQL Query endpoint (timeout, row limits)
  • Have type "sql" in the projection listing (vs "sdk" for handler-based projections)

Configuration Options

OptionDefaultDescription
name(required)Unique projection name
events(required)Event names to subscribe to. Both * and > match the remainder of the subject (order.* and order.> are equivalent — Ironflow rewrites * to > before creating the NATS consumer).
handler(required)Handler function (reducer for managed, side-effect for external)
initialState—Factory function returning initial state (triggers managed mode)
modeauto-detected"managed" or "external"
partitionKey—JSONPath for partition key extraction (e.g., "$.data.customerId")
maxRetries3Max retries per event before error
batchSize100Number of events to fetch per poll

REST API Reference

MethodEndpointDescription
GET/api/v1/projectionsList all projections
GET/api/v1/projections/{name}Get projection state
GET/api/v1/projections/{name}/statusGet projection status
GET/api/v1/projections/{name}/partitionsList partitions (partitioned mode)
POST/api/v1/projections/{name}/rebuildTrigger rebuild
GET/api/v1/projections/{name}/rebuildGet rebuild job progress
POST/api/v1/projections/{name}/cancelCancel an in-progress rebuild
POST/api/v1/projections/{name}/pausePause projection
POST/api/v1/projections/{name}/resumeResume projection
DELETE/api/v1/projections/{name}Delete projection

Query parameters for GET /api/v1/projections/{name}:

  • partition — Return state for a specific partition (default: __global__)

Error Handling

When a handler throws an error:

  1. The SDK retries the event up to maxRetries times (default 3)
  2. After all retries are exhausted, the projection status changes to error
  3. The errorMessage field contains details about the failure
  4. The projection pauses until you either rebuild it or fix and redeploy your handler

Throw NonRetryableError to skip retries and fail immediately.

Managed vs External — When to Use Which

ScenarioModeWhy
Dashboard aggregationsManagedQuery state directly from Ironflow
Search indexExternalData lives in Elasticsearch/Algolia
Cache warmingExternalData lives in Redis/Memcached
Reporting summariesManagedSimple reducer, queryable via API
Third-party syncExternalSide effects to external systems
Per-customer analyticsManaged + partitionedPartitioned state in Ironflow

Domain-Driven Design

Projections implement the Read Model side of CQRS — separate, query-optimized views derived from domain events. See CQRS with Projections for how this maps to Domain-Driven Design.