Architecture

Ironflow is designed as a Control Plane that decouples workflow state from execution. It collapses the traditional backend stack (Queue + Database + Orchestrator) into a single, high-performance binary.

---

The Stack

Ironflow leverages the Go ecosystem to deliver industrial-strength durability with zero external dependencies for local development.

Layer	Technology	Role
Gateway	ConnectRPC	Unified HTTP/1.1 and HTTP/2 gateway for Webhooks and Workers.
Event Bus	NATS JetStream	The internal “central nervous system” for all events and state changes.
State Store	SQLite / Postgres	Persistent storage for workflow runs, steps, organizations, projects, and environments.
Execution	Go Goroutines	Lightweight threads that manage state machines and retry logic.
Control Plane	React + Tailwind	Embedded Dashboard for real-time monitoring and hot-patching.

---

Execution Modes

Ironflow supports two execution modes: push and pull.

Mode A: Push (Serverless)

• Mechanism: Ironflow makes an outbound HTTP POST to your endpoint (Next.js, Lambda, etc.).
• Ideal for: High-concurrency, stateless, short-lived tasks (under 10s).
• Benefit: Scales to zero; works with any HTTP-capable platform.

Mode B: Pull (Workers)

• Mechanism: Your worker process polls Ironflow over HTTP for jobs. An opt-in ConnectRPC streaming worker is also available in Go (NewStreamingWorker), Node (@ironflow/node/worker-streaming, createStreamingWorker), and Python (ironflow.worker.StreamingWorker).
• Ideal for: GPU workloads, video processing, or long-running reasoning chains (minutes/hours).
• Benefit: The worker owns the execution, so there is no serverless timeout to bypass.

---

Durable Execution Engine

At the heart of Ironflow is a Write-Ahead-Log (WAL) approach to workflow logic.

1. Memoization: Before running any step, the engine checks the database. If a successful result exists, it is returned instantly.
2. Persistence: New results are committed to the database before the SDK proceeds to the next step.
3. Observability: Run and step state changes emit system events to NATS, powering the Dashboard and TUI without extra database queries.

---

Internal Networking

Synchronous Triggers (await)

When a client triggers a workflow synchronously (emitSync() or invoke()), Ironflow parks the request goroutine and subscribes to an internal NATS topic: ironflow.{project_name}.{environment_id}.results.run.{run_id}. When the worker completes the final step, the result is published to that topic, waking the goroutine to deliver the HTTP response. The handler also polls durable run state on a 100 ms cadence under the same wait budget, so a lost notification does not force a settled run to wait until timeout.

Smart Concurrency Lanes

Ironflow enforces per-function concurrency limits using lanes. When a function sets a concurrency config (a key such as orgId, resolved against the event payload, plus a limit), the engine derives a lane from the extracted key value and admits at most limit active runs per lane. Runs over the limit wait in a durable dispatch queue rather than being dropped. Lanes and their leases are tracked in the database — not in process memory — so limits hold across a multi-node cluster and survive restarts. This prevents a single tenant from starving the worker pool. See ADR 0037.

---

Security Model

Ironflow operates on an Identity-Embedded model.

• Single-Binary Auth: On first boot, the engine auto-bootstraps a root Organization and an Admin API Key.
• Resource Naming: Every internal resource follows the IRN (Ironflow Resource Name) format: irn:ironflow:{org}:{project}:{type}:{env}:{id}.
• Zero-Trust: Pull workers authenticate to the engine with the same API Key system as the external REST API. Push mode runs the other way — the engine calls your endpoint — so there is no API key on that hop; instead the engine signs the payload with HMAC-SHA256 into X-Ironflow-Signature, and only when IRONFLOW_SIGNING_KEY is set. Verify that header in your handler.

---

Debugging & Observability

Ironflow includes built-in debugging and observability features:

• History Navigation (time-travel debugging): Reconstruct run state at any historical timestamp by replaying recorded events. Available via CLI (ironflow inspect --at, --replay), TUI, SDK, and dashboard.
• History Editing (scoped injection): Pause running workflows at step boundaries, inspect completed step outputs, inject modified data, and resume execution. Supports both push and pull modes.
• Circuit Breaker: Push endpoints are protected by a circuit breaker (per environment, keyed by function ID) that opens when repeated errors occur. State lives in the database (circuit_breaker_states) and is evaluated at dispatch reservation time in the same transaction as the queue + capacity decision, so it is cluster-consistent and survives restarts and rolling deploys. Dashboard shows circuit state badges on the Functions page.
• Continuous History Log (audit stream): For each function with recording enabled (opt-in), an append-only event log of step starts, completions, failures, injections, and saga compensations. This is the foundation that powers history navigation.
• OpenTelemetry & Prometheus: Optional tracing via OTLP gRPC exporter with W3C propagation, and Prometheus metrics at /metrics. Zero overhead when disabled.

---

What Makes it Different?

1. Collapsing the Stack: Replaces Redis (Queue) and Temporal (Orchestrator) with one binary. State lives in embedded SQLite, or in PostgreSQL when you cluster.
2. Continuous History: With recording enabled, steps are recorded in one continuous history, so you can navigate between recorded moments and correct a step in real-time. Recording is best effort under load.
3. Pure Go: No CGO dependencies (ModernC SQLite), making Ironflow portable to any architecture without a toolchain.