Consumer Groups

Consumer groups enable load-balanced event delivery across multiple consumers. Match patterns are distributed among group members, ensuring each event is processed by exactly one consumer in the group.

Overview

Consumer groups provide:

• Load balancing: Events are distributed across group members on a first-available basis (JetStream pull consumer).
• Durability: Group state persists across server restarts.
• Resiliency: Rejoining a group resumes from the last acknowledged event. Note: groups start with DeliverNewPolicy, so events published before the group was first created are not delivered.
• At-least-once delivery: Combined with Manual Acknowledgment, events are redelivered if a consumer fails.

---

Creating Consumer Groups

Consumer groups must be created before they can be joined. This is an administrative task performed via the Go, Node or Python SDK, or the HTTP API.

Note

@ironflow/node exposes consumer-group management on createClient() as client.consumerGroups.{create, get, list, update, delete}. @ironflow/browser exposes the same five methods on ironflow.consumerGroups, plus the joinConsumerGroup helper for joining. Either SDK can also pass a consumerGroup option to a normal subscribe: createSubscriptionClient().subscribe() in Node, ironflow.subscribe() in the browser (createSubscriptionClient is a Node-only export).

Create the group first, with the right ackMode. A subscribe that names a group that does not exist creates it from the subscribe pattern in auto ack mode. This holds for the consumerGroup option and for the browser joinConsumerGroup(name, pattern), on both the WebSocket and the ConnectRPC transport. A mistyped group name then makes a new auto group, where ack() does nothing, instead of failing.

Go

client := ironflow.NewClient(ironflow.ClientConfig{
    APIKey: os.Getenv("IRONFLOW_API_KEY"),
})

// Create a consumer group for reliable processing
group, err := client.CreateConsumerGroup(ctx, ironflow.ConsumerGroupConfig{
    Name:             "order-workers",
    Pattern:          "events:order.>",
    AckMode:          ironflow.AckModeManual,
    MaxRedeliveries:  5,
    RedeliverDelayMs: 10000,
})

Python

import os

from ironflow import IronflowRPC
from ironflow.rpc.v1 import AckMode, CreateConsumerGroupRequest

with IronflowRPC(api_key=os.environ["IRONFLOW_API_KEY"]) as rpc:
    # Create a consumer group for reliable processing
    group = rpc.pubsub.create_consumer_group(CreateConsumerGroupRequest(
        name="order-workers",
        pattern="events:order.>",
        ack_mode=AckMode.MANUAL,
        max_redeliveries=5,
        redeliver_delay_ms=10000,
    ))

TypeScript (Node)

import { createClient } from "@ironflow/node";

const client = createClient({ apiKey: process.env.IRONFLOW_API_KEY });

const group = await client.consumerGroups.create({
  name: "order-workers",
  pattern: "events:order.>",
  ackMode: "manual",
  maxRedeliveries: 5,
  redeliverDelayMs: 10000,
});

CLI

# Not yet available via dedicated CLI command.
# Use cURL to create a group manually:
curl -X POST http://localhost:9123/ironflow.v1.PubSubService/CreateConsumerGroup \
  -H "Content-Type: application/json" \
  -d '{
    "name": "order-workers",
    "pattern": "events:order.>",
    "ack_mode": "ACK_MODE_MANUAL"
  }'

---

Joining a Consumer Group

Once created, multiple instances of your application can join the group to share the workload.

TypeScript

import { ironflow } from "@ironflow/browser";

// Standard join method
let sub: Awaited<ReturnType<typeof ironflow.joinConsumerGroup>>;
sub = await ironflow.joinConsumerGroup("order-workers", "events:order.>", {
  onEvent: async (event) => {
    console.log("Processing order:", event.data.orderId);
    await sub.ack(event.eventId);
  }
});

Go

// Join an existing group
sub, err := client.JoinConsumerGroup(ctx, "order-workers")
if err != nil {
    log.Fatal(err)
}

for event := range sub.Events() {
    fmt.Println("Processing:", event.Topic)
    sub.Ack(event.ID)
}

Python

from ironflow import IronflowRPC
from ironflow.rpc.v1 import JoinConsumerGroupRequest

with IronflowRPC(server_url="http://localhost:9123") as rpc:
    # Join an existing group
    for event in rpc.pubsub.join_consumer_group(JoinConsumerGroupRequest(group_name="order-workers")):
        print("Processing:", event.topic)

For a MANUAL group, acknowledge each event with ack_event. See the Python example in Manual Acknowledgment.

Joining a group that does not exist fails with NotFound on the JoinConsumerGroup RPC, in Go JoinConsumerGroup and on the stream below. The browser joinConsumerGroup helper does not fail: it creates the group (see the note above). Go and Node can also join over one ConnectRPC stream that carries the acks; see Ack over one stream.

---

Load Balancing Behavior

When multiple consumers (e.g., three browser tabs or four Go microservices) join the same group, each event is delivered to whichever member is next available to receive it. The pattern is not strict round-robin — a fast consumer that completes acks quickly may receive more events than a slow one.

If a consumer disconnects, its pending (unacknowledged) events are automatically redelivered to another group member after the visibility timeout (RedeliverDelayMs).

With the default buffer backpressure, an event that reaches a member whose buffer (MaxInflight) is full goes to the first member that has room, and the group waits up to RedeliverDelayMs for room. If no member frees room in that time, the event is redelivered after RedeliverDelayMs. Each redelivery spends one of the MaxRedeliveries attempts, and an event still not placed after the last attempt is dropped. Each attempt waits up to RedeliverDelayMs for room and then RedeliverDelayMs before the next one, so with the defaults (3 redeliveries, 5 seconds) an event is dropped after about 35 seconds with no free member. Events waiting for redelivery also count toward MaxInflight, so a group that has that many waiting pauses new deliveries until they are placed. Raise MaxRedeliveries or RedeliverDelayMs for consumers that can fall further behind.

When the last member leaves, the group stops pulling. Events published in the meantime wait in the group’s durable consumer, spend no redelivery attempts, and go to the next member that joins.

---

Consumer Groups vs. Regular Subscriptions

Feature	Regular Sub	Consumer Group
Delivery	Fan-out (everyone gets everything)	Load-balanced (one member gets it)
Persistence	Volatile (lost on disconnect)	Durable (rejoin to resume)
Acknowledgment	Automatic	Manual ack for at-least-once
Use Case	Real-time dashboards, debugging.	Critical workers, background jobs.