Consumer Groups

Consumer groups enable load-balanced event delivery across multiple consumers. Match patterns are distributed among group members, ensuring each event is processed by exactly one consumer in the group.

Overview

Consumer groups provide:

• Load balancing: Events are distributed across group members on a first-available basis (JetStream pull consumer).
• Durability: Group state persists across server restarts.
• Resiliency: Rejoining a group resumes from the last acknowledged event. Note: groups start with DeliverNewPolicy, so events published before the group was first created are not delivered.
• At-least-once delivery: Combined with Manual Acknowledgment, events are redelivered if a consumer fails.

---

Creating Consumer Groups

Consumer groups must be created before they can be joined. This is an administrative task typically performed via the Go SDK or HTTP API.

@ironflow/node does not expose consumer-group management or subscription APIs. Use the Go SDK, the browser SDK (for joining), or the HTTP API shown below.

Go

client := ironflow.NewClient(ironflow.ClientConfig{
    APIKey: os.Getenv("IRONFLOW_API_KEY"),
})

// Create a consumer group for reliable processing
err := client.CreateConsumerGroup(ctx, ironflow.ConsumerGroupConfig{
    Name:             "order-workers",
    Pattern:          "events:order.>",
    AckMode:          ironflow.AckModeManual,
    MaxRedeliveries:  5,
    RedeliverDelayMs: 10000,
})

CLI

# Not yet available via dedicated CLI command.
# Use cURL to create a group manually:
curl -X POST http://localhost:9123/ironflow.v1.PubSubService/CreateConsumerGroup \
  -H "Content-Type: application/json" \
  -d '{
    "name": "order-workers",
    "pattern": "events:order.>",
    "ack_mode": "manual"
  }'

---

Joining a Consumer Group

Once created, multiple instances of your application can join the group to share the workload.

TypeScript

import { ironflow } from "@ironflow/browser";

// Standard join method
let sub: Awaited<ReturnType<typeof ironflow.joinConsumerGroup>>;
sub = await ironflow.joinConsumerGroup("order-workers", "events:order.>", {
  onEvent: async (event) => {
    console.log("Processing order:", event.data.orderId);
    await sub.ack(event.eventId);
  }
});

Go

// Join an existing group
sub, err := client.JoinConsumerGroup(ctx, "order-workers")
if err != nil {
    log.Fatal(err)
}

for event := range sub.Events() {
    fmt.Println("Processing:", event.Topic)
    sub.Ack(event.ID)
}

---

Load Balancing Behavior

When multiple consumers (e.g., three browser tabs or four Go microservices) join the same group, each event is delivered to whichever member is next available to receive it. The pattern is not strict round-robin — a fast consumer that completes acks quickly may receive more events than a slow one.

If a consumer disconnects, its pending (unacknowledged) events are automatically redelivered to another group member after the visibility timeout (RedeliverDelayMs).

---

Consumer Groups vs. Regular Subscriptions

Feature	Regular Sub	Consumer Group
Delivery	Fan-out (everyone gets everything)	Load-balanced (one member gets it)
Persistence	Volatile (lost on disconnect)	Durable (rejoin to resume)
Acknowledgment	Optional	Required for at-least-once
Use Case	Real-time dashboards, debugging.	Critical workers, background jobs.