- Posts
- A Replay Count Is Not a Position
A Replay Count Is Not a Position
Your dashboard subscribes to events:order.* and runs for a week. On Tuesday a
load balancer recycles and the socket closes for ninety seconds. Forty orders are
placed while it is gone.
The client reconnects. What should it ask for?
For most of Ironflow’s life there was one answer, and it was a number:
await ironflow.subscribe("events:order.*", { replay: 100, onEvent: handleOrder,});The count is fixed, the gap is not
Section titled “The count is fixed, the gap is not”replay counts backwards from the tail. That is the right tool for a first
subscribe, when you want some history and do not much care how much. It is the
wrong tool for a reconnect.
Ninety seconds cost you forty events, so replay: 100 returns those forty and
sixty you have already processed. A twenty-minute outage costs you three hundred,
and the same replay: 100 returns the most recent hundred and leaves two hundred
behind. Same code, same option, opposite failure. Nothing in the delivered events
tells you which one just happened.
Ask for a position
Section titled “Ask for a position”A position does not care how long you were away.
TypeScript
Section titled “TypeScript”await ironflow.subscribe("events:order.*", { startAfterSequence: lastDeliveredSequence, onEvent: handleOrder,});cursor := uint64(1402)sub, err := subClient.Subscribe(ctx, "events:order.*", &ironflow.SubscribeOptions{ StartAfterSequence: &cursor,})Go takes a *uint64 rather than a uint64 deliberately. An explicit 0 means
from the beginning of the stream, and that has to stay distinct from not asking
for a position at all.
Setting a cursor is also the opt-in. There is no second flag to turn on, because a cursor that reconnect ignores has no use. A subscription carrying a position re-anchors after every event it hands you. A subscription without one keeps the behavior it has always had.
Where the number comes from
Section titled “Where the number comes from”The sequence arrives on the event, when you ask for metadata. Setting a cursor turns metadata on over WebSocket for you, since a reconnect cannot advance without it.
A live subscription advances its own cursor. You record the sequence so the next process can start where this one stopped:
let cursor = loadCursor(); // undefined on the first run
await ironflow.subscribe("events:order.*", { startAfterSequence: cursor, onEvent: (event) => { handleOrder(event); cursor = event.meta?.sequenceExact ? BigInt(event.meta.sequenceExact) : event.meta?.sequence; },});onEvent returns void and is not awaited, so an async handler runs
unsupervised and two of them can finish out of order. Keep it synchronous, as
above, or take ownership of the ordering yourself. Persisting a cursor from a
floating promise is how you write a sequence for work that has not finished.
sequenceExact is the same number as a decimal string, and it exists because
JavaScript cannot hold all of a uint64. Past Number.MAX_SAFE_INTEGER a
sequence rounds, and rounding upward skips an event on resume: you restart after
an event you were never given. Parsed to a bigint, the cursor you send back is
the sequence you were sent.
Reconnect had to learn your subscription’s name
Section titled “Reconnect had to learn your subscription’s name”The server side of this shipped one release before the Go and TypeScript SDKs could use it. Python exposed the cursor first. Building the other two turned up three things that had been wrong for longer than the feature had existed.
SubscriptionEvent.sequence was zero on every pub/sub path, for the whole life of
the API. The server assigns the sequence and returns it on the acknowledgement,
after the payload is already on the wire, so neither construction site could set
it, and no test asserted otherwise. A cursor is not buildable on a field that is
always zero, so this had to be fixed before anything else.
Go reconnects dropped the options that gave a subscription its meaning. Filter,
ConsumerGroup, IncludeMetadata, AckMode, Backpressure and Namespace were
all left behind, so a filtered load-balanced subscription came back as plain
fan-out and stayed that way. It kept delivering, which is what made it hard to
notice. If you added a downstream filter or your own deduplication to compensate,
take it out.
TypeScript replayed the caller’s original window after every reconnect, so a flaky connection re-ran the same historical events on each redial.
Preserving those fields was still not enough. A WebSocket reconnect receives a fresh server-owned subscription ID, so the client has to bind the replacement before it accepts the next frame. Without that, a successful resubscribe stranded the handle on an ID the server had forgotten, and unsubscribing acted on nothing.
What this does not do
Section titled “What this does not do”A fan-out subscription without a cursor can miss events. It reconnects at the
current tail. Anything published while it was offline is gone, and it will not be
mentioned. This is the shipped behavior and it is kept on purpose, because
tracking sequences automatically would change duplicate and gap handling for
callers who never asked for it. If losing events across a disconnect matters to
you, set startAfterSequence. Nothing else turns it on.
Delivery is at least once. The event in flight when the transport fails can be delivered again. Handlers have to tolerate seeing the same event twice.
A cursor cannot be combined with replay or a consumer group. Every server
transport returns INVALID_ARGUMENT rather than merging them quietly. A consumer
group already has a durable position that the server owns, and replay is a count,
which is the thing a cursor exists to replace.
Setup and the reconnect rules are in the subscriptions guide, the options are in the Go SDK and Node SDK references, and the contract both SDKs are held to is recorded in ADR 0064.
The gap was never the problem. Asking for it by size was.