Resolve a Reconciliation Exception

A bank statement rarely matches your books exactly. Most lines match. A few do not, and each one is a question: posted to the wrong account, or missed? A timing difference, or a real gap? An answer needs a human, sometimes a person outside your organization.

This tutorial walks through examples/agents/reconciliation-agent/. It builds a pipeline that matches transactions deterministically first, sends the unmatched residue to a model for triage, asks a human to approve contacting the counterparty, waits for their reply, and turns a confirmed answer into a rule the next statement can use without asking anyone again.

The ambiguous statement

The fixture statement carries 100 transactions. 90 of them match a line the application already booked — an exact lookup, no judgment required. The other 10 do not match anything, and they cluster by counterparty: several unmatched lines against the same vendor, in the same period, are one question, not several.

That question is a case: a cluster of unmatched lines against one counterparty for one period. Not one line — a single line rarely carries enough signal to tell a timing difference from a duplicate. Not the whole statement — too coarse to contact anyone about. The reconcile run emits reconciliation.case.requested with an idempotency key for each period/counterparty. The server starts one independent run for that case and reuses it when another statement requests it, even if the requests arrive concurrently. Each case run appends case.opened as its first act. A counterparty who never answers cannot block the other cases. A closed case is also reused while its request event is retained; reopening a case is outside this example’s scope.

Deterministic first is not a shortcut

It is tempting to read the deterministic pass as an optimization — do the cheap thing first, save the model calls. It does more than that. Two things are true at once:

• Correctness. A model is a poor fit for “does this transaction ID appear in this set.” It is probabilistic where the answer is exact. Running it here would mean accepting a nonzero error rate on the 90% of cases that have one clean, checkable answer.
• Cost. The deterministic pass is one step.run over the whole statement — a pure function, no external call, nothing to protect with a step boundary of its own. It costs nothing to run twice. Only the 10 escalated transactions reach an LLM, and reuse (below) shrinks that count further over time.

Two different numbers travel under the word “escalated”, so this tutorial keeps them apart: the fixture escalates 10 transactions, which cluster by counterparty into 5 cases. The run output’s escalated field counts the cases; matchedRatio counts the transactions.

The ratio is proven, not just stated — tests/fixtures.test.ts asserts the fixture is 90 matched and 10 escalated transactions, and the reconcile run logs matchedRatio on every real run. A later change that routed everything to the model would fail that assertion instead of quietly passing.

What the model is, and is not, allowed to do

Every field the model sees for triage passes through one function, redact(), before it gets there:

• amount, currency, and a date offset from the period start — not a calendar date
• a label shape (ADJ-0090 becomes ADJ-<n>) — the pattern the matcher reasons about, not the identifying digits
• a stable, pseudonymous counterparty reference — not the counterparty’s name

Only ADJ-<digits> and INV-<digits> labels produce a shape. All other labels become unknown, so names or memo text embedded in a label cannot cross that boundary. Ironflow’s own CEL policy layer cannot enforce this for you — it is deny-only and subtractive by design (see Policies), which means it can block a field but cannot map or rewrite one. Redaction has to live in application code, which is exactly where redact.ts puts it.

The model’s reply is also constrained on the way out. It returns exactly { classification, confidence, proposedActionId } — a structured object, no free text. proposedActionId is checked against a fixed allowlist before anything downstream uses it. That combination is the injection defense: a hostile or malformed value in the model’s input has no channel to act through, because the only thing the model can produce is a value from a list a human wrote.

The counterparty’s reply gets the same treatment on the way in. Nobody feeds their reply text to the model. redactReply() classifies the reply by keyword locally and hands the agent a classification, not the prose — the reply body is the most realistic injection vector in this whole pipeline, since it comes from outside the organization entirely, and this is why it never reaches a prompt.

Why the approver reads the draft

approve("contact", { payload }) puts on the gate the case id, the counterparty reference, the amounts, the model’s classification and confidence, the proposed action in plain language, and the exact message text that would go out, unedited.

If the approver cannot read the outbound message, the approval gate checks nothing. The gate exists to put a human between a model’s read of an ambiguous situation and a real message reaching a real counterparty. That only means something if the human sees what they are approving, not a summary of it.

The payload is stored as the parked step’s input. client.getRunSteps(runId) returns it on the approve.contact step, and the dashboard shows it under that step’s Input, so an approver reads the draft, the classification, the confidence and the delta before deciding.

pnpm approve -- <runId> true

The approval correlates on the run id, so this releases the case you named and no other.

Rejecting works the same way, with a reason attached to the audit trail:

pnpm approve -- <runId> false "needs a different counterparty contact"

Once approved, the agent sends the message through a tool call keyed on (runId, decisionId) — not step.run’s own memoization, which only protects a step that completed. The gap between “the provider accepted this message” and “the step result is durably recorded” is exactly where a crash could cause a second send. make demo-reconciliation-crash kills a worker inside that gap on a real server and asserts the delivery count stays at one.

What happens when nobody replies

Two things wait for a deadline in this pipeline: the approval gate, and the reply itself. Ironflow’s engine fails a run outright when a waitForEvent times out — it does not resume it. Left alone, an unanswered case would not become a clean business outcome; it would become an engine failure, indistinguishable from a bug.

A cron sweep runs every five minutes, reads a projection of open cases with an explicit stage and deadline, and emits the deadline event for whichever stage a case is stuck in — always before either wait’s own timeout could fire:

• stuck at the approval gate -> the sweep emits a genuine rejection, { approved: false, approver: "system:ttl", reason: "no approver responded" }
• stuck waiting on a reply -> the sweep emits { kind: "timeout" } on the same event name the real reply arrives on

Either way, the run resumes through its own code and completes normally with case.unresolved in its output. “The counterparty did not answer” is a business fact, not a crash. Anything that wants to alert on it subscribes to the event instead of watching for a failed run.

The sweep publishing into the approval-event subject is a demo affordance. A real deployment must not use it unchanged — anything with permission to emit that event could otherwise settle a pending approval. tests/sweep.test.ts asserts the invariant that bounds it in the example: the sweep can never emit approved: true, and because the gate correlates on the run id, one deadline rejects one run.

The sweep’s schedule is observable; its emit path is not. DEADLINE_HOURS is 24, so every tick in a walkthrough logs sweep complete { emitted: 0 }. deadlineEventFor is unit-tested in isolation, and nothing drives a deadline to expiry against a live server.

From a confirmed resolution to a rule

When a reply does arrive, the agent classifies it, resumes, and appends case.resolved with a learned rule attached. The predicate grammar is closed and human-written — label prefix, amount tolerance, date window — but this example only ever constructs the label-prefix case, fixed in code as ADJ-. What the confirmed reply supplies is the rule’s action id, checked against the same allowlist the model’s own proposal passed. The model never invents the rule grammar; it proposed a classification, a human approved contacting the counterparty, and the counterparty’s own answer authorizes which action the rule takes. A reply that does not confirm a specific, allowlisted action ends the case unresolved instead — no rule is learned from an unclear answer.

A separate, curated projection subscribes to case.resolved and nothing else — not the five other case events this pipeline emits. That is what makes “this memory is not raw audit history” a property you can check by reading the projection’s events array, not only a claim in this document.

The next statement’s deterministic pass loads these rules before it runs. A transaction that matches a learned rule is matched right there, in the same step that does exact lookups — it never reaches the model at all. Run the sequence yourself:

pnpm trigger                                  # statement #1: matchedRatio 0.90, escalated 5 (cases)
# approve one case, then:
pnpm reply -- <caseId> "the amount posted to our sibling account"
pnpm trigger                                  # statement #2, same fixture

Statement #2’s output carries rulesApplied non-empty and a higher matchedRatio than statement #1, with escalated dropping from 5 cases to 4. How far the ratio moves depends on which case you resolved, because the residue is spread unevenly across counterparties — 0.91, 0.92 or 0.93 for a cluster of 1, 2 or 3 transactions. That is what “memory reused across cases” means concretely: a rising match ratio, not a growing prompt.

Next steps

• examples/agents/reconciliation-agent/ — the full source, plus the stated bounds this design accepted
• Survive a Crash — the crash-resume mechanism this example builds on
• Code-review agent — approve() against an internal approver, without an external reply
• Policies — why redaction has to live in application code today