What does "delete" mean in an event-sourced system?

A customer asks you to remove their data. You find their entity stream and delete it. Then you open a completed run and their email address is still in the input. The account lookup step has another copy. Your customer projection still shows their name.

Each record has a different job. Removing one does not remove the others.

Ironflow v0.39.0 introduced data-plane deletion and redaction. v0.42.0 hardened run and step redaction against late blob uploads and stale lifecycle writes. Choosing among these operations starts with identifying which records hold the data and which history you still need to keep.

Follow one customer’s data

Suppose customer-42 has an entity stream. Its customer.registered event contains a name and email address. A workflow triggered by that event copies the payload into its run input, and a durable step records an account lookup result. A projection folds the event into a customer directory.

The email address now exists in several records. Audit payloads may contain it too, depending on capture policy. Large run and step payloads may be offloaded to object storage. Published messages have their own copies in JetStream.

There is no operation that finds every record belonging to a person. You must identify the relevant events, runs, steps, and derived views in your application.

Operation	What it removes or changes	What it preserves
Scheduled retention	Eligible records older than configured windows	Pinned records, live entity-stream history, and stream tombstones
Run deletion	A terminal run, its steps, and associated execution records	Its triggering event, registered deployment records, and separately stored projection state
Stream tombstone	Appends $stream.deleted, drops snapshots, and stops later appends	Earlier events and the tombstone itself
Stream purge	Deletes events below the tombstone and dependent records through cascades	The tombstone and separately stored projection state
Payload redaction	Replaces selected payloads with a placeholder	Record identity, event sequence, entity version, and other nonpayload columns

Retention manages age, not a person’s removal request

Scheduled retention gives the database a standing cleanup policy. Run retention applies to terminal runs. Ordinary event retention applies to non-entity events. Entity-event retention applies only to streams already tombstoned, and only to events below the tombstone. It leaves live entity streams alone.

The windows default to disabled. Enabling one requires at least seven days, and changing it requires a restart. The nightly pruner checks references as well as age: a retained run, undelivered outbox row, pending debounce fire, or outbox dead letter can keep an event past its configured window. An outbox dead letter has no timed expiry of its own.

Our customer’s records may be too young, or still pinned. Waiting for retention does not target their data. Use it for ongoing storage management, with the eligibility rules in the configuration reference.

Delete execution records when you no longer need the run

DeleteRun removes a terminal run and its steps. It refuses an active run. DeleteRuns applies the operation to a filtered batch.

Deleting the customer’s completed registration run removes its execution records. It leaves customer.registered in the event store. It also leaves the registered deployment records, and deleting a parent run does not delete its child runs.

Run deletion is therefore useful when you want to discard execution history. It does not scrub every audit payload or locate other runs containing the same customer data. The run deletion reference documents the cascade and batch behavior.

Tombstone the stream to close it, purge to remove its history

Deleting customer-42 appends $stream.deleted at the next entity version and drops its snapshots. Further appends fail. Earlier events remain readable.

With purge: true, Ironflow also removes the events below the tombstone. The tombstone stays: it records the stream’s final version and prevents later appends. Its payload includes the entity identifier, so a purged stream still retains that identifier.

Purge follows the committed tombstone. If cleanup fails, retry the delete with purge: true. A purge is refused while a run triggered by an event below the tombstone is active; the tombstone has already closed the stream. Once those runs are terminal, purging can cascade into their execution records.

The customer directory still needs attention. A projection receives the tombstone only if its registered event list matches $stream.deleted, and its handler must decide what to remove. Ironflow does not rewrite projection state for you. See the stream deletion reference.

Redact payloads when the record must remain

Sometimes you need the event’s position and version while removing its contents. RedactEvent replaces its whole payload with a marker containing $redacted, the original bytes’ SHA-256 hash, and a redaction timestamp. The event keeps its identity, sequence, name, and entity version.

Replay still encounters that position, but it encounters a placeholder. Your reducer must handle it, using the SDK’s isRedacted helper or Python’s is_redacted. Upcasters skip redacted payloads. Redacting an entity event also drops the stream’s snapshots because their state was derived from those bytes. Preserving replay order does not preserve the ability to reconstruct the old state.

For our customer, RedactEvent leaves the copies in run inputs and audit payloads. RedactRun scrubs the run’s input and output plus audit payloads for the run and its steps. It leaves the step rows’ outputs; RedactStep scrubs a step’s output and original output plus its audit payloads. Run and step redaction require a terminal run. These are separate targets, and the redaction reference describes each one.

In v0.42.0, run and step redaction also commit permanent cleanup markers with the database changes. Later writes cannot restore protected payloads, and a worker revisits the exact blob prefixes even after a previous sweep found them empty. This covers uploads that finish late. Run markers cover run payloads; step markers cover that step’s artifacts. Redacting a run alone does not grant cleanup authority over its step artifacts.

Plaintext can remain temporarily until a later successful sweep. Continued database, blob backend, and cleanup worker availability are required. The sweep interval is not an erasure deadline. The amended data-plane deletion ADR, which requires repository access, records this behavior and the distinction from ordinary best-effort deletion cleanup.

Retention changes what you can rebuild

Now consider an ordinary emitted event, customer.email_changed, feeding a managed projection. Set IRONFLOW_EVENT_RETENTION_DAYS=30 in an environment that has existed for months.

Time	What happens
Day 0	An email-change event enters the event store and updates the projection.
Day 20	That event remains inside the retention window; earlier history may already be outside it.
Day 31	The event is eligible for pruning once its references clear. The existing projection may still contain the value it derived.
Day 40	A full rebuild is refused because it would start before the 30-day horizon.

Rebuilds read database events, filtered by event name. With event retention enabled, Ironflow refuses an affected rebuild whose requested window starts before the horizon, including a dry run. This guard applies even if references have kept older rows around: the server cannot prove a complete history from the surviving rows.

You can explicitly accept shortened history with ironflow projection rebuild customer-directory --from <event-id>, naming an event created at or after the horizon. That rebuild cannot recover earlier values you removed. Choose a retention window with your future rebuild needs in mind. Projections matching only entity events are exempt from this ordinary event-retention guard; tombstoning, purge, and redaction still change the history available to them. The projection rebuild reference explains the refusal and partial-replay option.

Finish at the copies your application owns

For customer-42, choose whether to close and purge the stream or preserve event positions through redaction. Identify the related terminal runs and steps, then delete their execution records or redact the payloads you need removed. Update the customer directory and other derived views explicitly. A rebuild is useful only when the remaining history and your handlers can produce the intended state.

Published JetStream copies age out on their own schedule, up to seven days on EVENTS and PUBSUB. Immutable archives may retain bytes under object lock. Failed ordinary deletion blob sweeps can leave objects behind; durable run and step redaction cleanup retries only its marked prefixes. Copies your application exported to other systems need their own removal paths.

A delete or redaction also leaves an audit row recording that it happened. These operations do not promise immediate erasure of every copy or establish regulatory compliance. They give you specific ways to remove records or payloads, with explicit limits on what remains.

Run the customer example

For contributors with repository access, the customer-registration app reproduces the payload copies in this post. From a checkout of the Ironflow repository with Go and Python 3 installed, run:

go test ./tests/integration -run '^TestDeletionBlogCustomerApp$' -count=1 -v

The runner starts the app and an isolated Ironflow HTTP server, SQLite database, and embedded JetStream. It registers a recorded function and a managed projection, appends the customer’s event, and waits for the run to complete. Then it checks redaction, run deletion, and tombstone/purge independently, including the copies that remain in projection state and JetStream. It shuts down the app and server afterward.

The example’s README also lists the controlled-clock retention, rebuild, blob-cleanup, and SDK tests used to verify the other runtime behavior described here. The example does not provision a locked archive or operate on external exports.