Ironflow v0.52.0

Ironflow v0.52.0 changes how runs retry. A run now retries up to 25 times and then pauses instead of failing. You can also throttle how fast a function starts runs and tune the circuit breaker. Read the breaking changes before you upgrade: 13 items need action.

Breaking Changes

• Retries pause, not fail — A run retries up to 25 times by default (it was 3), from 1s, doubling, capped at 5 minutes. Then it pauses with reason retries_exhausted and keeps its completed steps. TriggerSync and InvokeSync return paused at once. ironflow run resume grants a fresh budget. Non-retryable errors still fail at once. Upgrading: set onExhausted: fail on a function whose callers or system.run.failed subscribers must see the final failure. Functions registered before the upgrade keep their stored 3-attempt policy until the SDK registers them again (#3151)
• Permanent push statuses fail on the first response — A push endpoint answer of 400, 401, 403, 405 or 422 fails the run at once. Other 4xx statuses (404, 408, 409, 425, 429) and every 5xx still retry. spec.engine.retry.nonRetryableStatuses replaces the list; an empty list retries every status (#3158)
• One durable attempt makes one HTTP call — The push transport also retried inside the dispatch slot, so maxAttempts: 3 meant up to 9 calls. It now retries in the slot once, after about 100ms, and only when the connection failed before any response. Every other failure waits for a durable retry (#3149)
• SDK projection runners need the projection lease — The Go, Node and Python runners stop with SERVER_TOO_OLD on a server without AcquireProjectionLease. They stop before any Poll, Stream, Save or Ack. Node and Python run() raise ServerTooOldError; the Go runner logs the error. The worker keeps running its functions. Upgrading: run a server at v0.44.0 or later before you move runners to this SDK. A server at v0.51.0 or later refuses a runner on an SDK older than v0.44.0, so upgrade every runner replica to v0.44.0 or later first (#504)
• A reused pub/sub idempotency key returns Duplicate after 2 minutes — PubSubService/Publish kept keys only in the JetStream duplicate window, so a late retry published a second message. The server now stores each key in the new pubsub_idempotency_keys table and returns the first publish’s event ID and sequence. Rows follow IRONFLOW_RUN_RETENTION_DAYS (default: keep forever) (#3170)
• DEPLOYMENT_SNAPSHOT_SCHEMA_VERSION in @ironflow/core is now 2 — registerDeployment declares 2 only for a snapshot that uses circuitBreaker, throttle or retry.onExhausted, and 1 otherwise. Upgrading: if you pass the constant as snapshotSchemaVersion yourself, pass 1 unless the snapshot uses one of those fields
• The Go SDK no longer defaults FunctionConfig.Retry and Timeout — Unset fields are not sent, so the server defaults apply. fn.Config.Retry can be nil after CreateFunction. Upgrading: set Retry and Timeout explicitly if you relied on the old values (3 attempts, 1s initial delay, factor 2.0, 5m max delay, 10m timeout), and nil-check fn.Config.Retry
• Go SDK Default* constants are removed — DefaultFunctionTimeout, DefaultRetryMaxAttempts, DefaultRetryInitialDelay, DefaultRetryBackoffFactor and DefaultRetryMaxDelay had no effect. Upgrading: use your own values, or leave Retry and Timeout unset (#3174)
• "pending" is removed from RunStatus in @ironflow/core — The engine stopped producing it. Upgrading: delete "pending" branches in code that switches on RunStatus. ListRunsOptions.status now takes any RunStatus (#3174)
• The metric ironflow_file_reclaim_aborted_total is removed — It was never incremented. Upgrading: delete alerts and dashboard panels that name it (#3174)
• The Helm value ingress.blockMetrics is removed — It had no effect. Upgrading: delete it from your values files (#3174)
• JSON mode exits 1 for a failed run — ironflow emit --wait --json and ironflow invoke --json printed the result and exited 0 when a run failed, so emit x --wait --json && next ran next after a failure. They now print the full result and exit 1. Upgrading: a script that expects exit 0 for a failed run must handle exit 1 (#3199)
• $enc and $redacted are reserved payload markers — A JSON object whose only member is $enc is reserved at any depth, and a top-level $redacted member is reserved. Event doors, invokes, step patches and injections, and worker step outputs and run results now reject a payload with a top-level $redacted member. $enc gets a meaning in a later release. Upgrading: rename either key in your own payloads (#3197)

Features

• Start-rate throttle per function — throttle: {limit, period_ms, key} caps how many new runs of a function start per rolling period, per key value when key is set. Runs over the limit wait in waiting_for_capacity and start in arrival order. Nothing is dropped. Retries and resumed runs are never throttled. The count is exact across engine nodes, in push and pull mode. A key that does not resolve to a scalar fails the run with THROTTLE_KEY_UNRESOLVED. New metric: ironflow_throttle_delayed_total (#3160)
• Configurable circuit breaker — spec.engine.circuitBreaker.{failureThreshold,openDuration,maxOpenDuration} set the engine-wide policy (defaults 5, 60s, 10m). A function’s circuit_breaker setting overrides any of them. The server config accepts at most 24h for either duration (#3159)
• Retry-After support — A push endpoint’s Retry-After on a 429 or 503 sets the delay before the next attempt. So does retry_after_ms from an SDK RetryAfterError, on push and pull. The delay is capped at the retry policy’s max delay. Retry-After: 0 or a past date never allows back-to-back attempts (#3158)
• Retry, breaker and throttle options in the Go, Node and Python SDKs — Set onExhausted / on_exhausted (pause or fail), circuitBreaker and throttle on a function. RetryAfterError (NewRetryAfterError in Go) asks the engine to wait a chosen delay. RunInfo carries the run’s retry budget; JS and Python add maxAttempts / max_attempts. With on_exhausted: fail, a retryable error on the final attempt runs the compensations. These options need a server that supports them (#3157)
• New retry settings and metric — spec.engine.retry.onExhausted and spec.engine.retry.nonRetryableStatuses in YAML config. ironflow_runs_paused_exhausted_total{function,environment} counts runs paused because their attempts ran out (#3158)

Fixes

• spec.engine.retry.* now applies — Registration stored the built-in defaults on every function, so the YAML settings never reached a run. An unset retry field now resolves to the server default when each run is created (#3151)
• One retry policy on every run path — Event, cron, invoke, debounce and child runs disagreed when a function set no retry. Resuming a run whose attempts ran out now grants new attempts, also when its last completed step was a sleep or a wait (#3152)
• ironflow invoke and ironflow emit --wait exit 1 for a run paused after exhausting its retries — The result has status PAUSED and the last error. ironflow run wait and the MCP ironflow_wait_for_run tool return such a run at once. The Helm chart gains a RunsPausedRetriesExhausted alert (#3158)
• Runs created before the upgrade still fail when attempts run out — They have no recorded exhaustion policy. The pause default applies only to runs created after the upgrade
• Server panic on shutdown — The server could panic with “Write called after Handler finished” when it stopped while a streaming worker disconnected. The engine now drops the stream when the worker’s connection ends (#3167)
• step.publish publishes once on a step retry — The Node, Go and Python SDKs send a default idempotency key (step- plus SHA-256 of the step ID) when you pass none. An explicit key still wins. The server dedups the key durably (#3166)
• Circuit breaker fixes — The dispatch gate uses the configured window, not a fixed 60 seconds. ironflow circuit-breaker reset clears the open gauge and publishes the closed event, so PushEndpointCircuitOpen stops firing. A probe whose run is cancelled or held before its request is sent is released at once (#3153)
• Go RetryAfter reaches the push wire — A handler error that carried RetryAfter lost the delay before the response. The Go client also clamps a negative or past Retry-After to 0
• IronflowPolicySetCompileErrors alert links to a public docs page — Its runbook_url pointed at a private runbook (#3172)
• No usage text after a run outcome — A failed run from emit --wait or invoke printed the full command usage on stderr, and in --json mode that was the only stderr output. Usage text now prints for usage errors only: a bad flag, wrong arguments or a missing required flag. It no longer prints after a run outcome or a runtime error (#3199)

Improvements

• The circuit breaker open window doubles after each failed probe, up to 10 minutes — It was a flat 60 seconds. Closing or resetting the breaker restores the first window (#3159)
• The streaming worker no longer cancels pull jobs after 10 minutes — Pull mode is unlimited, as documented. JobTimeout stays as an opt-in cap (#3154)
• The outbox publishes behind a circuit breaker — Five consecutive publish failures that say NATS is unavailable pause publishing for 60 seconds. A row that fails for its own reasons does not count. Retry delays get 10% jitter, capped at 60 seconds
• Run.timeout_at is no longer set on new runs — Nothing enforced it. The field stays for compatibility (#3148)
• The ironflow-ops and ironflow-start agent skills link to the deployment and setup docs — The skills no longer copy them. The docs gain facts the skill copies held and corrections to errors in those copies (#3172)
• Two standalone books on the docs site — The platform overview deck and the “What you can build” book each open as their own page (#3198)