SDK Code Generation

Ironflow uses a manifest-based code generation pipeline to produce SDK clients in multiple languages from a single source of truth: the server’s registered HTTP routes.

The registry is hand-maintained

buildRouteRegistry in internal/server/routes.go is written by hand. It is not derived from the mux. Registering a handler with mux.HandleFunc (or mux.Handle) does not add it to the registry — you must also add an add, addT or addStreaming line.

Nothing about a missing entry looks broken. The route serves traffic normally; it is simply absent from /api/v1/routes, from sdk-manifest.json, from the generated Python client and from api/openapi.json — and invisible to check-schema-coverage.sh, which can only reason about routes the manifest already contains. The spec keeps implying it describes the whole API.

This drifted: 25 public routes had accumulated unregistered before TestMuxRoutesAreInRegistry (below) was added to catch it.

How It Works

internal/server/routes.go   addT(pattern, category, group, routeTypes{Req, Resp, Query})
    |  server boots; internal/server/schemagen reflects the named Go structs
    v
GET /api/v1/routes → sdk-manifest.json   (per-route schemas + a shared "components" map)
    |
    v
cmd/sdk-gen (Go template engine)
    |
    +---> -lang python    → sdk/python/ironflow/client.py + models.py
    |
    +---> -lang openapi   → api/openapi.json   (OpenAPI 3.1, committed)

Python is the only generated SDK client. OpenAPI is a second generated output — a machine-readable contract, not a client — read by every future Tier-2 emitter. The Go and JS SDKs are hand-written and never consumed this pipeline; their emitters were deleted from cmd/sdk-gen. This split is deliberate: Go and TypeScript are Tier-1 SDKs (hand-written worker runtime), while Python and all future languages are Tier-2 generated clients — see SDK Tiers for the full model.

Annotating a Route with Schemas

The manifest carries request, response, and query-parameter shapes for most public REST routes (ADR 0054). A route registration picks one of three forms:

• add(pattern, category, group) — no schema. The route works, but the generated Python method returns Any and takes no query kwargs. This form stays legal forever, for routes nobody has annotated yet.
• addT(pattern, category, group, routeTypes{Req, Resp, Query}) — the route carries reflected JSON Schema. Use this for any new non-streaming, non-internal route. check-schema-coverage.sh fails the build — naming the route — if one that should be annotated lands on add instead.
• addStreaming / addConnectService — unchanged, carry no schema. A streaming route cannot be called as request/response JSON, and ConnectRPC methods are out of scope for this pipeline (#1526).

ConnectRPC served-set rules

The ConnectRPC part of GET /api/v1/routes describes the exact running server instance. It is not a catalog of every service defined in protobuf.

• Every generated ConnectRPC handler is mounted through Server.mountConnectService, which records the full protobuf service descriptor and route category in the same operation.
• A manifest read starts with the cached REST entries and expands each recorded service descriptor into its methods. Protobuf remains authoritative for method names and stream cardinality.
• A conditional service is present only when its handler is mounted. For example, WorkerService requires a worker manager, while PubSubService appears after its late startup registration.
• Any unmounted definitions remain available in api/proto/ironflow/v1. The proto directory is the protocol catalog; no second catalog is derived from the served-route manifest.

The committed ConnectRPC count is the sum of the mounted service descriptors, not a number maintained in documentation. Inspect the current artifact rather than copying its population into prose:

jq '[.routes[] | select(.group == "connectrpc")] | length' sdk-manifest.json
jq '[.routes[] | select(.group == "connectrpc") | .path | split("/")[1]] | unique | length' sdk-manifest.json

TestConnectRouteRegistryMatchesMux probes the real mux against all loaded ironflow.v1 descriptors in both directions.

Choosing Req, Resp, and Query

routeTypes{Req, Resp, Query any} takes zero-value struct instances, for example routeTypes{Resp: RunDetail{}}. Leave a field nil (the zero value) when the route has no request body, no response body, or no query parameters.

• Resp must be the exact value the handler passes to httputil.WriteJSON — a slice literal for a list route ([]*store.Project{}), a bare struct for a single-object route (store.Project{}). If the handler writes []envResponse{...}, annotate with []envResponse{}, not envResponse{}.

• Query is a small struct with json tags, one field per r.URL.Query().Get(...) key the handler reads. Type each field to match how the handler parses it: strconv.Atoi(...) becomes int; a value compared against the literal "true" becomes bool; anything else stays string. Every field carries ,omitempty — a query parameter is always optional.

• RespMayBe204 (a bool, not a type) says the handler can answer 204 No Content even though Resp describes its 200 body. Set it by reading the handler for a w.WriteHeader(http.StatusNoContent) branch — it is never inferred from the HTTP method. It makes the generated Python return type nullable and adds a 204 response to the OpenAPI operation. A route with no Resp does not need it: those already generate Any.

  Today one route sets it: GET /api/v1/workers/{workerId}/jobs, the pull-worker job poll, which answers 204 on every idle poll (handlePollJobsCapacity in internal/server/worker_rest.go).

• RespStatus (an int) is the success status the handler writes, when it is not 200. Leave it zero for 200; the emitter uses it as the OpenAPI response key. Like RespMayBe204, read the handler for it — a POST is not automatically 201, and several POST routes here answer 200. It does not change generated Python types: a 201 body has the same shape as a 200 one.

  Today seventeen routes set RespStatus: 201 — the create endpoints for apikeys, kv buckets, orgs (/orgs and /platform/orgs), policies, roles, projects, secrets, environments, users, POST /api/v1/tenants/provision, POST /api/v1/policy-templates/{id}/install, and the five platform creates (/platform/bootstrap, /platform/users, /platform/roles, /platform/policies, /platform/tenants). Count them rather than trusting this list: grep -c 'RespStatus: 201' internal/server/routes.go.

• Headers (a []HeaderParam) lists the request headers the handler reads, beyond the ones every route takes. Each carries a Required flag and a description written for an API consumer. They become in: header parameters in the OpenAPI operation.

  Required is the reason this cannot be derived. The same header name means different things on different routes: X-Ironflow-Environment is a hard 400 on all six secrets routes (secretsEnvironmentID in internal/server/secrets_handler.go), but on the three outbox dead-letter routes it is one of two accepted ways to name an environment — the ?env= query parameter is the other, and the handler rejects only when both are absent. OpenAPI cannot express “one of these two”, so those declare the header optional and put the rule in its description.

  Eleven routes declare headers today: the six secrets routes and the three outbox routes above, Idempotency-Key on POST /api/v1/cluster/rotate-token (optional; the handler acts normally when it is absent), and If-Match/If-None-Match on the KV put route, which is the only conditional-write endpoint.

  The generated Python client does not expose header arguments. Adding one would change every method signature. A caller who needs to set a header — which means every caller of the secrets routes — drops to IronflowHTTP.request in sdk/python/ironflow/_http.py, whose docstring already names If-Match and X-Ironflow-Environment as the headers only reachable there.

Worked example

The apikeys group is the pilot; every other batch followed its shape. internal/server/apikey_handlers.go:

type apiKeyResponse struct {
    ID            string   `json:"id"`
    Name          string   `json:"name"`
    KeyPrefix     string   `json:"key_prefix"`
    RoleIDs       []string `json:"role_ids,omitempty"`
    EnvironmentID string   `json:"environment_id,omitempty"`
    CreatedAt     string   `json:"created_at"`
    ExpiresAt     *string  `json:"expires_at,omitempty"`
    LastUsedAt    *string  `json:"last_used_at,omitempty"`
}

type listAPIKeysQuery struct {
    Platform bool `json:"platform,omitempty"` // handler: r.URL.Query().Get("platform") == "true"
}

Registered in internal/server/routes.go:

r.addT("GET /api/v1/apikeys", "supported", "apikeys",
    routeTypes{Resp: []apiKeyResponse{}, Query: listAPIKeysQuery{}})
r.addT("DELETE /api/v1/apikeys/{id}", "supported", "apikeys",
    routeTypes{}) // 204, no body — annotated as "no shapes"

ExpiresAt *string reflects to a nullable string (pointer -> nullable, omitempty already makes it optional). RoleIDs []string reflects to an optional array, because omitempty is set. ID, Name, KeyPrefix, and CreatedAt carry no omitempty, so they land in the schema’s required list.

Reflection rules (internal/server/schemagen)

schemagen.FromType walks a Go value with reflect and builds this JSON Schema subset:

• Embedded structs flatten — both value and pointer embeds. type createAPIKeyResponse struct { apiKeyResponse; Key string } produces one flat object with apiKeyResponse’s fields plus key, not a nested object. (reflect.VisibleFields does the flattening; schemagen skips an embedded field only to avoid double-walking it.)
• omitempty or omitzero -> optional. A field carrying neither is required unless it is also a pointer. Both matter: omitempty cannot omit a struct, so a zero time.Time is dropped from the wire only by omitzero (circuitbreaker.BreakerState.LastFailure is the live case). Reading just one of the two publishes a field as required that the handler never sends.
• Pointer -> nullable. Type becomes ["string", "null"] (or whatever the base type is) instead of "string". A pointer to a named struct resolves to a $ref, and a $ref takes no sibling type, so it becomes {"anyOf": [{"$ref": ...}, {"type": "null"}]} — valid JSON Schema and valid OpenAPI 3.1, rendered in Python as Model | None. A pointer to a type that reflects to the empty schema (json.RawMessage, any) is left as {}, which already admits null.
• time.Time -> {"type": "string", "format": "date-time"}.
• json.RawMessage, any, interface{} all reflect to an empty schema ({}) — free-form, no shape asserted.
• Maps with string keys -> {"type": "object", "additionalProperties": <value schema>}. A non-string map key is a reflection error (panics at registration).
• Cycles -> $ref. A struct that references itself (directly or through another struct) resolves to a $ref pointing at its own component entry instead of recursing forever.
• Every named struct type becomes a shared component, keyed by its Go type name with the first rune upper-cased (apiKeyResponse -> ApiKeyResponse). An anonymous struct type stays inline.

Component-name collisions

mustSchema panics when two distinct Go types produce the same first-rune-uppercased component name. The failure mode is a panic stack trace out of manifest extraction (make sdk-manifest), not a failing unit test — read the panic message, it names both types and the route that triggered it.

The fix is to rename the Go type. Never weaken the panic — a silent merge of two different shapes under one name is exactly the drift this pipeline exists to prevent.

Real precedent: GET /api/v1/events/{id} is annotated with responses.EventResponse. A worker-job payload in internal/server/worker_rest.go needed its own EventResponse-shaped type; since responses.EventResponse already claimed that component name, the new type was renamed to JobEventResponse instead.

The coverage gate

scripts/check-schema-coverage.sh runs two checks against sdk-manifest.json, in this order.

1. The predicate — no generatable route may be unannotated. “Generatable” is the route set cmd/sdk-gen emits for: non-internal category, not streaming, not in the connectrpc group. The gate lists every such route that carries no annotation and fails if the list is non-empty:

jq '[.routes[] | select(.category != "internal" and (.streaming | not) and (.schemaAnnotated | not) and .group != "connectrpc")] | length' sdk-manifest.json
# must print 0

This is what makes “add() is only legal for internal routes” enforceable. A new public route landing on add fails here by method and path — the count check below cannot see it, because the annotated count does not move.

2. The checkpoint — every annotated-count change is reviewed. sdk-schema-coverage.txt holds one integer: the number of addT-annotated routes as of the last commit. The script compares it against the live count and fails in either direction:

• Fewer annotated routes than recorded — an annotated route was deleted outright (a route that merely regressed from addT to add is already caught by check 1). Restore an accidental deletion. If an accepted contract decision removes the route, update the recorded count in the same change and cite that decision.

• More annotated routes than recorded — you annotated something new. Bump the file to record the new checkpoint:

  make sdk-manifest
  jq '[.routes[] | select(.schemaAnnotated == true)] | length' sdk-manifest.json | tee sdk-schema-coverage.txt
  git add sdk-schema-coverage.txt

  (zsh/bash: same commands, no change needed.)

Both checks run in make ci (Step 9.4) and in the sdk-freshness CI job, after check-sdk-gen.sh.

The annotatedRoutes table

internal/server/routes_schema_test.go keeps a map[string]wantShapes (req/resp/query booleans) for every addT-registered route, and TestRoutesAreSchemaAnnotated cross-checks it against the live registry. It catches:

• A route that lost a shape it used to carry (e.g. Resp silently dropped).
• An addT route with no row in the table.
• A row naming a route that is not registered with addT (or not registered at all).
• A $ref — in a request, response, or query schema — that points at a component missing from the manifest.

What it does not catch: the table stores booleans, not types. A row that says {resp: true} is satisfied by any non-nil Resp, including the wrong struct. It cannot detect “annotated with the right flags, but the wrong Go type.” Reading the handler and comparing it against the routeTypes{} call is the only real check — the test is a drift detector, not a correctness proof.

The two parity tests

Because the registry is hand-maintained (see the caution at the top), the mux and the registry can disagree in both directions. One test guards each, both in internal/server/route_registry_parity_test.go:

• TestRouteRegistryIsServedByMux (registry → mux) — the registry advertises a route the mux serves nothing for. It 404s at runtime while the manifest and every generated client keep offering it. Probes srv.mux.Handler(req) and compares the resolved pattern, since the /api/v1/ catch-all also answers 404.
• TestMuxRoutesAreInRegistry (mux → registry) — the server really serves a route the registry never heard of, so it is missing from the manifest, the SDK and the spec. This is the direction that let 25 routes drift.

TestMuxRoutesAreInRegistry reads source text, not the mux: http.ServeMux cannot enumerate its registered patterns, which is why the registry → mux direction is a probe and this one is a scan. It globs the non-test .go files of internal/server and internal/platform — both put handlers on the mux cmd/ironflow/serve.go builds — and matches .Handle( as well as .HandleFunc(. Both spellings matter: internal/server uses HandleFunc, internal/platform uses Handle, and a survey that grepped only HandleFunc missed all 22 platform routes.

It accepts any registration kind — add, addT or addStreaming. The question is “is this route known?”, not “is it annotated”; annotation coverage is check-schema-coverage.sh’s job.

One exemption list, internalMuxRoutes — routes that legitimately have no registry entry: GET /ready, GET /metrics, GET /api/v1/routes, GET /api/v1/auth/validate and POST /api/v1/auth/desktop-session. Each carries a reason. Every key is an exact METHOD /path; there is deliberately no prefix form, because a POST /api/v1/auth/ entry would auto-exempt a future POST /api/v1/auth/token. Keep it small: every entry is a route users can call but no client can discover. test-* fixtures need no entry — they live in _test.go files, which the scan skips.

The /api/v1/platform/* routes used to sit behind a second, temporary exemption list. They are all registered now, and that list is gone.

A vacuity guard fails the test if the scan finds fewer than 100 registrations — otherwise a refactor that moved patterns out of string literals would leave the test passing while checking nothing.

Known limit: the scan verifies what platform.RegisterRoutes registers, not that serve.go still calls it.

Limitations

The pipeline has real gaps. Read these before depending on a generated artifact for something load-bearing:

• No header slot. routeTypes has no way to describe a header. Idempotency-Key is read on POST /api/v1/cluster/rotate-token (internal/server/cluster_rotate_handler.go), and a generated client cannot set it — a retried cluster_rotate_token() call re-rotates the token instead of hitting the replay cache.
• Only the 2xx shape is described. No non-2xx body is documented anywhere machine-readable: routeTypes has no error slot, and api/openapi.json emits no 4xx/5xx responses. This is a documentation gap, not a runtime one — the Python client handles errors correctly. _attempt in _http.py catches HTTPError before the cast is reached and routes it through _from_http_error, which parses code and message off the body and raises IronflowError. So a 422 never reaches typed field access; what a caller cannot do is discover, from the spec or the types, which code values a given route can produce.
• None on an empty body is opt-in, not derived. _http.py returns None for any empty response body, so in principle any generated method could return None. The generated types do not say so: only a route whose registration sets RespMayBe204 gets a nullable return. That is a deliberate trade — typing every generated return X | None would push an is None guard onto callers of endpoints that never send 204 — but it means the honesty of a return type depends on someone having read the handler. When you add a w.WriteHeader(http.StatusNoContent) branch to a route that carries a Resp, set the flag in the same diff.
• The success status is carried, but by hand. RespStatus is what the emitter documents, and it is set by a human who read the handler — nothing verifies that the number matches what the handler writes, and a new 201 route that omits it is silently documented as 200. An annotated route with no Resp is documented as 204, which is correct for every such route today, but that too is an assumption the emitter makes, not a fact it reads. Count them with jq '[.routes[]|select(.schemaAnnotated==true and (.responseSchema|not))]|length' sdk-manifest.json rather than trusting a number written here — a hand-maintained population count rots by construction.
• An optional request body is carried by hand too. requestBody.required follows routeTypes.ReqOptional, which a human sets after reading the handler; the tell is a decode error the handler swallows when it is EOF. It is deliberately not derived from the schema: eleven update-style routes have bodies with no required properties that are still mandatory, so inferring would wrongly mark all of them optional.
• “Mirror types” drift silently. Some handlers write a bare map[string]any rather than marshaling a named struct — for example HandleListConfigs in internal/server/config_handler.go writes map[string]any{"configs": configs}, and the route is annotated with a hand-written listConfigsResponse{Configs []configEntry} that mirrors it by field name only. Nothing ties the two together; if the map’s keys change, the annotation does not notice. A few dozen of these live in internal/server — a type whose doc comment says it mirrors a handler literal and which nothing but routes.go references. The population only grows, so it is described here rather than counted: a written-down total was wrong twice already. The comment is the binding — but the file.go:NNN anchor inside it is now checked. TestMirrorTypeAnchorsResolve in internal/server/mirror_anchor_test.go resolves every anchor written in an internal/server comment and fails when one no longer starts the node its own prose claims (a “map literal” anchor must land on the map[...] that opens the literal). When it fails, open the named file at the named line, find the literal the comment describes, and correct the number — nothing else. It still cannot tell one map literal in a handler from another in the same handler, and it says nothing about whether the mirrored keys still match.
• Go nil slices marshal to null. A list-route response is typed as a required, non-nullable array ({"type": "array", ...}, no "null" option), but encoding/json marshals a nil Go slice to null. This is reachable: GET /api/v1/projects writes whatever store.ListProjects returns, and the SQLite implementation declares var projects []*Project — so an org with no projects answers null, not [], against a schema that says array.
• The annotation is an unenforced claim. Naming a struct in routeTypes does not make the handler marshal that struct. Nothing at compile time or runtime checks that they match — see the ADR 0054 “Negative” consequence. routes_schema_test.go narrows this (previous section), it does not close it.

Streaming endpoints (/ws, KV watch, config watch) are excluded from generation — see RouteEntry.Streaming in internal/server/routes.go. A generated request/response method would call them as ordinary JSON HTTP and could not work.

Run make sdk-health for the current method count. It is not listed here because it changes whenever routes change.

The server exposes all registered routes via GET /api/v1/routes with metadata:

• method: HTTP verb (GET, POST, PUT, PATCH, DELETE)
• path: URL pattern with path parameters (e.g., /api/v1/events/{id})
• category: supported, server-only, enterprise-only, or internal
• group: logical grouping (events, runs, projections, kv, etc.)

Generating SDKs

Full pipeline (extract manifest + generate all SDKs)

make sdk-manifest   # Start server, extract routes, write sdk-manifest.json
make sdk-gen        # Generate the Python client AND the OpenAPI spec from the manifest

Individual targets

make sdk-gen-python    # Python → sdk/python/ironflow/client.py + models.py
make sdk-gen-openapi   # OpenAPI 3.1 spec → api/openapi.json (ADR 0054)

Python is the only generated SDK client. The Go and JS SDKs are hand-written; when a route changes, update sdk/go/ironflow/ and sdk/js/* by hand. api/openapi.json is generated, never hand-edited — see Annotating a Route with Schemas above for what feeds it.

Test stubs and health report

make sdk-gen-test-stubs  # Generate per-method test stubs for Go + TS
make sdk-health          # Coverage report → sdk-health.json

When to Regenerate

Run make sdk-manifest && make sdk-gen whenever:

• A new HTTP route is added to internal/server/server.go
• A route’s method, path, or middleware changes
• A new ConnectRPC service is registered

Drift detection is a local gate, not a PR gate

The sdk-freshness job in .github/workflows/ci.yml runs scripts/check-sdk-gen.sh, which compares live routes against the manifest and regen-diffs all three generated artifacts — client.py, models.py, and api/openapi.json. It does not run on pull requests: the pull_request trigger in ci.yml is commented out, so the job fires only on the nightly schedule, on workflow_dispatch, and on the workflow_call from release.yml. The paths-ignore list is commented out with it and is inert.

That leaves the local run as the gate before a push. make ci runs sdk-gen-verify (Step 9.4), which regen-diffs the same three files from the committed sdk-manifest.json. It needs no server, so it catches a hand-edited or stale generated file but not a stale manifest — proving the manifest matches live routes needs a running server, which is make sdk-gen-check in make ci-full. Run make ci-full before you push, and regenerate deliberately.

Checking for Drift

make sdk-gen-check   # Starts server, compares routes against manifest

This compares the live server’s route table against sdk-manifest.json. If they differ, it prints the diff and exits non-zero.

Comparing Versions

To see what changed between two manifest snapshots (e.g., between releases):

# Save current manifest before changes
cp sdk-manifest.json sdk-manifest-before.json

# Make changes, regenerate
make sdk-manifest

# See what changed
make sdk-changelog OLD=sdk-manifest-before.json NEW=sdk-manifest.json

Output:

SDK Changelog
═══════════════════════════════════════════
Old: sdk-manifest-before.json (180 routes)
New: sdk-manifest.json (183 routes)

Summary: +3 added, -0 removed

Added endpoints:
  + POST /api/v1/workflows (workflows, supported)
  + GET /api/v1/workflows (workflows, supported)
  + GET /api/v1/workflows/{id} (workflows, supported)

Adding a New Language

To add SDK generation for a new language (e.g., C#, Rust):

1. Add a new template in cmd/sdk-gen/main.go
2. Add a generate{Lang}() function following the generatePython() pattern
3. Add the language case to the switch in main()
4. Add a make sdk-gen-{lang} target to the Makefile
5. Test that the generated code compiles in the target language

The buildGroups() function handles filtering, grouping, and deduplication — it’s shared across all languages. You only need to write the template.

Route Categories

Category	Included in	Description
supported	Go, Node, Browser, Python	Public API, all SDKs should support
server-only	Go, Node, Python	Requires server-side execution (secrets, users, workers)
enterprise-only	Go, Node, Python	Tenant-administration endpoints (orgs, roles, tenants, policies). Legacy category name — these are not license-gated; Ironflow ships a single build with no Enterprise/Core split (ADR 0015).
internal	None	Infrastructure endpoints (health, debug, auth login)

The Browser SDK excludes both server-only and enterprise-only endpoints for security (untrusted runtime).

Architecture

The generation pipeline has no external dependencies:

• Manifest extraction: curl against the running server
• Code generation: Go text/template engine (cmd/sdk-gen)
• No vendor tools: no Fern, Speakeasy, or Stainless dependency