RFC-0004: Linked controller/engine boundary

Version: 0.1.0 | Status: draft | Phase: spec Owners: @Will Eaton

1. Summary

[RFC-0004:C-SCOPE] Scope (Informative)

This RFC governs how the controller obtains work from the engine: rendered Kubernetes objects for turn and run pods, compiled workflows and their parameter schemas, run explainability documents, and the launching of an engine process for an agent turn or a local run. It applies to every deployment shape (pod dispatch, the local executor, the registry) and to both directions of version drift between a controller build and the engine it is paired with.

It does not govern what runs inside a pod once launched (the wrapper script, the loop, the judge), nor the human API the controller serves. Those keep their own contracts (RFC-0001, RFC-0002).

The motivating failure: the controller began sending a flag for a scenario's git ref weeks before any engine build accepted it. Nothing failed until a scenario with a ref was adopted in production, and then every one of them failed at argument parsing, ledgered only as a turn that produced no result.

[RFC-0004:C-BOUNDARY-INVENTORY] The controller-engine boundary (Informative)

The controller reaches the engine in two ways today, and this RFC treats them differently.

Renders and compilations are pure: given a profile, a manifest or workflow source, and a set of options, they produce a document (pod objects, a pack ConfigMap, a workflow graph, a parameter schema, a flow document). Today the controller spawns the engine executable for each of these and parses its stdout: turn-pod render, run-pod render, workflow compile, parameter schema, and flow.

Process launches are not pure: an agent turn or a plan run holds a checkout, spends money, streams a session, and must be killable. Today the controller spawns the engine for these too: scope, grounded rank, a local playbook run, and artifact fetch.

Both paths share one contract surface, the engine's command line, and one enforcement point, the engine's argument parser at dispatch time.

Three version-bearing things take part, and the clauses below name them consistently:

• the contract definition: the single shared description of every document that crosses the boundary, compiled into both sides, carrying the contract version;
• the linked engine revision: the engine source revision a controller build compiles its renders from;
• a dispatch image: an engine image the running controller launches pods or processes from. A controller may be configured with more than one (loop images, sandbox images, per-pack images), and a dispatch image can differ from the linked engine revision because images are pinned in deployment configuration, not at build time.

This RFC replaces the render path with in-process renders checked at build time, and constrains launches to a typed request whose contract version is checked before any work happens.

2. Specification

[RFC-0004:C-LINKED-RENDER] Renders are linked, not spawned (Normative)

A render or compilation the controller needs (a turn pod, a run pod and its pack ConfigMap, a compiled workflow, a workflow's parameter schema, a flow document) MUST be produced in the controller's own process from the linked engine revision. The controller MUST NOT obtain any of these by spawning an engine executable.

A controller build that requests a render option the linked engine revision does not define MUST fail to build; it MUST NOT be possible for such a controller to reach dispatch.

For the same inputs, a render produced in-process by the controller and one produced by the engine's own command line MUST be byte-identical. The existing rendered fixtures are the acceptance test.

A render MUST NOT depend on anything beyond the inputs it is handed (the profile, the pack, the workflow source, its options). A render MUST NOT resolve image tags to digests, contact a cluster, or read the process environment unless the caller passes that capability in explicitly; the controller decides when a render may reach the network.

This clause changes how a render is delivered, not what it contains: the parameter schema stays as RFC-0002:C-PLAYBOOK-PARAMS defines it and the compiled workflow stays as RFC-0001:C-WORKFLOW defines it.

Rationale: a render is a function of its inputs. A process boundary and an argument parser between the caller and that function add a contract nobody checks and remove build-time checking, which was the only tool able to check it.

[RFC-0004:C-TYPED-INVOCATION] Typed requests across a process boundary (Normative)

Where the engine MUST run as a separate process (an agent turn, a local plan run, work inside a pod), the request MUST be a single typed document described by the contract definition, carrying the contract version it was written under. The engine MUST reject a request whose document has an unknown field, or whose contract version is not equal to the engine's own, before it clones, spends, or writes anything, with an error that names the field or both versions.

The command line of such an invocation MUST carry only the request kind and the location of the request document; it MUST NOT carry request fields as flags. An environment variable MAY carry the document location where a file cannot (a pod wrapper), and the same rejection rules apply.

The engine's reply for an invocation MUST be a typed document under the same contract version, so that the controller parses a result the way it built the request, and a reply the controller cannot decode is a boundary failure per RFC-0004:C-BOUNDARY-FAILURE, not a missing result.

Rationale: a process boundary is unavoidable for work that must be isolated or killed, but the boundary can still be a document with a schema and a version instead of a flag list whose only validator is the parser on the far side.

[RFC-0004:C-CONTRACT-VERSION] One contract version, checked before dispatch (Normative)

The contract definition MUST carry exactly one contract version. Two contract versions match if and only if they are equal; there is no compatibility range. A controller build MUST record the contract version of the contract definition it was compiled against, and an engine build MUST record the one it was compiled against; both MUST be readable from the running executable and, for an image, from the image's labels.

A controller build whose linked engine revision was compiled against a different contract version than the controller MUST fail its build, so the mismatch is found in CI and not in production.

Because a dispatch image is pinned in deployment configuration and may differ from the linked engine revision, the running controller MUST, at startup and whenever a dispatch image pin changes, read the contract version from every dispatch image it is configured with and compare each against its own. It MUST expose the result per dispatch image (the image reference, both versions, and whether they match) on the same read endpoint that reports its effective configuration, so an operator can see a mismatch without reading logs.

A mismatch MUST NOT stop the controller from serving. The controller MUST refuse to launch a pod or a process from a mismatched dispatch image, and each refusal MUST be ledgered per RFC-0004:C-BOUNDARY-FAILURE. Pods already running when a pin changes are out of scope; they carry the request they were launched with.

Rationale: build-time checking removes drift for renders and for the linked revision; the runtime comparison covers the one place a link cannot reach, an image chosen by configuration, and turns a runtime surprise into a visible, explainable refusal.

[RFC-0004:C-BOUNDARY-FAILURE] Boundary failures are ledgered by name (Normative)

A failure at the controller-engine boundary MUST be recorded on the issue or run ledger with the request kind, the dispatch image or linked engine revision involved, the controller and engine contract versions, and the engine's own error text. It MUST NOT be reduced to a generic no-result outcome; the ledger entry alone MUST be enough to tell what was sent and why it was refused.

Boundary failures are of two kinds, and the ledger entry MUST say which:

• contract rejections are deterministic: a rejected contract version, an unknown field, a render option the engine does not define, a reply the controller cannot decode because its shape is wrong. The same request against the same engine cannot succeed, so a contract rejection MUST NOT be retried automatically.
• transport failures are not: a process killed before it replied, a truncated reply, an I/O error reading or writing the request document. These MAY be retried under the retry policy that already governs the launch in question, and the retry MUST be ledgered as a retry, not as a fresh attempt.

Rationale: the motivating failure was visible only to someone who read the raw ledger event; the summary said nothing, and the item stayed in its initial state as if never attempted. Retrying a deterministic rejection would have hidden the same fault behind a moving timestamp.

[RFC-0004:C-RUN-EVIDENCE] Pod evidence travels on a pod token (Normative)

A run's evidence is its session log, its captured files, its git memory, and its captured telemetry. A run pod MUST deliver its evidence to the controller's run evidence routes under /api/evidence/runs/{run}/{kind}, keyed by the run, never by the pod that produced it. The controller MUST store an upload as the run's own artifact; nothing at completion re-owns pod-keyed uploads to a run.

At dispatch the controller MUST mint one opaque pod token for the run and deliver it to the pod through the run's Secret, never in the pod spec, the launch row, or argv. The controller MUST store only a digest of its secret. A run's token grants exactly two things: writing the evidence of the run it was minted for, and reading the evidence of the run that run's launch reruns, when it reruns one. Every other route MUST refuse it. A run's token MUST expire no later than the run's ceiling allows the pod to live, and MUST be revoked when the run completes or fails.

A scope turn's evidence is its pack and its transcript. At dispatch the controller MUST mint a pod token for the turn's pod, delivered the same way. A turn's token grants exactly one thing: writing that pod's pack and transcript under /api/evidence/turns/{pod}/{kind}. Every route a pod reaches the controller on for evidence MUST sit under /api/evidence, so a deployment can expose pod evidence to remote clusters without exposing the rest of the API. The controller MUST store them keyed by the pod and MUST check each against the digest the turn's termination message reports before using it. A turn's token MUST be revoked when the turn is collected or fails to launch. No pod MUST be given a cluster-issued credential for delivering evidence.

Reading a run's session log, captured files, or git memory MUST also be open to any caller the human API admits, and a run the local executor produced MUST be served from its run directory when the controller stored no copy.

A rerun launched in a pod MUST name its parent as the parent's evidence URL on the controller, and the engine MUST fetch the parent's evidence over that URL into a directory laid out like a run's state directory before it reads the parent, with the same refusals as a parent read from disk.

3. Compatibility

[RFC-0004:C-MIGRATION] Migration and flag ordering (Normative)

Until a render is delivered in-process, adding or changing an option on an engine command the controller invokes MUST land in the engine first, and the controller change MUST NOT be released before a dispatch image carrying that engine is deployed. A controller change that would send an option the linked engine revision does not accept MUST fail the controller's build.

Moving a render in-process MUST NOT change the rendered document for the same inputs; the existing rendered fixtures are the acceptance test.

Existing process-launch kinds MAY keep their current command-line form during migration, but a new launch kind MUST be introduced only as a typed request per RFC-0004:C-TYPED-INVOCATION.

The contract version starts at the version the contract definition carries when this RFC is finalized. A controller or engine that cannot report a contract version MUST be treated as mismatched per RFC-0004:C-CONTRACT-VERSION.

Changelog

v0.1.0 (2026-08-25)

Initial draft