coilyco-flight-deck/ward

Fork 0

[epic] Config-driven ward: one precompiled binary, personal config resolved live from a git ref (supersedes the build-variant matrix) #650

New issue

Open

opened 2026-07-07 23:40:21 +00:00 by coilyco-ops · 5 comments

coilyco-ops commented

2026-07-07 23:40:21 +00:00

Member

Goal

Replace the emerging "two build variants" release matrix (neutral ward vs coilyco-embedded ward) with a single precompiled binary whose personal configuration is pure config, resolved live from a git ref. This supersedes the build-time-overlay direction of ward#503 step 3 (#644) and the build-from-source tap overlay (homebrew-tap#3). Decided with Kai over the 2026-07-07 director session.

Settled decisions (do not re-litigate - design within these)

One precompiled binary per platform. Byte-identical across GitHub + Forgejo releases, plus scoop and a single brew formula. No build-from-source at install, no per-variant builds, no matrix. Channels download + verify a prebuilt artifact.
Exactly three things, no more:
- (1) ward source code - the binary behavior, carrying a baked-in neutral default config so a bare, unconfigured ward is self-contained and safe.
- (2) a KDL config source - aos for Kai (coilyco-flight-deck/agentic-os, the .ward/ bundle), an external user's own repo for them. Holds guardfiles + fleet and all topology that is currently WARD_* env vars (forge base, SSM paths, ollama/signoz endpoints, clone base, target, etc.).
- (3) one self-describing git-ref env var pointing at (2), e.g. forgejo.coilysiren.me/coilyco-flight-deck/agentic-os@main//.ward - enough to fetch on its own.
Resolution is live, via a plain git ref. ward parses the ref as a plain git ref and does not care whether it is a branch, tag, or sha - pinning-vs-tracking-main is the operator's choice, not ward's policy. Reuse ward's existing periodic git-sync machinery (identify the exact one) rather than a new fetch path. Local-file mode (--bundle / file://) is a secondary escape hatch added later.
Precedence is trivial: var set -> use (2); var unset -> the baked-in neutral default. That is the whole precedence.

Why this is a small lift (grounding, verified 2026-07-07)

ward already compiles KDL at launch. cmd/ward/ops.go (specverb.Build) and cmd/ward/wardkdl_exec.go (execverb.Build) read guardfile + spec-lock bytes from an embed.FS and build the verb surface live. The compiler is already in the shipped binary. The only hard-wired thing is the source: opsAssets / execAssets are embed.FS, but the build paths take fs.ReadFile / fs.ReadDir, which work over any fs.FS. Swapping embed.FS for os.DirFS(<synced local copy of (2)>) is the core change.
ward already resolves config location with loud-fail: --config > $WARD_CONFIG > walk-up, loud-fail on a set-but-missing override (docs/config-discovery.md). Extend that same pattern to the KDL bundle.
The ward-specs bundle already lives in aos as a pinned release asset (aos#315) and aos#332 is homing ward's complete current bundle there now.

Hard non-goal (parked - do NOT open this)

What the neutral default (1) actually contains. This drifts immediately into "seed" / template-repo territory, which Kai wants kept out of this design. Treat the neutral default as an opaque placeholder: the architecture needs only "a safe minimal default exists," never its contents. Design around it. Do not propose seed/template-repo structure.

What the advisor must resolve

Fetch transport under the git ref: archive-tarball download (no git binary, lighter, fits a self-contained binary, forge archive API) vs git shell-out (any ref, private repos via existing creds). Recommend one, with reasoning.
Refresh cadence + the exact existing sync mechanism to reuse: which trigger drives the live sync (per-session / TTL / on-version-change), how it caches so a hot-path gate does not fetch per-invocation and does not brick offline (fall back to cache), and which existing ward periodic-sync code is the reuse target.
The var grammar: the precise self-describing ref format (host/owner/repo@ref//subpath) and how ward parses it.
Topology migration: moving the ~14 WARD_* topology values into the KDL bundle without breaking the container/infra that sets them today (staged, backward-compatible).
Integrity + fail-loud (ward's job even in mechanism-not-policy mode): ward does not police which ref, but must fail loud if the named source will not resolve, consistent with config-discovery today. Define that behavior.
In-flight reconciliation + new issue breakdown: aos#332 stays the linchpin (the KDL source). #644 / homebrew-tap#3 / ward#645 fold into "add the config-resolve path + keep the baked-in default"; the build-from-source tap overlay is dropped and channels become download-and-verify. Produce the concrete replacement issue list.

Deliverable

A written design proposal posted to this issue: the resolution of the six items above, the config-resolve architecture (source selection, live sync + cache, launch-time compile from the resolved fs.FS), the topology-fold plan, and the superseding issue breakdown. Ground every claim in the actual tree (read ward's cmd/ward/*.go config/compile/sync paths and aos's .ward/ bundle).

aos#332 (KDL source, in flight), #644 (build overlay, superseded), homebrew-tap#3 (tap overlay, dropped), ward#645 (neutralize, folds in), #503 / #453 (the cutover), #441 / #395 (why coilyco values were externalized), #539 (ward-kdl -> ward migration). Filed from the read-only director surface (she/her).

## Goal Replace the emerging "two build variants" release matrix (neutral ward vs coilyco-embedded ward) with a **single precompiled binary whose personal configuration is pure config, resolved live from a git ref**. This supersedes the build-time-overlay direction of ward#503 step 3 (#644) and the build-from-source tap overlay (homebrew-tap#3). Decided with Kai over the 2026-07-07 director session. ## Settled decisions (do not re-litigate - design within these) 1. **One precompiled binary per platform.** Byte-identical across GitHub + Forgejo releases, plus scoop and a single brew formula. No build-from-source at install, no per-variant builds, no matrix. Channels download + verify a prebuilt artifact. 2. **Exactly three things, no more:** - **(1) ward source code** - the binary behavior, carrying a baked-in **neutral default** config so a bare, unconfigured ward is self-contained and safe. - **(2) a KDL config source** - aos for Kai (`coilyco-flight-deck/agentic-os`, the `.ward/` bundle), an external user's own repo for them. Holds guardfiles + fleet **and all topology that is currently `WARD_*` env vars** (forge base, SSM paths, ollama/signoz endpoints, clone base, target, etc.). - **(3) one self-describing git-ref env var** pointing at (2), e.g. `forgejo.coilysiren.me/coilyco-flight-deck/agentic-os@main//.ward` - enough to fetch on its own. 3. **Resolution is live, via a plain git ref.** ward parses the ref as a plain git ref and does not care whether it is a branch, tag, or sha - pinning-vs-tracking-main is the operator's choice, not ward's policy. Reuse **ward's existing periodic git-sync machinery** (identify the exact one) rather than a new fetch path. Local-file mode (`--bundle` / `file://`) is a secondary escape hatch added later. 4. **Precedence is trivial:** var set -> use (2); var unset -> the baked-in neutral default. That is the whole precedence. ## Why this is a small lift (grounding, verified 2026-07-07) - **ward already compiles KDL at launch.** `cmd/ward/ops.go` (`specverb.Build`) and `cmd/ward/wardkdl_exec.go` (`execverb.Build`) read guardfile + spec-lock bytes from an `embed.FS` and build the verb surface live. The compiler is already in the shipped binary. The only hard-wired thing is the **source**: `opsAssets` / `execAssets` are `embed.FS`, but the build paths take `fs.ReadFile` / `fs.ReadDir`, which work over any `fs.FS`. Swapping `embed.FS` for `os.DirFS(<synced local copy of (2)>)` is the core change. - **ward already resolves config location with loud-fail:** `--config` > `$WARD_CONFIG` > walk-up, loud-fail on a set-but-missing override (`docs/config-discovery.md`). Extend that same pattern to the KDL bundle. - The ward-specs bundle already lives in aos as a pinned release asset (aos#315) and aos#332 is homing ward's complete current bundle there now. ## Hard non-goal (parked - do NOT open this) **What the neutral default (1) actually contains.** This drifts immediately into "seed" / template-repo territory, which Kai wants kept out of this design. Treat the neutral default as an **opaque placeholder**: the architecture needs only "a safe minimal default exists," never its contents. Design around it. Do not propose seed/template-repo structure. ## What the advisor must resolve 1. **Fetch transport under the git ref:** archive-tarball download (no `git` binary, lighter, fits a self-contained binary, forge archive API) vs `git` shell-out (any ref, private repos via existing creds). Recommend one, with reasoning. 2. **Refresh cadence + the exact existing sync mechanism to reuse:** which trigger drives the live sync (per-session / TTL / on-version-change), how it caches so a hot-path gate does not fetch per-invocation and does not brick offline (fall back to cache), and which existing ward periodic-sync code is the reuse target. 3. **The var grammar:** the precise self-describing ref format (`host/owner/repo@ref//subpath`) and how ward parses it. 4. **Topology migration:** moving the ~14 `WARD_*` topology values into the KDL bundle without breaking the container/infra that sets them today (staged, backward-compatible). 5. **Integrity + fail-loud (ward's job even in mechanism-not-policy mode):** ward does not police which ref, but must fail loud if the named source will not resolve, consistent with config-discovery today. Define that behavior. 6. **In-flight reconciliation + new issue breakdown:** aos#332 stays the linchpin (the KDL source). #644 / homebrew-tap#3 / ward#645 fold into "add the config-resolve path + keep the baked-in default"; the build-from-source tap overlay is dropped and channels become download-and-verify. Produce the concrete replacement issue list. ## Deliverable A written design proposal posted to this issue: the resolution of the six items above, the config-resolve architecture (source selection, live sync + cache, launch-time compile from the resolved `fs.FS`), the topology-fold plan, and the superseding issue breakdown. Ground every claim in the actual tree (read ward's `cmd/ward/*.go` config/compile/sync paths and aos's `.ward/` bundle). ## Related aos#332 (KDL source, in flight), #644 (build overlay, superseded), homebrew-tap#3 (tap overlay, dropped), ward#645 (neutralize, folds in), #503 / #453 (the cutover), #441 / #395 (why coilyco values were externalized), #539 (ward-kdl -> ward migration). Filed from the read-only director surface (she/her).

coilyco-ops added the

consult

labels

2026-07-07 23:40:21 +00:00

coilyco-ops commented

2026-07-07 23:44:23 +00:00

Author

Member

🔎 ward agent advisor

ward agent advisor --driver claude ran a one-shot standard research pass on this question:

Produce the full design proposal this epic asks for. Work strictly within the settled decisions and respect the hard non-goal (do not open the seed/neutral-default-contents question). Resolve all six open items - fetch transport (archive-download vs git shell-out, recommend one), refresh cadence + the exact existing ward periodic-sync mechanism to reuse, the self-describing git-ref var grammar, the WARD_* topology-into-KDL migration plan, integrity/fail-loud behavior, and the superseding issue breakdown (aos#332 stays; #644 / homebrew-tap#3 / ward#645 fold in). Ground every claim in the real tree: read cmd/ward/ops.go, cmd/ward/wardkdl_exec.go, cmd/ward/fleet.go, docs/config-discovery.md, the existing periodic-sync code, and aos's .ward/ bundle. Post the design as a comment on the epic.

Config-driven ward: design proposal (ward#650)

Grounded in the current tree: cmd/ward/ops.go, cmd/ward/wardkdl_exec.go, cmd/ward/fleet.go, cmd/ward/container_bootstrap.go, cmd/ward/container_compute.go, docs/config-discovery.md, docs/container-env.md, docs/container-substrate.md. All six open items resolved inside the settled decisions. The neutral default stays an opaque placeholder throughout - nothing here touches its contents.

The core change, confirmed against the tree

The epic's grounding holds. The KDL compiler is already in the shipped binary and already reads through an fs.FS:

buildForgejoOps (ops.go:62) does opsAssets.ReadFile(...) -> guardfile.Parse -> specverb.Build. opsAssets is //go:embed (ops.go:30).
mountWardKdlExec (wardkdl_exec.go:29) does fs.ReadDir(execAssets, execAssetsDir), then execAssets.ReadFile -> execverb.Parse -> execverb.Build per file. execAssets is //go:embed (wardkdl_exec.go:22).
fleet.go parses fleetGeneratedKDL (//go:embed) via fleetconfig.Parse.

Every read is fs.ReadFile / fs.ReadDir against an embed.FS. Both satisfy fs.FS, so the only hard-wired thing is the source handle. The change is to make the three build sites take an fs.FS chosen at launch:

var unset -> the baked embed.FS (today's exact behavior, the neutral default).
var set -> os.DirFS(<synced local checkout of the bundle subpath>).

That is the whole behavioral pivot. specverb.Build / execverb.Build / fleetconfig.Parse are unchanged.

1. Fetch transport: git shell-out (recommended)

Recommend git shell-out over a bare mirror, not the forge archive-tarball API. Reasoning grounded in the tree:

ward already shells git for exactly this. warmSubstrateRepo (container_bootstrap.go:900) does git clone --mirror then git -C <mirror> remote update --prune then a working git clone. seedExternalContextMirrors (agent_context_seed.go) does git clone --mirror into the gitcache volume. git_clone.go / git_auth.go already own authenticated clone. Adding an archive-download path is net-new surface competing with machinery that already exists and is drift-tested.
The settled grammar is "a plain git ref, branch/tag/sha, ward does not care which" (decision 3). git fetch/git clone --mirror resolves all three uniformly. Forge archive APIs are inconsistent here: Forgejo/Gitea archive-by-sha support and pathing differ from GitHub's, and pinning to a sha (the operator's integrity story) is a first-class git operation but a second-class archive one.
Private repos + existing creds. The bundle for Kai is coilyco-flight-deck/agentic-os behind the same Forgejo the binary already authenticates to for releases (WARD_FORGEJO_BASE, forge.go:45). git shell-out reuses git_auth.go. An archive API path would need its own token plumbing.
The "no git binary, lighter" argument does not apply to this consumer. ward's whole agent/container half hard-depends on a local git, and the bundle-resolving consumer is Kai's fleet, not a bare external adopter. External adopters use a local .ward/ward.yaml plus the baked neutral default and never resolve a remote bundle at all, so they never need git for this path.

Escape hatch (decision 3, secondary): a file:// / --bundle <dir> form skips git entirely and hands os.DirFS(dir) straight to the build sites. Land it after the git path.

2. Refresh cadence + the exact reuse target

Reuse target, named precisely: the substrate warmer in cmd/ward/container_bootstrap.go - warmSubstrate (:940), warmSubstrateRepo (:900), and substrateMirrorStale (:889) - driven by WARD_SUBSTRATE_TTL (default 600s, container_compute.go:636, docs/container-substrate.md). Its properties are exactly what a hot-path config gate needs:

TTL-gated fetch keyed on FETCH_HEAD mtime. substrateMirrorStale stats <mirror>/FETCH_HEAD and calls it stale only once now - mtime >= ttl. A missing FETCH_HEAD (fresh clone/hydrate) is fresh. So a burst of ward exec build calls inside the TTL window does zero network I/O.
flock serialization. warmSubstrateRepo wraps the ensure-and-freshen in r.withFlock(lock, ...) (cli-guard pkg/flock, imported container_bootstrap.go:19), so concurrent ward processes racing the same mirror do not corrupt it.
Cache-fallback, never brick offline. On remote update failure it logs substrate: refresh failed ... (using cached state) and proceeds on the cached mirror. That is the offline story verbatim.

Proposed factoring: extract the mirror-ensure-and-freshen core of warmSubstrateRepo into a shared syncGitRef(ctx, spec, ttl) that returns a local working-checkout path, and have both the substrate warmer and the new config-bundle resolver call it. Same TTL gate, same flock, same cache-fallback, one code path.

Trigger + cache location: resolve-on-launch, TTL-gated. On host, cache the bundle mirror + checkout under ~/.cache/ward/config-bundle/<hash-of-ref>/ (sibling to config.CacheDir, referenced in agent_log_drain.go:146); in a container reuse the ward-gitcache volume the substrate warmer already uses. TTL default 600s to match WARD_SUBSTRATE_TTL, overridable by a new WARD_CONFIG_TTL. First resolve populates the cache, then the FETCH_HEAD gate keeps hot-path invocations network-free. Because the checkout is a real directory, os.DirFS(checkout/<subpath>) feeds the build sites with zero further copying.

3. The var grammar

Proposed var name WARD_CONFIG_REF (parallels WARD_CONFIG, which already selects the allowlist per docs/config-discovery.md). Grammar, go-getter / Terraform-module style so the // subpath split is unambiguous:

<host>/<owner>/<repo>[@<ref>]//<subpath>

Example from the epic: forgejo.coilysiren.me/coilyco-flight-deck/agentic-os@main//.ward

Parse order (self-describing, no forge assumptions):

Split once on the first // -> left = repo-spec, right = subpath (.ward). // cannot appear in a host/owner/repo, so the split is safe.
Split left once on @ -> repospec (host/owner/repo) + ref (main). No @ -> ref defaults to HEAD / the remote default branch.
Clone URL = https://<repospec>.git. ref is passed to git untouched - branch, tag, or sha, ward does not classify it (decision 3).
The resolved fs.FS is os.DirFS(filepath.Join(checkout, subpath)).

Parsing is a ~20-line pure function with a table-driven test, mirroring the shape of parseConfigOverrides (container_compute.go:576).

4. Topology fold: the ~14 WARD_* values into the bundle

The fold is a small, staged, backward-compatible step because the fleet already treats these as env-over-baked-default DATA, not identity. container_compute.go:100 already says so: "Tailnet + tower topology (ward#395): infra DATA, not baked identity. Each value takes a WARD_* env override, the old literal kept as the fail-safe default." And ward#616 already resolves --config agent.<name>.<key> into WARD_* env keys (configEnvKeys, container_compute.go:555). The bundle just becomes a third, middle tier in an ordering that already exists.

New precedence (a strict superset of today's env > baked default):

WARD_* env > KDL bundle topology block > baked neutral default

Nothing that sets an env var today changes behavior - env still wins. The bundle only fills the slot the baked coilyco literal fills now.

The values to fold (from docs/container-env.md + the code), each with its current hard-wired home:

Forge / clone base - forgejoBaseURL = "https://forgejo.coilysiren.me" (forgejo_ops.go:23), WARD_FORGEJO_BASE, WARD_FORGE / WARD_CLONE_BASE.
Forgejo token SSM path - value ssm "/forgejo/coilyco-ops/api-token" (opsassets/forgejo.guardfile.generated.kdl:18), resolved by forgejoTokenResolver / ssmValueResolver (ops.go:112,:381).
Tailnet + tower - WARD_TAILNET_NETWORK (ward-tailnet), WARD_TAILNET_PROXY (mac-proxy:1055), WARD_TOWER_HOST (kai-tower-3026), WARD_TOWER_OLLAMA_PORT (11434) - the default* consts at container_compute.go:104-127.
ollama / observe endpoints - WARD_OLLAMA_URL, WARD_TOWER_OLLAMA (towerOllamaURL, :151).
Substrate topology - WARD_SUBSTRATE_SEED / _DEST / _MANIFEST / _TTL (container_compute.go:633-636).
Clone base + target are per-run and stay env-set by the container driver.

Staging (no flag-day):

Stage 1 (pure addition): add a topology section to aos's .ward/ bundle (aos#332), add the bundle-resolve path, and insert the bundle tier between env and baked default in each resolver (envOr -> envOrBundleOr). Baked defaults and every WARD_* env override stay intact. A run with WARD_CONFIG_REF unset is byte-for-byte today.
Stage 2: point the fleet's container_compute.go driver and infra at WARD_CONFIG_REF. The container path that sets these WARD_* vars for children (wardEnv, container_compute.go:605) resolves them from the bundle first, then still passes them through as env, so the in-container entrypoint.sh and container_bootstrap.go are unchanged - the env contract in docs/container-env.md is preserved as the transport, the bundle just becomes its source. The now-redundant literal env sets become harmless.
Stage 3 (later, optional, out of this epic's critical path): delete the coilyco-specific baked literals, leaving only the opaque neutral default. Gated on the bundle path being proven in the fleet.

5. Integrity + fail-loud (mechanism, not policy)

ward does not police which ref (decision 3), but it fails loud whenever the named source cannot resolve, consistent with docs/config-discovery.md's set-but-missing override rule. The behavior matrix:

WARD_CONFIG_REF unset -> baked neutral default embed.FS. Never an error. This is decision 4's trivial precedence.
Set but malformed grammar (no //, empty subpath, empty repospec) -> fail loud naming the exact grammar violation, before any network. Same spirit as parseConfigOverrides' loud unknown-key error.
Set, well-formed, but clone/fetch fails AND no cache exists -> fail loud naming the unresolvable ref and URL. This is the direct analogue of config-discovery: "an explicit override is never silently ignored or downgraded to the walk-up." ward must not silently fall back to the neutral default when the operator explicitly named a source - that would mask a broken pin.
Set, fetch fails, but a cached checkout exists -> warn and use the cache (the substrate warmer's using cached state). Offline resilience, not a hard fail, because the operator's intent (use the bundle) is still honored.
Resolved, but subpath missing or KDL does not parse -> fail loud. This is already how the current embed path degrades: opsCommand (ops.go:42) catches a build error and mounts a leaf that surfaces guardfile runtime failed to mount: %w on invocation. Reuse that exact degrade-to-error-leaf pattern so a bad bundle cannot silently drop a whole verb surface.
Integrity pin = the ref itself. A sha ref is the integrity guarantee; ward adds no separate checksum policy (that would be policy, not mechanism). ward records the resolved sha in the audit row (ward already stamps repo_root per AGENTS.md Safety), so every guarded invocation is attributable to the exact bundle commit it compiled from.

6. In-flight reconciliation + superseding issue breakdown

aos#332 stays the linchpin, unchanged - it homes ward's complete current KDL bundle in aos's .ward/. This design's Stage 1 adds a topology section to that same bundle. It is the upstream dependency for everything below: the ward-side resolver has nothing to resolve until the bundle exists.
#644 (build-time overlay) + homebrew-tap#3 (build-from-source tap overlay) - superseded/dropped. The build-variant matrix is replaced by decision 1's single prebuilt byte-identical artifact. homebrew-tap#3's build-from-source overlay is dropped: the single brew formula + scoop become download-and-verify of the prebuilt binary (this dovetails with the existing release plumbing in AGENTS.md Release, where bump-tap-formula already rewrites the formula url+sha256 to a released artifact - keep that, drop any from-source path).
ward#645 (neutralize) folds in as the "keep the baked-in neutral default" half of the config-resolve issue below.

Concrete replacement issue list (all land in coilyco-flight-deck/ward, filed off this epic; ordered by dependency):

Config-resolve path + keep the baked neutral default (folds #644, homebrew-tap#3's ward-side, ward#645). Make the three build sites (ops.go buildForgejoOps + graftForgejoAdminExec, wardkdl_exec.go mountWardKdlExec, fleet.go loadFleetConfig) take an fs.FS selected at launch: embed.FS when WARD_CONFIG_REF is unset (neutral default), os.DirFS(resolved subpath) when set. Reuse the opsCommand degrade-to-error-leaf on parse failure. Depends on aos#332 for a real bundle to point at (tests use file:// fixtures).
WARD_CONFIG_REF grammar + the shared syncGitRef TTL-cache resolver. The host/owner/repo@ref//subpath parser (pure, table-tested) plus the git shell-out resolver factored out of warmSubstrateRepo (TTL gate via substrateMirrorStale, withFlock, cache-fallback). Delivers the fs.FS that issue 1 consumes. Depends on issue 1's fs.FS seam being in place.
Topology fold, staged. Add the topology section to the bundle and insert the bundle tier into each envOr resolver (env > bundle > baked default), Stage 1 pure-addition first. Depends on issues 1+2 (needs the resolved bundle) and on aos#332 carrying the section.
Fail-loud + audit integrity. The behavior matrix in section 5, including recording the resolved bundle sha in the audit row. Small, depends on issue 2's resolver.
Channels become download-and-verify (release plumbing). Confirm the single brew formula + scoop + byte-identical GitHub/Forgejo artifact (decision 1); retire any build-from-source tap path. Cross-refs homebrew-tap; can proceed in parallel with 1-4 since it is release infra, not the resolve path.

Escape hatch, deferred: --bundle / file:// local-file mode (decision 3, secondary) as a follow-up once the git path lands - it is a trivial branch in the issue-2 resolver (os.DirFS directly, no sync).

Summary of the recommendation

One prebuilt binary, fs.FS chosen at launch behind the existing config-discovery precedence, resolved live via git shell-out over a bare mirror through the substrate warmer's TTL+flock+cache-fallback machinery (warmSubstrateRepo/substrateMirrorStale), WARD_CONFIG_REF=host/owner/repo@ref//subpath, topology folded in as an env > bundle > baked-default middle tier that supersets today's ward#395 ordering, and fail-loud on an unresolvable-and-uncached named source per config-discovery. Every claim ties to a specific site in the current tree, and no part of it opens the neutral-default-contents question.

Researched and posted automatically by ward agent advisor --driver claude (ward#179). This is one-shot research, not a carried change - verify before acting on it.

— Claude (she/her), via ward agent

### 🔎 ward agent advisor `ward agent advisor --driver claude` ran a one-shot **standard** research pass on this question: > Produce the full design proposal this epic asks for. Work strictly within the settled decisions and respect the hard non-goal (do not open the seed/neutral-default-contents question). Resolve all six open items - fetch transport (archive-download vs git shell-out, recommend one), refresh cadence + the exact existing ward periodic-sync mechanism to reuse, the self-describing git-ref var grammar, the WARD_* topology-into-KDL migration plan, integrity/fail-loud behavior, and the superseding issue breakdown (aos#332 stays; #644 / homebrew-tap#3 / ward#645 fold in). Ground every claim in the real tree: read cmd/ward/ops.go, cmd/ward/wardkdl_exec.go, cmd/ward/fleet.go, docs/config-discovery.md, the existing periodic-sync code, and aos's .ward/ bundle. Post the design as a comment on the epic. --- # Config-driven ward: design proposal (ward#650) Grounded in the current tree: `cmd/ward/ops.go`, `cmd/ward/wardkdl_exec.go`, `cmd/ward/fleet.go`, `cmd/ward/container_bootstrap.go`, `cmd/ward/container_compute.go`, `docs/config-discovery.md`, `docs/container-env.md`, `docs/container-substrate.md`. All six open items resolved inside the settled decisions. The neutral default stays an opaque placeholder throughout - nothing here touches its contents. ## The core change, confirmed against the tree The epic's grounding holds. The KDL compiler is already in the shipped binary and already reads through an `fs.FS`: - `buildForgejoOps` (`ops.go:62`) does `opsAssets.ReadFile(...)` -> `guardfile.Parse` -> `specverb.Build`. `opsAssets` is `//go:embed` (`ops.go:30`). - `mountWardKdlExec` (`wardkdl_exec.go:29`) does `fs.ReadDir(execAssets, execAssetsDir)`, then `execAssets.ReadFile` -> `execverb.Parse` -> `execverb.Build` per file. `execAssets` is `//go:embed` (`wardkdl_exec.go:22`). - `fleet.go` parses `fleetGeneratedKDL` (`//go:embed`) via `fleetconfig.Parse`. Every read is `fs.ReadFile` / `fs.ReadDir` against an `embed.FS`. Both satisfy `fs.FS`, so the only hard-wired thing is the **source handle**. The change is to make the three build sites take an `fs.FS` chosen at launch: - var unset -> the baked `embed.FS` (today's exact behavior, the neutral default). - var set -> `os.DirFS(<synced local checkout of the bundle subpath>)`. That is the whole behavioral pivot. `specverb.Build` / `execverb.Build` / `fleetconfig.Parse` are unchanged. ## 1. Fetch transport: git shell-out (recommended) Recommend **`git` shell-out over a bare mirror**, not the forge archive-tarball API. Reasoning grounded in the tree: - **ward already shells `git` for exactly this.** `warmSubstrateRepo` (`container_bootstrap.go:900`) does `git clone --mirror` then `git -C <mirror> remote update --prune` then a working `git clone`. `seedExternalContextMirrors` (`agent_context_seed.go`) does `git clone --mirror` into the gitcache volume. `git_clone.go` / `git_auth.go` already own authenticated clone. Adding an archive-download path is net-new surface competing with machinery that already exists and is drift-tested. - **The settled grammar is "a plain git ref, branch/tag/sha, ward does not care which" (decision 3).** `git fetch`/`git clone --mirror` resolves all three uniformly. Forge archive APIs are inconsistent here: Forgejo/Gitea archive-by-sha support and pathing differ from GitHub's, and pinning to a sha (the operator's integrity story) is a first-class git operation but a second-class archive one. - **Private repos + existing creds.** The bundle for Kai is `coilyco-flight-deck/agentic-os` behind the same Forgejo the binary already authenticates to for releases (`WARD_FORGEJO_BASE`, `forge.go:45`). git shell-out reuses `git_auth.go`. An archive API path would need its own token plumbing. - **The "no git binary, lighter" argument does not apply to this consumer.** ward's whole agent/container half hard-depends on a local `git`, and the bundle-resolving consumer is Kai's fleet, not a bare external adopter. External adopters use a local `.ward/ward.yaml` plus the baked neutral default and never resolve a remote bundle at all, so they never need git for this path. **Escape hatch (decision 3, secondary):** a `file://` / `--bundle <dir>` form skips git entirely and hands `os.DirFS(dir)` straight to the build sites. Land it after the git path. ## 2. Refresh cadence + the exact reuse target **Reuse target, named precisely:** the substrate warmer in `cmd/ward/container_bootstrap.go` - `warmSubstrate` (`:940`), `warmSubstrateRepo` (`:900`), and `substrateMirrorStale` (`:889`) - driven by `WARD_SUBSTRATE_TTL` (default 600s, `container_compute.go:636`, `docs/container-substrate.md`). Its properties are exactly what a hot-path config gate needs: - **TTL-gated fetch keyed on FETCH_HEAD mtime.** `substrateMirrorStale` stats `<mirror>/FETCH_HEAD` and calls it stale only once `now - mtime >= ttl`. A missing FETCH_HEAD (fresh clone/hydrate) is fresh. So a burst of `ward exec build` calls inside the TTL window does **zero** network I/O. - **`flock` serialization.** `warmSubstrateRepo` wraps the ensure-and-freshen in `r.withFlock(lock, ...)` (cli-guard `pkg/flock`, imported `container_bootstrap.go:19`), so concurrent ward processes racing the same mirror do not corrupt it. - **Cache-fallback, never brick offline.** On `remote update` failure it logs `substrate: refresh failed ... (using cached state)` and proceeds on the cached mirror. That is the offline story verbatim. **Proposed factoring:** extract the mirror-ensure-and-freshen core of `warmSubstrateRepo` into a shared `syncGitRef(ctx, spec, ttl)` that returns a local working-checkout path, and have both the substrate warmer and the new config-bundle resolver call it. Same TTL gate, same flock, same cache-fallback, one code path. **Trigger + cache location:** resolve-on-launch, TTL-gated. On host, cache the bundle mirror + checkout under `~/.cache/ward/config-bundle/<hash-of-ref>/` (sibling to `config.CacheDir`, referenced in `agent_log_drain.go:146`); in a container reuse the `ward-gitcache` volume the substrate warmer already uses. TTL default 600s to match `WARD_SUBSTRATE_TTL`, overridable by a new `WARD_CONFIG_TTL`. First resolve populates the cache, then the FETCH_HEAD gate keeps hot-path invocations network-free. Because the checkout is a real directory, `os.DirFS(checkout/<subpath>)` feeds the build sites with zero further copying. ## 3. The var grammar Proposed var name **`WARD_CONFIG_REF`** (parallels `WARD_CONFIG`, which already selects the allowlist per `docs/config-discovery.md`). Grammar, go-getter / Terraform-module style so the `//` subpath split is unambiguous: `<host>/<owner>/<repo>[@<ref>]//<subpath>` Example from the epic: `forgejo.coilysiren.me/coilyco-flight-deck/agentic-os@main//.ward` Parse order (self-describing, no forge assumptions): 1. Split once on the first `//` -> left = repo-spec, right = `subpath` (`.ward`). `//` cannot appear in a host/owner/repo, so the split is safe. 2. Split `left` once on `@` -> `repospec` (`host/owner/repo`) + `ref` (`main`). No `@` -> `ref` defaults to `HEAD` / the remote default branch. 3. Clone URL = `https://<repospec>.git`. `ref` is passed to git untouched - branch, tag, or sha, ward does not classify it (decision 3). 4. The resolved `fs.FS` is `os.DirFS(filepath.Join(checkout, subpath))`. Parsing is a ~20-line pure function with a table-driven test, mirroring the shape of `parseConfigOverrides` (`container_compute.go:576`). ## 4. Topology fold: the ~14 WARD_* values into the bundle The fold is a small, staged, backward-compatible step because **the fleet already treats these as env-over-baked-default DATA**, not identity. `container_compute.go:100` already says so: "Tailnet + tower topology (ward#395): infra DATA, not baked identity. Each value takes a WARD_* env override, the old literal kept as the fail-safe default." And ward#616 already resolves `--config agent.<name>.<key>` into WARD_* env keys (`configEnvKeys`, `container_compute.go:555`). The bundle just becomes a **third, middle tier** in an ordering that already exists. **New precedence (a strict superset of today's `env > baked default`):** `WARD_* env > KDL bundle `topology` block > baked neutral default` Nothing that sets an env var today changes behavior - env still wins. The bundle only fills the slot the baked coilyco literal fills now. The values to fold (from `docs/container-env.md` + the code), each with its current hard-wired home: - Forge / clone base - `forgejoBaseURL = "https://forgejo.coilysiren.me"` (`forgejo_ops.go:23`), `WARD_FORGEJO_BASE`, `WARD_FORGE` / `WARD_CLONE_BASE`. - Forgejo token SSM path - `value ssm "/forgejo/coilyco-ops/api-token"` (`opsassets/forgejo.guardfile.generated.kdl:18`), resolved by `forgejoTokenResolver` / `ssmValueResolver` (`ops.go:112`,`:381`). - Tailnet + tower - `WARD_TAILNET_NETWORK` (`ward-tailnet`), `WARD_TAILNET_PROXY` (`mac-proxy:1055`), `WARD_TOWER_HOST` (`kai-tower-3026`), `WARD_TOWER_OLLAMA_PORT` (`11434`) - the `default*` consts at `container_compute.go:104-127`. - ollama / observe endpoints - `WARD_OLLAMA_URL`, `WARD_TOWER_OLLAMA` (`towerOllamaURL`, `:151`). - Substrate topology - `WARD_SUBSTRATE_SEED` / `_DEST` / `_MANIFEST` / `_TTL` (`container_compute.go:633-636`). - Clone base + target are per-run and stay env-set by the container driver. **Staging (no flag-day):** - **Stage 1 (pure addition):** add a `topology` section to aos's `.ward/` bundle (aos#332), add the bundle-resolve path, and insert the bundle tier between env and baked default in each resolver (`envOr` -> `envOrBundleOr`). Baked defaults and every WARD_* env override stay intact. A run with `WARD_CONFIG_REF` unset is byte-for-byte today. - **Stage 2:** point the fleet's `container_compute.go` driver and infra at `WARD_CONFIG_REF`. The container path that **sets** these WARD_* vars for children (`wardEnv`, `container_compute.go:605`) resolves them from the bundle first, then still passes them through as env, so the in-container `entrypoint.sh` and `container_bootstrap.go` are unchanged - the env contract in `docs/container-env.md` is preserved as the transport, the bundle just becomes its source. The now-redundant literal env sets become harmless. - **Stage 3 (later, optional, out of this epic's critical path):** delete the coilyco-specific baked literals, leaving only the opaque neutral default. Gated on the bundle path being proven in the fleet. ## 5. Integrity + fail-loud (mechanism, not policy) ward does not police which ref (decision 3), but it fails loud whenever the **named source cannot resolve**, consistent with `docs/config-discovery.md`'s set-but-missing override rule. The behavior matrix: - **`WARD_CONFIG_REF` unset** -> baked neutral default `embed.FS`. Never an error. This is decision 4's trivial precedence. - **Set but malformed grammar** (no `//`, empty subpath, empty repospec) -> fail loud naming the exact grammar violation, before any network. Same spirit as `parseConfigOverrides`' loud unknown-key error. - **Set, well-formed, but clone/fetch fails AND no cache exists** -> fail loud naming the unresolvable ref and URL. This is the direct analogue of config-discovery: "an explicit override is never silently ignored or downgraded to the walk-up." ward must not silently fall back to the neutral default when the operator explicitly named a source - that would mask a broken pin. - **Set, fetch fails, but a cached checkout exists** -> warn and use the cache (the substrate warmer's `using cached state`). Offline resilience, not a hard fail, because the operator's intent (use the bundle) is still honored. - **Resolved, but subpath missing or KDL does not parse** -> fail loud. This is already how the current embed path degrades: `opsCommand` (`ops.go:42`) catches a build error and mounts a leaf that surfaces `guardfile runtime failed to mount: %w` on invocation. Reuse that exact degrade-to-error-leaf pattern so a bad bundle cannot silently drop a whole verb surface. - **Integrity pin = the ref itself.** A sha ref *is* the integrity guarantee; ward adds no separate checksum policy (that would be policy, not mechanism). ward **records** the resolved sha in the audit row (ward already stamps `repo_root` per `AGENTS.md` Safety), so every guarded invocation is attributable to the exact bundle commit it compiled from. ## 6. In-flight reconciliation + superseding issue breakdown - **aos#332 stays the linchpin, unchanged** - it homes ward's complete current KDL bundle in aos's `.ward/`. This design's Stage 1 adds a `topology` section to that same bundle. It is the upstream dependency for everything below: the ward-side resolver has nothing to resolve until the bundle exists. - **#644 (build-time overlay) + homebrew-tap#3 (build-from-source tap overlay) - superseded/dropped.** The build-variant matrix is replaced by decision 1's single prebuilt byte-identical artifact. homebrew-tap#3's build-from-source overlay is **dropped**: the single brew formula + scoop become download-and-verify of the prebuilt binary (this dovetails with the existing release plumbing in `AGENTS.md` Release, where `bump-tap-formula` already rewrites the formula `url`+`sha256` to a released artifact - keep that, drop any from-source path). - **ward#645 (neutralize) folds in** as the "keep the baked-in neutral default" half of the config-resolve issue below. **Concrete replacement issue list (all land in `coilyco-flight-deck/ward`, filed off this epic; ordered by dependency):** 1. **Config-resolve path + keep the baked neutral default** (folds #644, homebrew-tap#3's ward-side, ward#645). Make the three build sites (`ops.go` `buildForgejoOps` + `graftForgejoAdminExec`, `wardkdl_exec.go` `mountWardKdlExec`, `fleet.go` `loadFleetConfig`) take an `fs.FS` selected at launch: `embed.FS` when `WARD_CONFIG_REF` is unset (neutral default), `os.DirFS(resolved subpath)` when set. Reuse the `opsCommand` degrade-to-error-leaf on parse failure. Depends on aos#332 for a real bundle to point at (tests use `file://` fixtures). 2. **`WARD_CONFIG_REF` grammar + the shared `syncGitRef` TTL-cache resolver.** The `host/owner/repo@ref//subpath` parser (pure, table-tested) plus the git shell-out resolver factored out of `warmSubstrateRepo` (TTL gate via `substrateMirrorStale`, `withFlock`, cache-fallback). Delivers the `fs.FS` that issue 1 consumes. Depends on issue 1's `fs.FS` seam being in place. 3. **Topology fold, staged.** Add the `topology` section to the bundle and insert the bundle tier into each `envOr` resolver (`env > bundle > baked default`), Stage 1 pure-addition first. Depends on issues 1+2 (needs the resolved bundle) and on aos#332 carrying the section. 4. **Fail-loud + audit integrity.** The behavior matrix in section 5, including recording the resolved bundle sha in the audit row. Small, depends on issue 2's resolver. 5. **Channels become download-and-verify (release plumbing).** Confirm the single brew formula + scoop + byte-identical GitHub/Forgejo artifact (decision 1); retire any build-from-source tap path. Cross-refs homebrew-tap; can proceed in parallel with 1-4 since it is release infra, not the resolve path. **Escape hatch, deferred:** `--bundle` / `file://` local-file mode (decision 3, secondary) as a follow-up once the git path lands - it is a trivial branch in the issue-2 resolver (`os.DirFS` directly, no sync). ## Summary of the recommendation One prebuilt binary, `fs.FS` chosen at launch behind the existing config-discovery precedence, resolved live via `git` shell-out over a bare mirror through the substrate warmer's TTL+flock+cache-fallback machinery (`warmSubstrateRepo`/`substrateMirrorStale`), `WARD_CONFIG_REF=host/owner/repo@ref//subpath`, topology folded in as an `env > bundle > baked-default` middle tier that supersets today's ward#395 ordering, and fail-loud on an unresolvable-and-uncached named source per config-discovery. Every claim ties to a specific site in the current tree, and no part of it opens the neutral-default-contents question. --- Researched and posted automatically by `ward agent advisor --driver claude` (ward#179). This is one-shot research, not a carried change - verify before acting on it.   — Claude (she/her), via `ward agent`

Rows
Columns

[epic] Config-driven ward: one precompiled binary, personal config resolved live from a git ref (supersedes the build-variant matrix) #650

Goal

Settled decisions (do not re-litigate - design within these)

Why this is a small lift (grounding, verified 2026-07-07)

Hard non-goal (parked - do NOT open this)

What the advisor must resolve

Deliverable

Related

🔎 ward agent advisor

Config-driven ward: design proposal (ward#650)

The core change, confirmed against the tree

1. Fetch transport: git shell-out (recommended)

2. Refresh cadence + the exact reuse target

3. The var grammar

4. Topology fold: the ~14 WARD_* values into the bundle

5. Integrity + fail-loud (mechanism, not policy)

6. In-flight reconciliation + superseding issue breakdown

Summary of the recommendation

In-flight reconciliation + filed breakdown (director surface, 2026-07-08)

The linchpin already landed

Filed breakdown (all in ward, ordered by dependency)

The model-config layer the from-scratch design missed (#658)

Folded in / superseded (closing)

Coordinate, not close

Dispatch PAUSED 2026-07-08 (Kai traveling) - state for a cold pickup