coilyco-flight-deck/ward

Fork 0

Spec: dissolve the eco pipeline into ward ops eco (one area, native|server target), extend the exec dialect, validate live-fire #585

New issue

Closed

opened 2026-07-03 19:56:18 +00:00 by coilysiren · 4 comments

coilysiren commented

2026-07-03 19:56:18 +00:00

Owner

Problem

ward#582 landed ward eco {test,promote} - a verb plus a Go smoke-gate + snapshot/health/rollback state machine - in ward. Eco is Kai-fleet-specific: no place in the product shipped to external users. Same config-placement violation as --aws/--tailnet (ward#578). It "lived in coily" (retired) and migrated to the wrong home.

Target: `ward ops eco {test,promote}`, ONE area, expressed in the KDL spec - NO standalone binary

Eco is an operator area like forgejo/aws/kubectl/tailscale, so ward eco -> ward ops eco, defined by a ward-kdl guardfile (exec dialect) wrapping the infra shell scripts. ward#582's Go EcoExecutor + state machine dissolve - orchestration becomes the KDL spec composing wrapped commands, not compiled Go. ward loses all eco vocabulary.

native vs server: a target selector, not a hardcoded ssh (the complexity to design for)

Eco runs sometimes native (Kai's local Windows box) and sometimes on the server (kai-server, Linux). ward ops eco is one area with a target selector - native | server - that picks the transport per step:

server -> ssh, already a KDL primitive: cli-guard's execverb supports passthrough ssh / passthrough tailscale ssh (reaches kai-server over the tailnet). Reuse it, do not invent it.
native -> local exec on the Windows box.

The orchestration primitives (rollback, canary) stay transport-agnostic: they compose steps, each step runs through the selected target's transport.

The real complexity: native is Windows-local, server is Linux-ssh, so the same logical op (snapshot, restart, health-check) may need a per-target implementation - one bash/Linux script cannot run native on Windows. The spec must never hardcode kai-server-ssh; promote/ops target wherever the Eco currently runs. The ssh half is solved; the native-Windows half + the target selection is the new surface.

The dialect work: make the KDL spec express the pipeline

The spec-verb / exec dialect (cli-guard's specverb) already does multi-step sequences + data-threading + fail-when (e.g. ward ops forgejo issue comment). The eco pipeline needs:

a health-gated rollback / compensating step - on a failed post-restart health-check, run rollback;
a canary watch - poll over a window, roll back on degradation.

Grow the dialect so it CAN express those (transport-agnostic), not a Go binary. Lands in cli-guard's specverb (the engine), so it generalizes to any guarded-rollback ops pipeline. That extension is the deliverable; eco is its first consumer.

Validation: live-fire, not unit tests

Go unit tests cannot prove the two things that matter: the gate's log markers matching a real EcoServer boot, and rollback actually recovering the target. Prove it by live-fire - run ward ops eco test/promote against a real/throwaway Eco (native and server) and observe the gate catch a real ModKit failure and rollback restore. Live-fire is the only honest validation for a markers-match-reality / auto-rollback path. Folds in the log-marker check flagged as real-boot-only.

Migration

Retire ward eco + the Go EcoExecutor/state-machine from ward - all eco vocab out.
Extend cli-guard's specverb / exec dialect with the transport-agnostic rollback + canary primitives.
Express ward ops eco {test,promote} as a ward-kdl guardfile with a native|server target selector: server reuses passthrough (tailscale) ssh, native is Windows-local exec.
Provide per-target op implementations where Windows-local and Linux-ssh diverge.
Fold ward#584 (apply the tested working copy) + log-marker tuning in, proven by live-fire.

ward#582 (code to dissolve), ward#584 (apply-step), ward#578 (same principle), eco-ops#30 (pipeline spec), cli-guard execverb/specverb (ssh passthrough already exists; rollback/canary primitives grow here), AGENTS.md operator-verb doctrine.

## Problem ward#582 landed `ward eco {test,promote}` - a verb plus a Go smoke-gate + snapshot/health/rollback state machine - in **ward**. Eco is Kai-fleet-specific: no place in the product shipped to external users. Same config-placement violation as `--aws`/`--tailnet` (ward#578). It "lived in coily" (retired) and migrated to the wrong home. ## Target: `ward ops eco {test,promote}`, ONE area, expressed in the KDL spec - NO standalone binary Eco is an operator area like forgejo/aws/kubectl/tailscale, so `ward eco` -> `ward ops eco`, defined by a **ward-kdl guardfile (exec dialect)** wrapping the infra shell scripts. **ward#582's Go `EcoExecutor` + state machine dissolve** - orchestration becomes the KDL spec composing wrapped commands, not compiled Go. ward loses all eco vocabulary. ## native vs server: a target selector, not a hardcoded ssh (the complexity to design for) Eco runs **sometimes native** (Kai's local Windows box) and **sometimes on the server** (kai-server, Linux). `ward ops eco` is one area with a **target selector** - `native` | `server` - that picks the transport per step: - **server** -> ssh, **already a KDL primitive**: cli-guard's execverb supports `passthrough ssh` / `passthrough tailscale ssh` (reaches kai-server over the tailnet). Reuse it, do not invent it. - **native** -> local exec on the Windows box. The orchestration primitives (rollback, canary) stay **transport-agnostic**: they compose steps, each step runs through the selected target's transport. **The real complexity:** native is **Windows-local**, server is **Linux-ssh**, so the same logical op (snapshot, restart, health-check) may need a **per-target implementation** - one bash/Linux script cannot run native on Windows. The spec must **never hardcode kai-server-ssh**; promote/ops target wherever the Eco currently runs. The ssh half is solved; the native-Windows half + the target selection is the new surface. ## The dialect work: make the KDL spec express the pipeline The spec-verb / exec dialect (cli-guard's specverb) already does multi-step sequences + data-threading + `fail-when` (e.g. `ward ops forgejo issue comment`). The eco pipeline needs: - a **health-gated rollback / compensating step** - on a failed post-restart health-check, run rollback; - a **canary watch** - poll over a window, roll back on degradation. **Grow the dialect so it CAN express those** (transport-agnostic), not a Go binary. Lands in **cli-guard's specverb** (the engine), so it generalizes to any guarded-rollback ops pipeline. That extension is the deliverable; eco is its first consumer. ## Validation: live-fire, not unit tests Go unit tests cannot prove the two things that matter: the gate's log markers matching a **real EcoServer boot**, and rollback actually recovering the target. **Prove it by live-fire** - run `ward ops eco test`/`promote` against a real/throwaway Eco (native and server) and observe the gate catch a real ModKit failure and rollback restore. Live-fire is the only honest validation for a markers-match-reality / auto-rollback path. Folds in the log-marker check flagged as real-boot-only. ## Migration 1. Retire `ward eco` + the Go `EcoExecutor`/state-machine from ward - all eco vocab out. 2. Extend cli-guard's specverb / exec dialect with the transport-agnostic rollback + canary primitives. 3. Express `ward ops eco {test,promote}` as a ward-kdl guardfile with a `native`|`server` target selector: `server` reuses `passthrough (tailscale) ssh`, `native` is Windows-local exec. 4. Provide per-target op implementations where Windows-local and Linux-ssh diverge. 5. Fold ward#584 (apply the tested working copy) + log-marker tuning in, proven by live-fire. ## Related ward#582 (code to dissolve), ward#584 (apply-step), ward#578 (same principle), eco-ops#30 (pipeline spec), cli-guard execverb/specverb (ssh passthrough already exists; rollback/canary primitives grow here), AGENTS.md operator-verb doctrine.

coilysiren changed title from ~~Spec: relocate the eco pipeline out of ward to the operator-verb layer (ward ops eco)~~ to Spec: dissolve the eco pipeline into the KDL spec (ward ops eco), extend the exec dialect, validate by live-fire

2026-07-03 19:59:46 +00:00

coilysiren changed title from ~~Spec: dissolve the eco pipeline into the KDL spec (ward ops eco), extend the exec dialect, validate by live-fire~~ to Spec: dissolve the eco pipeline into ward ops eco (one area, native|server target), extend the exec dialect, validate live-fire

2026-07-03 20:03:53 +00:00

coilyco-ops commented

2026-07-03 20:06:01 +00:00

Member

🔒 Reserved by ward agent --driver claude — container engineer-claude-ward-585 on host KAI-DESKTOP-TOWER is carrying this issue (reserved 2026-07-03T20:05:56Z). Concurrent ward agent runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); --force overrides.

Do not comment on or edit this issue to steer the run while it is reserved. The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a new issue, dispatched fresh — that is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494).

— Claude (she/her), via ward agent

🔒 Reserved by `ward agent --driver claude` — container `engineer-claude-ward-585` on host `KAI-DESKTOP-TOWER` is carrying this issue (reserved 2026-07-03T20:05:56Z). Concurrent `ward agent` runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); `--force` overrides. **Do not comment on or edit this issue to steer the run while it is reserved.** The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a **new issue, dispatched fresh** — that is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494).  — Claude (she/her), via `ward agent`

coilysiren referenced this issue from a commit

2026-07-03 20:25:35 +00:00

docs(eco): design spec for dissolving the eco pipeline into ward ops eco (closes #585)

coilysiren closed this issue

2026-07-03 20:25:35 +00:00

coilysiren commented

2026-07-03 20:27:53 +00:00

Author

Owner

Honest retro: this one fought back on its premises more than its plumbing. Two things surfaced early that reshaped the whole thing.

First, the "native = Kai's local Windows box" axis - the issue's stated core complexity - doesn't match the repo. Eco is Linux-only on kai-server, "native" in the eco runbooks is the non-k3s LAN process (not Windows), and the Windows-native tower is blocked upstream on infrastructure#356. So the per-OS Windows/Linux divergence the issue wanted me to build has nothing to build against yet. I rescoped native to a Linux local-vs-ssh transport axis in the spec and deferred Windows behind that foundation.

Second, "extend the exec dialect / specverb" turned out to be a genuine new-engine ask, not a tweak. specverb's action engine is multi-step but HTTP-bound with no compensation; execverb owns ssh/local transport but has zero sequencing; no "fire a resolved step" abstraction spans both. That's real architecture, and it belongs in cli-guard - hence cli-guard#187 rather than a speculative cross-repo push (also the run's own grant steer).

And the issue says plainly that live-fire is the only honest validation for the auto-rollback path. This is a headless Linux box with no Eco, no kai-server, no Windows tower - I can't live-fire it, and I wasn't going to rip out the working, unit-tested Go pipeline and replace it with an un-live-fired declarative engine across three repos. So ward eco test stays live; dissolution trails a validated replacement (migration order in the spec).

Confidence: high on the analysis, the engine-gap diagnosis, and the decomposition. Low-to-nil on anything I couldn't run, which is the point - the engine build and the live-fire are the real work and they're owner/hardware-gated. Rough edges / follow-ups: cli-guard#187, infrastructure#461, Windows gated on infrastructure#356. I closed #585 as the spec deliverable given the "Spec:" framing - reopen it if you wanted the code landed this run and I misread the intent.

WARD-OUTCOME: done - landed the ward#585 design spec (docs/ops-eco.md) + filed the implementation as cli-guard#187 (engine) and infrastructure#461 (transport wiring); the Go dissolution and live-fire are deliberately deferred, not done this run. Honest retro: this one fought back on its premises more than its plumbing. Two things surfaced early that reshaped the whole thing. First, the "native = Kai's local Windows box" axis - the issue's stated core complexity - doesn't match the repo. Eco is Linux-only on kai-server, "native" in the eco runbooks is the non-k3s LAN process (not Windows), and the Windows-native tower is blocked upstream on infrastructure#356. So the per-OS Windows/Linux divergence the issue wanted me to build has nothing to build against yet. I rescoped native to a Linux local-vs-ssh transport axis in the spec and deferred Windows behind that foundation. Second, "extend the exec dialect / specverb" turned out to be a genuine new-engine ask, not a tweak. specverb's action engine is multi-step but HTTP-bound with no compensation; execverb owns ssh/local transport but has zero sequencing; no "fire a resolved step" abstraction spans both. That's real architecture, and it belongs in cli-guard - hence cli-guard#187 rather than a speculative cross-repo push (also the run's own grant steer). And the issue says plainly that live-fire is the only honest validation for the auto-rollback path. This is a headless Linux box with no Eco, no kai-server, no Windows tower - I can't live-fire it, and I wasn't going to rip out the working, unit-tested Go pipeline and replace it with an un-live-fired declarative engine across three repos. So `ward eco test` stays live; dissolution trails a validated replacement (migration order in the spec). Confidence: high on the analysis, the engine-gap diagnosis, and the decomposition. Low-to-nil on anything I couldn't run, which is the point - the engine build and the live-fire are the real work and they're owner/hardware-gated. Rough edges / follow-ups: cli-guard#187, infrastructure#461, Windows gated on infrastructure#356. I closed #585 as the spec deliverable given the "Spec:" framing - reopen it if you wanted the code landed this run and I misread the intent.

coilysiren reopened this issue

2026-07-03 20:28:22 +00:00

coilysiren commented

2026-07-03 20:28:22 +00:00

Author

Owner

⚠️ Reopened: a granted `--repo` push did not land

This run held --repo grants and closed against coilyco-flight-deck/ward, but the reaper could not confirm every granted repo's work reached its main. A secondary push can be silently rejected (a non-fast-forward on a busy main, a dead/rotated PAT) while the primary push succeeds, so the issue is reopened rather than left reading "done" with the cross-repo half lost.

`coilyco-flight-deck/cli-guard`

Verdict: 1 local commit(s) never reached origin/main.
Preserved on: ward-salvage/cli-guard-672418b2

git fetch https://forgejo.coilysiren.me/coilyco-flight-deck/cli-guard.git ward-salvage/cli-guard-672418b2
git checkout -b ward-salvage/cli-guard-672418b2 FETCH_HEAD

`coilyco-flight-deck/infrastructure`

Verdict: 1 local commit(s) never reached origin/main.
Preserved on: ward-salvage/infrastructure-f7037e8a

git fetch https://forgejo.coilysiren.me/coilyco-flight-deck/infrastructure.git ward-salvage/infrastructure-f7037e8a
git checkout -b ward-salvage/infrastructure-f7037e8a FETCH_HEAD

Re-run the cross-repo half, or - per ward#291 - file it as a native issue in the granted repo so it becomes a single-repo run that sidesteps this failure mode.

— Claude (she/her), via ward agent

## ⚠️ Reopened: a granted `--repo` push did not land This run held `--repo` grants and closed against `coilyco-flight-deck/ward`, but the reaper could not confirm every granted repo's work reached its `main`. A secondary push can be silently rejected (a non-fast-forward on a busy `main`, a dead/rotated PAT) while the primary push succeeds, so the issue is **reopened** rather than left reading "done" with the cross-repo half lost. ### `coilyco-flight-deck/cli-guard` - **Verdict:** 1 local commit(s) never reached `origin/main`. - **Preserved on:** `ward-salvage/cli-guard-672418b2` ```bash git fetch https://forgejo.coilysiren.me/coilyco-flight-deck/cli-guard.git ward-salvage/cli-guard-672418b2 git checkout -b ward-salvage/cli-guard-672418b2 FETCH_HEAD ``` ### `coilyco-flight-deck/infrastructure` - **Verdict:** 1 local commit(s) never reached `origin/main`. - **Preserved on:** `ward-salvage/infrastructure-f7037e8a` ```bash git fetch https://forgejo.coilysiren.me/coilyco-flight-deck/infrastructure.git ward-salvage/infrastructure-f7037e8a git checkout -b ward-salvage/infrastructure-f7037e8a FETCH_HEAD ``` Re-run the cross-repo half, or - per ward#291 - file it as a native issue in the granted repo so it becomes a single-repo run that sidesteps this failure mode.  — Claude (she/her), via `ward agent`

coilysiren referenced this issue

2026-07-03 20:34:55 +00:00

reaper: granted-repo landing check false-positives, spurious empty salvage + reopen when a granted main advanced after clone #587

coilysiren commented

2026-07-03 20:35:28 +00:00

Author

Owner

Reconcile: the reopen was a spurious empty-salvage, no cross-repo work was lost

Kai asked me (director read-only surface) to verify the two salvage branches from the reaper's reopen above. Both are empty:

ward-salvage/cli-guard-672418b2 tip 8886aee is the current cli-guard main tip. Zero commits unique to main, zero file diff. The commit is fix(sandbox): exec a symlinked tool... Fixes ward#546, unrelated to eco.
ward-salvage/infrastructure-f7037e8a tip 7364d83 is an ancestor of current infrastructure main (ddce631). Zero unique commits, zero file diff. The commit is Merge issue-460... CI_RELEASE_TOKEN, unrelated to eco.

So the reaper's "1 local commit never reached origin/main" fired on both, but what it salvaged was already-landed, unrelated HEAD, not eco content. Nothing is recoverable-but-stranded, and the git fetch + re-push instruction in the reopen is a no-op. Root cause and fix are tracked in ward#587 (the granted-repo landing check is HEAD-ancestry based, not content based, so it false-positives whenever a granted main advanced after clone).

Actual state of the eco work from this run:

Spec landed on ward main: docs/ops-eco.md, docs/eco-test.md.
ward NOT dissolved yet - the Go code (cmd/ward/eco*.go + tests) is still present, exactly as the engineer's retro said (dissolution deferred behind a validated, live-fired replacement).
Implementation deferred to open child issues: cli-guard#187 (specverb transport-agnostic step + rollback/compensate + canary-watch engine), infrastructure#461 (native|server transport wiring, Windows behind infrastructure#356), ward#584 (promote ships the tested working-copy artifact).

Closing again as the spec deliverable. The remaining implementation lives in those three open child issues, and the reopen cause (lost cross-repo commits) is void. Reopen if the intent was to land the code this run rather than the spec plus decomposition.

Claude (she/her), director surface

## Reconcile: the reopen was a spurious empty-salvage, no cross-repo work was lost Kai asked me (director read-only surface) to verify the two salvage branches from the reaper's reopen above. Both are **empty**: - `ward-salvage/cli-guard-672418b2` tip `8886aee` **is** the current cli-guard `main` tip. Zero commits unique to `main`, zero file diff. The commit is `fix(sandbox): exec a symlinked tool... Fixes ward#546`, unrelated to eco. - `ward-salvage/infrastructure-f7037e8a` tip `7364d83` is an **ancestor** of current infrastructure `main` (`ddce631`). Zero unique commits, zero file diff. The commit is `Merge issue-460... CI_RELEASE_TOKEN`, unrelated to eco. So the reaper's "1 local commit never reached origin/main" fired on both, but what it salvaged was already-landed, unrelated HEAD, not eco content. Nothing is recoverable-but-stranded, and the `git fetch` + re-push instruction in the reopen is a no-op. Root cause and fix are tracked in **ward#587** (the granted-repo landing check is HEAD-ancestry based, not content based, so it false-positives whenever a granted `main` advanced after clone). Actual state of the eco work from this run: - **Spec landed** on ward `main`: `docs/ops-eco.md`, `docs/eco-test.md`. - **ward NOT dissolved yet** - the Go code (`cmd/ward/eco*.go` + tests) is still present, exactly as the engineer's retro said (dissolution deferred behind a validated, live-fired replacement). - **Implementation deferred to open child issues:** cli-guard#187 (specverb transport-agnostic step + rollback/compensate + canary-watch engine), infrastructure#461 (native|server transport wiring, Windows behind infrastructure#356), ward#584 (promote ships the tested working-copy artifact). Closing again as the spec deliverable. The remaining implementation lives in those three open child issues, and the reopen cause (lost cross-repo commits) is void. Reopen if the intent was to land the code this run rather than the spec plus decomposition. - Claude (she/her), director surface

coilysiren closed this issue

2026-07-03 20:35:28 +00:00