coilyco-flight-deck/infrastructure
2
0
Fork 0
Code Issues 118 Pull requests Projects Releases Packages Wiki Activity Actions 2

Add a fully self-hosted Goose → Ward director → Goose engineer acceptance run #520

New issue
Open
opened 2026-07-10 05:39:49 +00:00 by coilysiren · 4 comments
coilysiren commented 2026-07-10 05:39:49 +00:00
Owner
Copy link

Goal

Build a repeatable end-to-end acceptance run proving that a self-hosted model can bootstrap Ward in a clean environment and carry a real Forgejo issue through completion:

  1. An outer Goose one-shot installs and configures Ward.
  2. It launches a Ward director for one specific issue.
  3. The director uses Goose to make the dispatch decision.
  4. The director dispatches a Goose engineer.
  5. The engineer makes and lands a real code change.
  6. The engineer posts its Ward outcome and closes the issue.
  7. The director observes completion and drains.
  8. An independent verifier confirms the complete chain.

The target repository is coilysiren/inbox.

Hard requirement: Goose at every model boundary

All model-backed work must use the self-hosted Goose/Ollama path:

  • Bootstrap operator: Goose.
  • Ward director dispatch decision: Goose.
  • Ward engineer: Goose.

Claude and Codex must not participate.

Ward’s current local-harness behavior—printing that self-assessment is unavailable and dispatching the top issue by rank—does not satisfy this issue. The director must make a real Goose model call and receive a parsed DISPATCH decision.

Implement a local-model-safe director decision path. Do not merely remove the existing host-one-shot trust gate and expose an unsandboxed host execution path.

Issue-specific director scope

The director must be constrained to the exact evaluation issue and must never dispatch another backlog item.

Support an invocation shaped like:

warded director coilysiren/inbox#123 \
  --harness goose \
  --engineer-harness goose \
  --max-parallel 1 \
  --no-triage \
  --poll-interval 5s

A full Forgejo issue URL should also be acceptable.

When given an issue ref, the director should:

  • Derive its repository scope from the ref.
  • Fetch and validate that issue.
  • Require it to be open and eligible for headless execution.
  • Expose only that issue to the dispatch decision.
  • Refuse or exit cleanly if it is closed, ineligible, or already reserved.
  • Never pick another issue from the repository backlog.

This should reconcile the existing documentation implying warded director #N support with the currently repo-scoped implementation.

Evaluation issue

The evaluator creates one real issue in coilysiren/inbox:

Title: [ward-eval:<run-id>] Record execution timestamp

Create `runs/<run-id>.txt`.

The file must contain exactly one RFC3339 UTC timestamp followed by a newline.
Generate the timestamp while completing this issue.

Do not modify any other files.

Commit and land the change using the repository's configured workflow. Close
this issue only after the change is present on the default branch.

Apply the existing P4 and headless labels.

A unique file per run avoids overwriting previous evidence and makes verification unambiguous.

Kubernetes execution environment

Run each evaluation as a disposable k3s Job.

Use one privileged Ubuntu runner container containing only the prerequisites:

  • Non-root operator user.
  • Homebrew.
  • Git and curl.
  • Docker engine and CLI.
  • Goose.
  • Access to the self-hosted Ollama endpoint.
  • No Ward installation.
  • No Ward configuration or caches from previous runs.

The container entrypoint should:

  1. Start dockerd.
  2. Wait for Docker to become healthy.
  3. Launch the outer Goose one-shot.
  4. Collect Ward logs, audit records, Docker events, and the one-shot transcript.
  5. Stop dockerd.
  6. Exit with the evaluation result.

Use a single runner/DinD container rather than a permanent sidecar so:

  • Ward’s bind-source paths share one filesystem.
  • $HOME, /workspace, staged credentials, and Ward assets resolve correctly.
  • The Job can terminate naturally after the director drains.

The Ollama service should remain outside the disposable Job and be reached through the cluster network.

Bootstrap one-shot

The outer Goose model receives:

  • The Ward repository URL.
  • The exact evaluation issue URL.
  • WARD_CONFIG_REF.
  • The Forgejo and Ollama environment contract.
  • An instruction that it must use a Ward director and must not invoke an engineer directly.

It must:

  1. Install Ward using the documented Homebrew path.
  2. Run ward setup and resolve the supplied runtime bundle.
  3. Inspect the director interface.
  4. Launch the Goose director for the exact issue.
  5. Wait for the director to drain.
  6. Report the resulting issue, commit, and outcome.

The outer model must not clone coilysiren/inbox, edit files, commit, push, or close the issue itself.

Engineer behavior

The director-dispatched engineer must:

  • Run in Ward’s normal ephemeral engineer container.
  • Use Goose against the self-hosted Ollama endpoint.
  • Clone coilysiren/inbox.
  • Create only runs/<run-id>.txt.
  • Write exactly one valid RFC3339 UTC timestamp plus a trailing newline.
  • Commit the change.
  • Land it through the repository’s configured direct-main workflow.
  • Post WARD-OUTCOME: done.
  • Close the issue only after the commit is on the default branch.

Credentials and isolation

Use a dedicated Forgejo evaluation bot with access limited to coilysiren/inbox.

It needs only the permissions required to:

  • Read and clone the repository.
  • Push the evaluation commit.
  • Read and update issues.
  • Read the Ward runtime bundle and agent image, if private.

The Job must:

  • Run only on a dedicated or disposable k3s worker.
  • Disable service-account token automounting.
  • Receive no Kubernetes RBAC.
  • Use no host-path mounts.
  • Receive no unrelated infrastructure credentials.
  • Have a hard execution deadline.
  • Delete its Docker state and writable filesystem after completion.

Independent verification

Do not trust the model’s final response as proof.

The verifier must confirm:

  • The outer one-shot installed Ward in the fresh Job.
  • A Goose-backed director model call occurred.
  • The director returned a parsed decision selecting the exact issue.
  • The ranked fallback path was not used.
  • The director launched an engineer-goose-* container for the issue.
  • Docker events prove that engineer container existed.
  • Ward’s audit records director launch followed by engineer dispatch.
  • The issue contains the expected Ward reservation and WARD-OUTCOME: done.
  • The issue is closed.
  • A new commit reached the default branch after the issue was created.
  • The commit changes only runs/<run-id>.txt.
  • The file contains one valid RFC3339 UTC timestamp within the run window.
  • The commit carries the expected Goose attribution.
  • No other issue was dispatched, modified, or closed.
  • No Claude or Codex invocation occurred.
  • No Forgejo token appears in collected logs or audit output.

Failure handling

If the Job times out or dies:

  • Mark the evaluation failed.
  • Stop any surviving engineer container.
  • Preserve the collected evidence.
  • Close the abandoned evaluation issue with a cleanup comment.
  • Do not treat cleanup closure as a successful Ward outcome.

Done when

A single command can launch a fresh k3s Job and produce a verified chain:

Goose bootstrap
  → cold Ward installation
  → Goose Ward director
  → Goose dispatch decision
  → Goose Ward engineer
  → real commit on coilysiren/inbox
  → real issue closure
  → director drain
  → independently verified PASS
## Goal Build a repeatable end-to-end acceptance run proving that a self-hosted model can bootstrap Ward in a clean environment and carry a real Forgejo issue through completion: 1. An outer Goose one-shot installs and configures Ward. 2. It launches a Ward director for one specific issue. 3. The director uses Goose to make the dispatch decision. 4. The director dispatches a Goose engineer. 5. The engineer makes and lands a real code change. 6. The engineer posts its Ward outcome and closes the issue. 7. The director observes completion and drains. 8. An independent verifier confirms the complete chain. The target repository is `coilysiren/inbox`. ## Hard requirement: Goose at every model boundary All model-backed work must use the self-hosted Goose/Ollama path: * Bootstrap operator: Goose. * Ward director dispatch decision: Goose. * Ward engineer: Goose. Claude and Codex must not participate. Ward’s current local-harness behavior—printing that self-assessment is unavailable and dispatching the top issue by rank—does **not** satisfy this issue. The director must make a real Goose model call and receive a parsed `DISPATCH` decision. Implement a local-model-safe director decision path. Do not merely remove the existing host-one-shot trust gate and expose an unsandboxed host execution path. ## Issue-specific director scope The director must be constrained to the exact evaluation issue and must never dispatch another backlog item. Support an invocation shaped like: ```bash warded director coilysiren/inbox#123 \ --harness goose \ --engineer-harness goose \ --max-parallel 1 \ --no-triage \ --poll-interval 5s ``` A full Forgejo issue URL should also be acceptable. When given an issue ref, the director should: * Derive its repository scope from the ref. * Fetch and validate that issue. * Require it to be open and eligible for headless execution. * Expose only that issue to the dispatch decision. * Refuse or exit cleanly if it is closed, ineligible, or already reserved. * Never pick another issue from the repository backlog. This should reconcile the existing documentation implying `warded director #N` support with the currently repo-scoped implementation. ## Evaluation issue The evaluator creates one real issue in `coilysiren/inbox`: ```markdown Title: [ward-eval:<run-id>] Record execution timestamp Create `runs/<run-id>.txt`. The file must contain exactly one RFC3339 UTC timestamp followed by a newline. Generate the timestamp while completing this issue. Do not modify any other files. Commit and land the change using the repository's configured workflow. Close this issue only after the change is present on the default branch. ``` Apply the existing `P4` and `headless` labels. A unique file per run avoids overwriting previous evidence and makes verification unambiguous. ## Kubernetes execution environment Run each evaluation as a disposable k3s `Job`. Use one privileged Ubuntu runner container containing only the prerequisites: * Non-root operator user. * Homebrew. * Git and curl. * Docker engine and CLI. * Goose. * Access to the self-hosted Ollama endpoint. * No Ward installation. * No Ward configuration or caches from previous runs. The container entrypoint should: 1. Start `dockerd`. 2. Wait for Docker to become healthy. 3. Launch the outer Goose one-shot. 4. Collect Ward logs, audit records, Docker events, and the one-shot transcript. 5. Stop `dockerd`. 6. Exit with the evaluation result. Use a single runner/DinD container rather than a permanent sidecar so: * Ward’s bind-source paths share one filesystem. * `$HOME`, `/workspace`, staged credentials, and Ward assets resolve correctly. * The Job can terminate naturally after the director drains. The Ollama service should remain outside the disposable Job and be reached through the cluster network. ## Bootstrap one-shot The outer Goose model receives: * The Ward repository URL. * The exact evaluation issue URL. * `WARD_CONFIG_REF`. * The Forgejo and Ollama environment contract. * An instruction that it must use a Ward director and must not invoke an engineer directly. It must: 1. Install Ward using the documented Homebrew path. 2. Run `ward setup` and resolve the supplied runtime bundle. 3. Inspect the director interface. 4. Launch the Goose director for the exact issue. 5. Wait for the director to drain. 6. Report the resulting issue, commit, and outcome. The outer model must not clone `coilysiren/inbox`, edit files, commit, push, or close the issue itself. ## Engineer behavior The director-dispatched engineer must: * Run in Ward’s normal ephemeral engineer container. * Use Goose against the self-hosted Ollama endpoint. * Clone `coilysiren/inbox`. * Create only `runs/<run-id>.txt`. * Write exactly one valid RFC3339 UTC timestamp plus a trailing newline. * Commit the change. * Land it through the repository’s configured `direct-main` workflow. * Post `WARD-OUTCOME: done`. * Close the issue only after the commit is on the default branch. ## Credentials and isolation Use a dedicated Forgejo evaluation bot with access limited to `coilysiren/inbox`. It needs only the permissions required to: * Read and clone the repository. * Push the evaluation commit. * Read and update issues. * Read the Ward runtime bundle and agent image, if private. The Job must: * Run only on a dedicated or disposable k3s worker. * Disable service-account token automounting. * Receive no Kubernetes RBAC. * Use no host-path mounts. * Receive no unrelated infrastructure credentials. * Have a hard execution deadline. * Delete its Docker state and writable filesystem after completion. ## Independent verification Do not trust the model’s final response as proof. The verifier must confirm: * The outer one-shot installed Ward in the fresh Job. * A Goose-backed director model call occurred. * The director returned a parsed decision selecting the exact issue. * The ranked fallback path was not used. * The director launched an `engineer-goose-*` container for the issue. * Docker events prove that engineer container existed. * Ward’s audit records director launch followed by engineer dispatch. * The issue contains the expected Ward reservation and `WARD-OUTCOME: done`. * The issue is closed. * A new commit reached the default branch after the issue was created. * The commit changes only `runs/<run-id>.txt`. * The file contains one valid RFC3339 UTC timestamp within the run window. * The commit carries the expected Goose attribution. * No other issue was dispatched, modified, or closed. * No Claude or Codex invocation occurred. * No Forgejo token appears in collected logs or audit output. ## Failure handling If the Job times out or dies: * Mark the evaluation failed. * Stop any surviving engineer container. * Preserve the collected evidence. * Close the abandoned evaluation issue with a cleanup comment. * Do not treat cleanup closure as a successful Ward outcome. ## Done when A single command can launch a fresh k3s Job and produce a verified chain: ```text Goose bootstrap → cold Ward installation → Goose Ward director → Goose dispatch decision → Goose Ward engineer → real commit on coilysiren/inbox → real issue closure → director drain → independently verified PASS ```
coilyco-ops commented 2026-07-10 09:02:39 +00:00
Member
Copy link

WARD-DISPATCH: failed ❌

failure details

This forwarded dispatch failed after the issue was already reserved.

Attempted harness: goose
Attempted run: ward agent engineer coilyco-flight-deck/infrastructure#520 --harness goose --repo coilyco-flight-deck/ward --ward-version v0.569.0 --details Carry the first landable slice toward the issue 520 acceptance loop. Start from the current repository state, preserve Goose at every model-backed boundary in the acceptance flow, and keep the implementation focused on making the repeatable k3s Goose bootstrap -> Ward director -> Goose engineer verifier more real. Do not use Claude or Codex inside the acceptance flow being built. --print
Container: engineer-goose-infrastructure-520
Container created: no running engineer was observed.
Host log: /Users/kai/.ward/agent-logs/dispatch/20260710T090234Z-director-codex-vg55-coilyco-flight-deck-infrastructure-520.log
Failure: ward agent engineer --harness goose: refusing to dispatch a container pinned to ward v0.569.0, older than this host's ward v0.573.0: the in-container reaper is the last line against lost or false-salvaged work, and an older reaper can reproduce already-fixed bugs (ward#529). Pass --allow-ward-downgrade to override, or clear --ward-version / WARD_AGENT_VERSION to inherit this host's ward

Retry: choose another harness if the first one is down, or rerun with --force if the reservation is stale.

— Goose, via ward agent

<!-- ward-agent-reservation-released --> <!-- ward-needs-redispatch --> WARD-DISPATCH: failed ❌ <details><summary>failure details</summary> This forwarded dispatch failed after the issue was already reserved. Attempted harness: `goose` Attempted run: `ward agent engineer coilyco-flight-deck/infrastructure#520 --harness goose --repo coilyco-flight-deck/ward --ward-version v0.569.0 --details Carry the first landable slice toward the issue 520 acceptance loop. Start from the current repository state, preserve Goose at every model-backed boundary in the acceptance flow, and keep the implementation focused on making the repeatable k3s Goose bootstrap -> Ward director -> Goose engineer verifier more real. Do not use Claude or Codex inside the acceptance flow being built. --print` Container: `engineer-goose-infrastructure-520` Container created: no running engineer was observed. Host log: `/Users/kai/.ward/agent-logs/dispatch/20260710T090234Z-director-codex-vg55-coilyco-flight-deck-infrastructure-520.log` Failure: `ward agent engineer --harness goose: refusing to dispatch a container pinned to ward v0.569.0, older than this host's ward v0.573.0: the in-container reaper is the last line against lost or false-salvaged work, and an older reaper can reproduce already-fixed bugs (ward#529). Pass --allow-ward-downgrade to override, or clear --ward-version / WARD_AGENT_VERSION to inherit this host's ward` Retry: choose another harness if the first one is down, or rerun with `--force` if the reservation is stale. </details> <!-- ward-agent-signature --> — Goose, via `ward agent`
coilyco-ops commented 2026-07-10 09:02:48 +00:00
Member
Copy link

WARD-DISPATCH: failed ❌

failure details

This forwarded dispatch failed after the issue was already reserved.

Attempted harness: goose
Attempted run: ward agent engineer coilyco-flight-deck/infrastructure#520 --harness goose --repo coilyco-flight-deck/ward --ward-version v0.569.0 --details Carry the first landable slice toward the issue 520 acceptance loop. Start from the current repository state, preserve Goose at every model-backed boundary in the acceptance flow, and keep the implementation focused on making the repeatable k3s Goose bootstrap -> Ward director -> Goose engineer verifier more real. Do not use Claude or Codex inside the acceptance flow being built.
Container: engineer-goose-infrastructure-520
Container created: no running engineer was observed.
Host log: /Users/kai/.ward/agent-logs/dispatch/20260710T090243Z-director-codex-vg55-coilyco-flight-deck-infrastructure-520.log
Failure: ward agent engineer --harness goose: refusing to dispatch a container pinned to ward v0.569.0, older than this host's ward v0.573.0: the in-container reaper is the last line against lost or false-salvaged work, and an older reaper can reproduce already-fixed bugs (ward#529). Pass --allow-ward-downgrade to override, or clear --ward-version / WARD_AGENT_VERSION to inherit this host's ward

Retry: choose another harness if the first one is down, or rerun with --force if the reservation is stale.

— Goose, via ward agent

<!-- ward-agent-reservation-released --> <!-- ward-needs-redispatch --> WARD-DISPATCH: failed ❌ <details><summary>failure details</summary> This forwarded dispatch failed after the issue was already reserved. Attempted harness: `goose` Attempted run: `ward agent engineer coilyco-flight-deck/infrastructure#520 --harness goose --repo coilyco-flight-deck/ward --ward-version v0.569.0 --details Carry the first landable slice toward the issue 520 acceptance loop. Start from the current repository state, preserve Goose at every model-backed boundary in the acceptance flow, and keep the implementation focused on making the repeatable k3s Goose bootstrap -> Ward director -> Goose engineer verifier more real. Do not use Claude or Codex inside the acceptance flow being built.` Container: `engineer-goose-infrastructure-520` Container created: no running engineer was observed. Host log: `/Users/kai/.ward/agent-logs/dispatch/20260710T090243Z-director-codex-vg55-coilyco-flight-deck-infrastructure-520.log` Failure: `ward agent engineer --harness goose: refusing to dispatch a container pinned to ward v0.569.0, older than this host's ward v0.573.0: the in-container reaper is the last line against lost or false-salvaged work, and an older reaper can reproduce already-fixed bugs (ward#529). Pass --allow-ward-downgrade to override, or clear --ward-version / WARD_AGENT_VERSION to inherit this host's ward` Retry: choose another harness if the first one is down, or rerun with `--force` if the reservation is stale. </details> <!-- ward-agent-signature --> — Goose, via `ward agent`
coilyco-ops commented 2026-07-10 09:03:09 +00:00
Member
Copy link

WARD-RESERVATION: held 🔒

reservation details

Holder: container engineer-goose-infrastructure-520 on host kais-macbook-pro-2.local.

Reserved by ward agent --harness goose (reserved 2026-07-10T09:03:09Z). Concurrent ward agent runs are blocked until it finishes or the reservation goes stale (1h TTL). --force overrides.

Do not comment on or edit this issue to steer the run while it is reserved. The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a new issue, dispatched fresh. That is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494).

run seed context — what this run is carrying (ward#609)
  • Resolved: coilyco-flight-deck/infrastructure#520 · branch issue-520 · harness goose · workflow direct-main
  • Run: engineer-goose-infrastructure-520 · ward v0.573.0 · dispatched 2026-07-10T09:03:09Z
  • Comment thread: 2 included in the pre-flight read, 0 stripped (ward's own automated comments).

Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.573.0).

— Goose, via ward agent

<!-- ward-agent-reservation --> WARD-RESERVATION: held 🔒 <details><summary>reservation details</summary> Holder: container `engineer-goose-infrastructure-520` on host `kais-macbook-pro-2.local`. Reserved by `ward agent --harness goose` (reserved 2026-07-10T09:03:09Z). Concurrent `ward agent` runs are blocked until it finishes or the reservation goes stale (1h TTL). `--force` overrides. **Do not comment on or edit this issue to steer the run while it is reserved.** The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a **new issue, dispatched fresh**. That is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494). <details><summary>run seed context — what this run is carrying (ward#609)</summary> - **Resolved:** `coilyco-flight-deck/infrastructure#520` · branch `issue-520` · harness `goose` · workflow `direct-main` - **Run:** `engineer-goose-infrastructure-520` · ward `v0.573.0` · dispatched `2026-07-10T09:03:09Z` - **Comment thread:** 2 included in the pre-flight read, 0 stripped (ward's own automated comments). - included: @coilyco-ops (2026-07-10T09:02:39Z), @coilyco-ops (2026-07-10T09:02:48Z) Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.573.0). </details> </details> <!-- ward-agent-signature --> — Goose, via `ward agent`
coilyco-ops commented 2026-07-10 12:17:55 +00:00
Member
Copy link

🔎 ward agent advisor

ward agent advisor --harness codex ran a one-shot standard research pass on this question:

Read the issue title, body, and comment thread below as the research brief. Identify the decisions, options, open questions, and risks, then give a clear advisory recommendation that can be posted back on the issue.

pre-commit installed at .git/hooks/pre-commit
pre-commit installed at .git/hooks/commit-msg

Researched and posted automatically by ward agent advisor --harness codex (ward#179). This is one-shot research, not a carried change - verify before acting on it.

— Codex, via ward agent

### 🔎 ward agent advisor `ward agent advisor --harness codex` ran a one-shot **standard** research pass on this question: > Read the issue title, body, and comment thread below as the research brief. Identify the decisions, options, open questions, and risks, then give a clear advisory recommendation that can be posted back on the issue. --- pre-commit installed at .git/hooks/pre-commit pre-commit installed at .git/hooks/commit-msg --- Researched and posted automatically by `ward agent advisor --harness codex` (ward#179). This is one-shot research, not a carried change - verify before acting on it. <!-- ward-agent-reply --> <!-- ward-agent-signature --> — Codex, via `ward agent`
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
issue-515
issue-486
issue-529
issue-500
issue-483
No results found.
Labels
Clear labels   
burndown-2026-06
Backlog burndown June 2026

  
coherence-core
Core review set for the warded control plane coherence milestone. These issues form the release spine; adjacent milestone issues are stretch or supporting work.

  
consult
A human decision, design, or external action must happen first. Fail-closed default for unlabeled/low-confidence.

  
headless
Agent can take it from open issue to merged change with no human in the loop. The auto-burndown queue.

  
interactive
Agent does the work but pauses at a human checkpoint mid-flight (design choice, destructive step, human-only verification).

  
P0
now / blocking / urgent

  
P1
next / important

  
P2
backlog / will do

  
P3
someday / low

  
P4
icebox / frozen / reopen if it becomes real

No labels
burndown-2026-06
coherence-core
consult
headless
interactive
P0
P1
P2
P3
P4
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
coilyco-flight-deck/infrastructure#520
No description provided.