Raise director burndown capacity after stability pressure relieved #1310

New issue

Closed

opened 2026-07-15 04:26:27 +00:00 by coilyco-ops · 2 comments

coilyco-ops commented

2026-07-15 04:26:27 +00:00

Member

Those pressures have mostly relieved. Retune the director / engineer safety layer so an interactive burndown can run roughly double that capacity, targeting about 6 total in-flight engineers across a 2-repo scope, while keeping the guardrails that prevent host thrash.

Acceptance:

Find the actual limiting layer rather than only changing the advertised director default. Check director max-parallel handling, reservation / pool ceilings, open-PR or per-repo backpressure, broker launch limits, and any host-memory/OOM preflight gate.
Raise the default or config path that currently causes the observed 3-engineer ceiling so a normal interactive burndown can reach roughly 6 total engineers across 2 repos.
Keep an explicit emergency escape hatch separate from the default. --override-capacity remains per-launch only and should not become the director path.
Preserve warnings or hard stops for real memory pressure. The target is relaxed steady-state capacity, not ignoring OOM signals.
Update docs/help where operators learn the expected capacity.
Add or adjust focused tests around the limit calculation or config parsing.

Context from the director surface on 2026-07-15: user requested roughly doubling capacity because the original caution was calibrated for merge conflicts, broker instability, and memory issues, and that pressure has mostly relieved.

Current director burndown tuning is still conservative from the earlier failure mode: merge conflicts, dispatch broker instability, and memory pressure limited practical throughput to about 3 engineer containers total across a 2-repo scope. Those pressures have mostly relieved. Retune the director / engineer safety layer so an interactive burndown can run roughly double that capacity, targeting about 6 total in-flight engineers across a 2-repo scope, while keeping the guardrails that prevent host thrash. Acceptance: * Find the actual limiting layer rather than only changing the advertised director default. Check director max-parallel handling, reservation / pool ceilings, open-PR or per-repo backpressure, broker launch limits, and any host-memory/OOM preflight gate. * Raise the default or config path that currently causes the observed 3-engineer ceiling so a normal interactive burndown can reach roughly 6 total engineers across 2 repos. * Keep an explicit emergency escape hatch separate from the default. `--override-capacity` remains per-launch only and should not become the director path. * Preserve warnings or hard stops for real memory pressure. The target is relaxed steady-state capacity, not ignoring OOM signals. * Update docs/help where operators learn the expected capacity. * Add or adjust focused tests around the limit calculation or config parsing. Context from the director surface on 2026-07-15: user requested roughly doubling capacity because the original caution was calibrated for merge conflicts, broker instability, and memory issues, and that pressure has mostly relieved.

coilyco-ops added the

headless

labels

2026-07-15 04:26:27 +00:00

coilyco-ops commented

2026-07-15 04:27:21 +00:00

Author

Member

WARDED_WORKFLOW: reservation-held

reservation details

Holder: launch intent for container engineer-codex-ward-1310 on host kais-macbook-pro-2.local.

Accepted by ward agent --harness codex (reserved 2026-07-15T04:27:20Z). Concurrent ward agent runs are blocked until this intent becomes visible or the intent is released. The stale-intent fallback is still TTL-bounded (3h TTL). --override-reservation overrides.

Do not comment on or edit this issue to steer the run while it is reserved. The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a new issue, dispatched fresh. That is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494).

run seed context — what this run is carrying (ward#609)

Resolved: coilyco-flight-deck/ward#1310 · branch issue-1310 · harness codex · workflow pull-request-and-merge
Run: engineer-codex-ward-1310 · ward v0.697.0 · dispatched 2026-07-15T04:27:10Z
Comment thread: 0 included in the pre-flight read, 0 stripped (ward's own automated comments).

Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.697.0).

— Codex, via ward agent

WARDED_WORKFLOW: reservation-held <details><summary>reservation details</summary> Holder: launch intent for container `engineer-codex-ward-1310` on host `kais-macbook-pro-2.local`. Accepted by `ward agent --harness codex` (reserved 2026-07-15T04:27:20Z). Concurrent `ward agent` runs are blocked until this intent becomes visible or the intent is released. The stale-intent fallback is still TTL-bounded (3h TTL). `--override-reservation` overrides. **Do not comment on or edit this issue to steer the run while it is reserved.** The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a **new issue, dispatched fresh**. That is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494). <details><summary>run seed context — what this run is carrying (ward#609)</summary> - **Resolved:** `coilyco-flight-deck/ward#1310` · branch `issue-1310` · harness `codex` · workflow `pull-request-and-merge` - **Run:** `engineer-codex-ward-1310` · ward `v0.697.0` · dispatched `2026-07-15T04:27:10Z` - **Comment thread:** 0 included in the pre-flight read, 0 stripped (ward's own automated comments). Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.697.0). </details> </details>  — Codex, via `ward agent`

coilyco-ops referenced this issue from a pull request that will close it,

2026-07-15 04:36:27 +00:00

feat(director): raise burndown default to 6 #1311

coilyco-ops closed this issue

2026-07-15 04:38:35 +00:00

coilyco-ops commented

2026-07-15 04:39:07 +00:00

Author

Member

WARDED_WORKFLOW: #1311

details

workflow: pull-request-and-merge; review summary: skipped because the temporary ward default is pending brokered QA
felt: direct. the cap change was mostly a config and docs alignment once the live default path was pinned down.
confidence: high
surprises: the bundled defaults had drifted above the intended 6-slot target, so I aligned the source bundle, baked defaults, and help text together.
follow-ups: none

WARDED_WORKFLOW: https://forgejo.coilysiren.me/coilyco-flight-deck/ward/pulls/1311 <details><summary>details</summary> workflow: pull-request-and-merge; review summary: skipped because the temporary ward default is pending brokered QA felt: direct. the cap change was mostly a config and docs alignment once the live default path was pinned down. confidence: high surprises: the bundled defaults had drifted above the intended 6-slot target, so I aligned the source bundle, baked defaults, and help text together. follow-ups: none </details>