aos publish-dev-base exceeds the shared runner 15m cap - release run 1102 cancelled by 'maximum run time' #551
Labels
No labels
burndown-2026-06
coherence-core
consult
headless
interactive
P0
P1
P2
P3
P4
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
coilyco-flight-deck/infrastructure#551
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Fresh evidence from tonight's two-stage rollout (aos run 1102, release branch, 2026-07-12 21:40Z): the run's test + plan jobs went green, then publish-dev-base died at the runner wall - the job log ends with 'The runner cancelled the job because it exceeds the maximum run time' and Forgejo records the run as cancelled. No runner pod restarts this time (all 2d4h stable, node at 47% memory after the comfyui scale-down per #547), so this is the OTHER phantom-cancellation class behind the aos 'publish-dev-base red on main' history (aos#452): the shared docker runner's runner.timeout: 15m (deploy/forgejo-runner.yml, docs/forgejo-runner-timeouts.md) vs a multi-arch dev-base family build that needs well beyond 15m when caches are cold.
docs/forgejo-runner-timeouts.md already names the remedy: a job family that regularly needs more than the runner cap gets a dedicated label/runner, not a fatter shared cap. tap-writer is host-executor and DinD-free, so it does not fit. Proposal: a 'heavy' docker runner class (capacity 1, runner.timeout ~90m) that aos release.yml's publish-dev-base targets via runs-on, leaving the shared 15m envelope intact.
Until then aos stage 2 only completes when the buildx/registry cache is warm. ward (v0.631.0) and cli-guard (v0.96.0) already released end to end tonight - their release jobs fit the cap.
Refs #547, aos#452, aos#477, ward#1117.