coilyco-flight-deck/agentic-os

Watch

Add timeout and diagnostics to dev-base publish-image CI #335

New issue

Closed

opened 2026-07-08 10:27:17 +00:00 by coilyco-ops · 2 comments

coilyco-ops commented

2026-07-08 10:27:17 +00:00

Owner

Why

Run 695 is still running in the publish-image job long after the normal release job finished:

URL: https://forgejo.coilysiren.me/coilyco-flight-deck/agentic-os/actions/runs/695/jobs/1/attempt/1
Commit: 0b78030c218721ace1a164bce39a43ed7cd6ae19
Title: chore(dev-base): install mcporter in dev-base image
release job: success in about 12 seconds (09:53:52Z to 09:54:04Z)
publish-image job: started 09:54:05Z, still reported running after 30+ minutes

Recent successful publish-image jobs on this repo finished in roughly 25-35 seconds, so this is outside the normal envelope. The current workflow has no timeout-minutes on publish-image, and the docker buildx build --push step has no phase-level timeout or diagnostic trap, so a stuck npm install, buildx cache exchange, registry push, or builder hang can sit until the runner-level timeout.

Grounding

The triggering Dockerfile change adds:

ARG MCPORTER_VERSION=0.12.3
RUN set -eux; \
    npm install -g "mcporter@${MCPORTER_VERSION}"; \
    mcporter --help >/dev/null

The publish job in .forgejo/workflows/release.yml then builds and pushes linux/amd64,linux/arm64 with buildx and registry cache.

Scope

Add an explicit timeout-minutes to the publish-image job. Pick a practical bound that allows real multi-arch dev-base builds but fails a wedged build loudly.
Add diagnostics around the build/push path so the next incident says where it stopped. Useful minimum: print builder state before and after bootstrap, docker/buildx versions, selected DOCKER_HOST, and enough buildx progress output to locate the stalled Dockerfile step.
Consider --progress=plain for docker buildx build so the Forgejo log records the active Dockerfile step instead of an opaque spinner.
If a shell-level timeout is better for just docker buildx build --push, wrap that step and preserve the job-level timeout as the outer guard.
If the mcporter install itself is the suspect, make the Dockerfile step fail fast or verify the package install in a way that does not hang indefinitely.

Acceptance

publish-image cannot run indefinitely when buildx, npm, cache export, or registry push wedges.
A failed or timed-out publish leaves enough CI log context to identify the phase that hung.
The normal release/tag job behavior remains unchanged.
A fresh dev-base publish still pushes both ${TAG} and latest for linux/amd64,linux/arm64.

Codex, via warded director

## Why Run 695 is still running in the `publish-image` job long after the normal release job finished: - URL: https://forgejo.coilysiren.me/coilyco-flight-deck/agentic-os/actions/runs/695/jobs/1/attempt/1 - Commit: `0b78030c218721ace1a164bce39a43ed7cd6ae19` - Title: `chore(dev-base): install mcporter in dev-base image` - `release` job: success in about 12 seconds (`09:53:52Z` to `09:54:04Z`) - `publish-image` job: started `09:54:05Z`, still reported `running` after 30+ minutes Recent successful `publish-image` jobs on this repo finished in roughly 25-35 seconds, so this is outside the normal envelope. The current workflow has no `timeout-minutes` on `publish-image`, and the `docker buildx build --push` step has no phase-level timeout or diagnostic trap, so a stuck npm install, buildx cache exchange, registry push, or builder hang can sit until the runner-level timeout. ## Grounding The triggering Dockerfile change adds: ```Dockerfile ARG MCPORTER_VERSION=0.12.3 RUN set -eux; \ npm install -g "mcporter@${MCPORTER_VERSION}"; \ mcporter --help >/dev/null ``` The publish job in `.forgejo/workflows/release.yml` then builds and pushes `linux/amd64,linux/arm64` with buildx and registry cache. ## Scope - Add an explicit `timeout-minutes` to the `publish-image` job. Pick a practical bound that allows real multi-arch dev-base builds but fails a wedged build loudly. - Add diagnostics around the build/push path so the next incident says where it stopped. Useful minimum: print builder state before and after bootstrap, docker/buildx versions, selected `DOCKER_HOST`, and enough buildx progress output to locate the stalled Dockerfile step. - Consider `--progress=plain` for `docker buildx build` so the Forgejo log records the active Dockerfile step instead of an opaque spinner. - If a shell-level timeout is better for just `docker buildx build --push`, wrap that step and preserve the job-level timeout as the outer guard. - If the mcporter install itself is the suspect, make the Dockerfile step fail fast or verify the package install in a way that does not hang indefinitely. ## Acceptance - `publish-image` cannot run indefinitely when buildx, npm, cache export, or registry push wedges. - A failed or timed-out publish leaves enough CI log context to identify the phase that hung. - The normal release/tag job behavior remains unchanged. - A fresh dev-base publish still pushes both `${TAG}` and `latest` for `linux/amd64,linux/arm64`.  - Codex, via `warded director`

coilyco-ops commented

2026-07-08 10:27:25 +00:00

Author

Owner

🔒 Reserved by ward agent --harness codex — container engineer-codex-agentic-os-335 on host kais-macbook-pro-2.local is carrying this issue (reserved 2026-07-08T10:27:23Z). Concurrent ward agent runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); --force overrides.

Do not comment on or edit this issue to steer the run while it is reserved. The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a new issue, dispatched fresh — that is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494).

run seed context — what this run is carrying (ward#609)

Resolved: coilyco-flight-deck/agentic-os#335 · branch issue-335 · harness codex · workflow direct-main
Run: engineer-codex-agentic-os-335 · ward v0.449.0 · dispatched 2026-07-08T10:27:23Z
Comment thread: 0 included in the pre-flight read, 0 stripped (ward's own automated comments).

Issue body as seeded:

## Why

Run 695 is still running in the `publish-image` job long after the normal release job finished:

- URL: https://forgejo.coilysiren.me/coilyco-flight-deck/agentic-os/actions/runs/695/jobs/1/attempt/1
- Commit: `0b78030c218721ace1a164bce39a43ed7cd6ae19`
- Title: `chore(dev-base): install mcporter in dev-base image`
- `release` job: success in about 12 seconds (`09:53:52Z` to `09:54:04Z`)
- `publish-image` job: started `09:54:05Z`, still reported `running` after 30+ minutes

Recent successful `publish-image` jobs on this repo finished in roughly 25-35 seconds, so this is outside the normal envelope. The current workflow has no `timeout-minutes` on `publish-image`, and the `docker buildx build --push` step has no phase-level timeout or diagnostic trap, so a stuck npm install, buildx cache exchange, registry push, or builder hang can sit until the runner-level timeout.

## Grounding

The triggering Dockerfile change adds:

` ` `Dockerfile
ARG MCPORTER_VERSION=0.12.3
RUN set -eux; \
    npm install -g "mcporter@${MCPORTER_VERSION}"; \
    mcporter --help >/dev/null
` ` `

The publish job in `.forgejo/workflows/release.yml` then builds and pushes `linux/amd64,linux/arm64` with buildx and registry cache.

## Scope

- Add an explicit `timeout-minutes` to the `publish-image` job. Pick a practical bound that allows real multi-arch dev-base builds but fails a wedged build loudly.
- Add diagnostics around the build/push path so the next incident says where it stopped. Useful minimum: print builder state before and after bootstrap, docker/buildx versions, selected `DOCKER_HOST`, and enough buildx progress output to locate the stalled Dockerfile step.
- Consider `--progress=plain` for `docker buildx build` so the Forgejo log records the active Dockerfile step instead of an opaque spinner.
- If a shell-level timeout is better for just `docker buildx build --push`, wrap that step and preserve the job-level timeout as the outer guard.
- If the mcporter install itself is the suspe

… (truncated to 2000 chars; full body is on this issue)

Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.449.0).

— Codex, via ward agent

🔒 Reserved by `ward agent --harness codex` — container `engineer-codex-agentic-os-335` on host `kais-macbook-pro-2.local` is carrying this issue (reserved 2026-07-08T10:27:23Z). Concurrent `ward agent` runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); `--force` overrides. **Do not comment on or edit this issue to steer the run while it is reserved.** The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a **new issue, dispatched fresh** — that is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494). <details><summary>run seed context — what this run is carrying (ward#609)</summary> - **Resolved:** `coilyco-flight-deck/agentic-os#335` · branch `issue-335` · harness `codex` · workflow `direct-main` - **Run:** `engineer-codex-agentic-os-335` · ward `v0.449.0` · dispatched `2026-07-08T10:27:23Z` - **Comment thread:** 0 included in the pre-flight read, 0 stripped (ward's own automated comments). **Issue body as seeded:** ``` ## Why Run 695 is still running in the `publish-image` job long after the normal release job finished: - URL: https://forgejo.coilysiren.me/coilyco-flight-deck/agentic-os/actions/runs/695/jobs/1/attempt/1 - Commit: `0b78030c218721ace1a164bce39a43ed7cd6ae19` - Title: `chore(dev-base): install mcporter in dev-base image` - `release` job: success in about 12 seconds (`09:53:52Z` to `09:54:04Z`) - `publish-image` job: started `09:54:05Z`, still reported `running` after 30+ minutes Recent successful `publish-image` jobs on this repo finished in roughly 25-35 seconds, so this is outside the normal envelope. The current workflow has no `timeout-minutes` on `publish-image`, and the `docker buildx build --push` step has no phase-level timeout or diagnostic trap, so a stuck npm install, buildx cache exchange, registry push, or builder hang can sit until the runner-level timeout. ## Grounding The triggering Dockerfile change adds: ` ` `Dockerfile ARG MCPORTER_VERSION=0.12.3 RUN set -eux; \ npm install -g "mcporter@${MCPORTER_VERSION}"; \ mcporter --help >/dev/null ` ` ` The publish job in `.forgejo/workflows/release.yml` then builds and pushes `linux/amd64,linux/arm64` with buildx and registry cache. ## Scope - Add an explicit `timeout-minutes` to the `publish-image` job. Pick a practical bound that allows real multi-arch dev-base builds but fails a wedged build loudly. - Add diagnostics around the build/push path so the next incident says where it stopped. Useful minimum: print builder state before and after bootstrap, docker/buildx versions, selected `DOCKER_HOST`, and enough buildx progress output to locate the stalled Dockerfile step. - Consider `--progress=plain` for `docker buildx build` so the Forgejo log records the active Dockerfile step instead of an opaque spinner. - If a shell-level timeout is better for just `docker buildx build --push`, wrap that step and preserve the job-level timeout as the outer guard. - If the mcporter install itself is the suspe ``` … (truncated to 2000 chars; full body is on this issue) Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.449.0). </details>  — Codex, via `ward agent`

coilyco-ops referenced this issue from a commit

2026-07-08 10:30:32 +00:00

fix(ci): add publish-image timeout and diagnostics

coilyco-ops closed this issue

2026-07-08 10:30:32 +00:00

coilyco-ops commented

2026-07-08 10:31:01 +00:00

Author

Owner

WARD-OUTCOME: done - added a publish-image job timeout, plain buildx progress, builder diagnostics, and bounded the mcporter install in the Dockerfile.

The change was straightforward once the failure mode was pinned down. The only thing that fought back was making the diagnostics useful without creating a new failure path, so I kept the host check after DOCKER_HOST selection and used an explicit status block around the timed build. Confidence is good, with the main rough edge being that the real proof will still come from the next publish run.

WARD-OUTCOME: done - added a publish-image job timeout, plain buildx progress, builder diagnostics, and bounded the mcporter install in the Dockerfile. The change was straightforward once the failure mode was pinned down. The only thing that fought back was making the diagnostics useful without creating a new failure path, so I kept the host check after DOCKER_HOST selection and used an explicit status block around the timed build. Confidence is good, with the main rough edge being that the real proof will still come from the next publish run.

coilyco-ops referenced this issue

2026-08-29 11:25:35 +00:00

An argN or any-arg guard on a kubectl grant would silently not bind, because umbra's valueFlags table is AWS-shaped #1351

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

coilyco-flight-deck/agentic-os#335

No description provided.

Rows
Columns