coilyco-gaming/sirens-echo

Watch

main's tip does not publish an image: publish-echo-image fails at 40s while test and image-build pass #1042

New issue

Closed

opened 2026-08-19 04:00:40 +00:00 by coilyco-ops · 1 comment

coilyco-ops commented

2026-08-19 04:00:40 +00:00

Member

Engineer seat, 2026-08-19. Not urgent for the stream, and it will be the moment anything else needs to ship.

What is failing

main at 90eb583c (the #1040 merge):

ci / test               success   56s
ci / image-build        success   30s
ci / publish-echo-image failure   40s
ci / publish-observed   failure    4s

The gate passes and the publish does not. That is a different failure from the layout breakage earlier tonight, where test failed and publish was skipped as a consequence.

What still deploys

The newest successfully published commit is 5a1c89fd, an ancestor of the tip. coilyco-bridge/deploy reconciles pins from published images every five minutes, so the lanes can reach that.

5a1c89fd carries every code fix merged tonight: the answered-round notice fix (#1003), the tool breaker (#1007), the scratch search bound (#1016), and the progress backoff (#1021), plus the docs change (#1039).

What is not deployed is #1040 itself, the /mcp roster re-export, which its own title records as off by default. So the member-visible impact right now is nil.

What breaks next is everything after it. Nothing merged from here reaches the cluster until this is fixed, which is the same wall that hid tonight's fixes for two hours earlier.

What it is not

513766c6 and a7cca400 both published successfully within the hour, so the lane worked recently.

Nothing between the last good publish and the tip touches the publish path. The diff is mcpreexport.go and its test, mcpserver.go, agent.go, config.go, featureflags.go, a new docs page, the knob reference, a board fixture, and my docs edits. No Dockerfile, no scripts/publish-image.sh, no .forgejo/workflows/ci.yml.

So this does not look like #1040's code. It looks environmental.

Where I would look

scripts/publish-image.sh runs on runs-on: deploy and refuses immediately without either secret:

REGISTRY_TOKEN, used for docker login forgejo.coilysiren.me --username coilyco-ops
FORGEJO_EGRESS_PROXY, required for the dependency build

A 40-second failure is long enough to have logged in and started work, so an absent secret is the less likely of the two shapes and an expired token or a registry or proxy fault is the more likely. publish-observed failing at 4s is consistent with it simply following its needs.

What I could not do

Read the job log. This Forgejo does not expose the Actions job endpoints: /actions/runs/1315/jobs and /actions/jobs/{id}/logs both return 404 resource does not exist. So the cause above is inference from timing and the diff, not from the failure text. Somebody with the web UI can settle it in one click, and that is the next step rather than any code change.

Handing off rather than acting: this is a live publish lane and a credential question, which is the ops seat's rather than mine.

**Engineer seat, 2026-08-19. Not urgent for the stream, and it will be the moment anything else needs to ship.** ## What is failing `main` at **`90eb583c`** (the #1040 merge): ``` ci / test success 56s ci / image-build success 30s ci / publish-echo-image failure 40s ci / publish-observed failure 4s ``` **The gate passes and the publish does not.** That is a different failure from the layout breakage earlier tonight, where `test` failed and publish was skipped as a consequence. ## What still deploys The newest **successfully published** commit is **`5a1c89fd`**, an ancestor of the tip. `coilyco-bridge/deploy` reconciles pins from published images every five minutes, so the lanes can reach that. `5a1c89fd` carries every code fix merged tonight: the answered-round notice fix (#1003), the tool breaker (#1007), the scratch search bound (#1016), and the progress backoff (#1021), plus the docs change (#1039). **What is not deployed is #1040 itself**, the `/mcp` roster re-export, which its own title records as off by default. So the member-visible impact right now is nil. **What breaks next is everything after it.** Nothing merged from here reaches the cluster until this is fixed, which is the same wall that hid tonight's fixes for two hours earlier. ## What it is not `513766c6` and `a7cca400` both published successfully within the hour, so the lane worked recently. Nothing between the last good publish and the tip touches the publish path. The diff is `mcpreexport.go` and its test, `mcpserver.go`, `agent.go`, `config.go`, `featureflags.go`, a new docs page, the knob reference, a board fixture, and my docs edits. **No Dockerfile, no `scripts/publish-image.sh`, no `.forgejo/workflows/ci.yml`.** So this does not look like #1040's code. It looks environmental. ## Where I would look `scripts/publish-image.sh` runs on `runs-on: deploy` and refuses immediately without either secret: * `REGISTRY_TOKEN`, used for `docker login forgejo.coilysiren.me --username coilyco-ops` * `FORGEJO_EGRESS_PROXY`, required for the dependency build A 40-second failure is long enough to have logged in and started work, so an absent secret is the less likely of the two shapes and an expired token or a registry or proxy fault is the more likely. `publish-observed` failing at 4s is consistent with it simply following its `needs`. ## What I could not do **Read the job log.** This Forgejo does not expose the Actions job endpoints: `/actions/runs/1315/jobs` and `/actions/jobs/{id}/logs` both return `404 resource does not exist`. So the cause above is inference from timing and the diff, not from the failure text. **Somebody with the web UI can settle it in one click**, and that is the next step rather than any code change. Handing off rather than acting: this is a live publish lane and a credential question, which is the ops seat's rather than mine.

coilyco-ops added the

labels

2026-08-19 04:07:01 +00:00

coilyco-ops commented

2026-08-19 04:11:28 +00:00

Author

Member

Resolved on its own. Closing, with the one part worth keeping.

69d7b96, the #1027 merge and the current tip, is green on all four:

ci / test               success  57s
ci / image-build        success  32s
ci / publish-echo-image success  43s
ci / publish-observed   success   3s

So the failure on 90eb583c was transient, not the expired token or proxy fault this issue guessed at. My inference was wrong in the direction of a standing cause, and the diff evidence that pointed away from #1040's code was the half that held up.

The part that is not moot

A transient publish failure leaves main red with no retry. Nothing re-runs the job, so the state persisted until an unrelated merge happened to run it again roughly forty minutes later. During that window the newest published image was two commits behind the tip.

That is survivable here because merges are frequent. It is worth knowing because the recovery was luck rather than a mechanism, and the image-coverage workflow that asks the registry whether main's tip has an image runs hourly, so it would have reported the gap rather than closed it.

Not filing a retry as work. Recording it so the next person reading a red publish knows to re-run before investigating.

**Resolved on its own. Closing, with the one part worth keeping.** `69d7b96`, the #1027 merge and the current tip, is green on all four: ``` ci / test success 57s ci / image-build success 32s ci / publish-echo-image success 43s ci / publish-observed success 3s ``` So the failure on `90eb583c` was **transient**, not the expired token or proxy fault this issue guessed at. My inference was wrong in the direction of a standing cause, and the diff evidence that pointed away from #1040's code was the half that held up. ## The part that is not moot **A transient publish failure leaves `main` red with no retry.** Nothing re-runs the job, so the state persisted until an unrelated merge happened to run it again roughly forty minutes later. During that window the newest published image was two commits behind the tip. That is survivable here because merges are frequent. It is worth knowing because the recovery was luck rather than a mechanism, and the `image-coverage` workflow that asks the registry whether main's tip has an image runs hourly, so it would have reported the gap rather than closed it. Not filing a retry as work. Recording it so the next person reading a red publish knows to re-run before investigating.

coilyco-ops closed this issue

2026-08-19 04:11:37 +00:00

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

coilyco-gaming/sirens-echo#1042

No description provided.

Rows
Columns