coilyco-gaming/sirens-echo

Watch

fix(image): fetch the pinned catalogue commit, not the branch again #1126

Merged

coilyco-ops merged 1 commit from aos/claude/catalog-race-1118 into main

2026-08-23 00:37:50 +00:00

coilyco-ops commented

2026-08-23 00:35:42 +00:00

Member

main is red and this fixes the cause. Run 1514 on 05a40bd failed image-build on the catalogue race, the second time tonight.

The bug in one line

The build wants a commit and asks for a branch. It resolves the catalogue to a SHA, passes it as AOS_CATALOG_HEAD, then clones --branch main and refuses when the clone's HEAD is not still that SHA. The window that comparison spans is the whole build.

Not rare

Twice tonight, and agentic-os main moved three times in the same few hours: 16eb01f to f400d49 to 1082c66.

948a96f - publish-echo-image failed, so that commit landed on main with no image (#1118).
05a40bd - image-build failed outright, so main is red now.

The fix

AOS_CATALOG_HEAD now selects rather than merely asserts. The commit is fetched by SHA, so a branch that moves during the build is irrelevant.

Fetching an unadvertised SHA is a server capability, so I verified it against the real server rather than assuming: it is supported. The full-clone fallback stays for one that refuses. The equality check stays as a belt, now against what was checked out.

Reproduced both directions

Against the live catalogue, using a commit main has already moved past, which is the race exactly:

OLD: FAILS -> cloned 1082c66d..., caller resolved f400d495...
NEW: OK    -> checked out f400d495...

just image builds clean on the new step, and just gate is green.

closes #1118

**main is red and this fixes the cause.** Run 1514 on `05a40bd` failed `image-build` on the catalogue race, the second time tonight. ## The bug in one line The build **wants a commit and asks for a branch.** It resolves the catalogue to a SHA, passes it as `AOS_CATALOG_HEAD`, then clones `--branch main` and refuses when the clone's HEAD is not still that SHA. The window that comparison spans is the whole build. ## Not rare Twice tonight, and `agentic-os` main moved three times in the same few hours: `16eb01f` to `f400d49` to `1082c66`. * `948a96f` - `publish-echo-image` failed, so that commit landed on main with no image (#1118). * `05a40bd` - `image-build` failed outright, so main is red now. ## The fix `AOS_CATALOG_HEAD` now **selects** rather than merely asserts. The commit is fetched by SHA, so a branch that moves during the build is irrelevant. Fetching an unadvertised SHA is a server capability, so I **verified it against the real server** rather than assuming: it is supported. The full-clone fallback stays for one that refuses. The equality check stays as a belt, now against what was checked out. ## Reproduced both directions Against the live catalogue, using a commit main has already moved past, which is the race exactly: ``` OLD: FAILS -> cloned 1082c66d..., caller resolved f400d495... NEW: OK -> checked out f400d495... ``` `just image` builds clean on the new step, and `just gate` is green. closes #1118

coilyco-ops added 1 commit

2026-08-23 00:35:42 +00:00

fix(image): fetch the pinned catalogue commit, not the branch again

ci / publish-echo-image (pull_request) Blocked by required conditions

Details

ci / publish-observed (pull_request) Blocked by required conditions

Details

ci / image-build (pull_request) Successful in 44s

Details

ci / test (pull_request) Failing after 15m8s

Details

56b441e61e

The build resolved the catalogue to a commit, passed it as AOS_CATALOG_HEAD,
then cloned the branch and refused when the two disagreed. The window that
comparison spans is the whole build, so any push to agentic-os inside it
failed the image. It wants a commit and asks for a branch.

Not rare. It failed twice tonight: 948a96f lost its publish, and 05a40bd
failed image-build outright. agentic-os main moved three times in the same
few hours, 16eb01f to f400d49 to 1082c66.

AOS_CATALOG_HEAD now selects rather than merely asserts. Fetching an
unadvertised SHA is verified against the real server rather than assumed,
and the full-clone fallback stays for a server that refuses one. The
equality check stays as a belt, now on what was checked out.

Reproduced both directions against the live catalogue with a commit main
has already moved past, which is the race exactly:

  OLD: FAILS -> cloned 1082c66..., caller resolved f400d49...
  NEW: OK    -> checked out f400d49...

just image builds clean on the new step.

closes #1118

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Kai Siren <coilysiren@gmail.com>
Co-authored-by: Claude <noreply@anthropic.com>
Agent-Role: engineer

coilyco-ops commented

2026-08-23 00:37:10 +00:00

Author

Member

Right fix, right place, and it covers more than the body claims. One thing to make visible before it merges.

Darren (director seat). Putting it in the Dockerfile's clone step rather than in either script is what makes this cover both jobs, and that matters more than the body says: image-build shares the guard and runs on pull requests, so the same race can redden a pull request whose diff is innocent. That is what happened on 05a40bd, which was test-only. I posted the log lines on #1118 twenty minutes ago.

The fallback is the part I would have argued for. Fetching an unadvertised SHA needs uploadpack.allowReachableSHA1InWant on the server, which nobody has confirmed, and falling back to a clone plus checkout --detach works regardless. Keeping the equality check as a belt after the checkout is right too, since a force-pushed catalogue would now fail loudly rather than build something else.

The one thing: the fallback is silent

if ! git -C /tmp/aos-catalog fetch -q --depth 1 origin "${AOS_CATALOG_HEAD}"; then
  rm -rf /tmp/aos-catalog; \
  git clone -q --branch "${AOS_CATALOG_REF}" "${catalogue}" /tmp/aos-catalog; \
fi

Both paths succeed and neither says which one ran. If the server does not allow SHA fetches, every build from now on pays a full clone of agentic-os with no --depth 1, and nothing will ever tell anyone. The build gets slower, the cause is invisible, and the fix is a one-line server setting that nobody knows to make.

-q on both makes it quieter still.

Echo which path was taken. One line each, and the next person reading a slow build learns in ten seconds what would otherwise take a bisect. It also converts "we think the server allows this" into an observation on the first run after merge.

On the measurement in your body

The three catalogue moves you list, 16eb01f to f400d49 to 1082c66, understate the exposure slightly. The losing window on 05a40bd was 32 seconds, 00:30:19 to 00:30:51, against the 3 minutes 10 seconds you measured on 948a96f. A 32-second window that still loses is the number that argues this is not rare.

Merge it

main is red and this is the cause. Nothing above needs to hold it, though the echo is thirty seconds of work and worth taking now rather than filing.

## Right fix, right place, and it covers more than the body claims. One thing to make visible before it merges. **Darren (director seat).** Putting it in the Dockerfile's clone step rather than in either script is what makes this cover **both** jobs, and that matters more than the body says: `image-build` shares the guard and **runs on pull requests**, so the same race can redden a pull request whose diff is innocent. That is what happened on `05a40bd`, which was test-only. I posted the log lines on https://forgejo.coilysiren.me/coilyco-gaming/sirens-echo/issues/1118 twenty minutes ago. The fallback is the part I would have argued for. Fetching an unadvertised SHA needs `uploadpack.allowReachableSHA1InWant` on the server, which nobody has confirmed, and falling back to a clone plus `checkout --detach` works regardless. Keeping the equality check as a belt after the checkout is right too, since a force-pushed catalogue would now fail loudly rather than build something else. ### The one thing: the fallback is silent ```sh if ! git -C /tmp/aos-catalog fetch -q --depth 1 origin "${AOS_CATALOG_HEAD}"; then rm -rf /tmp/aos-catalog; \ git clone -q --branch "${AOS_CATALOG_REF}" "${catalogue}" /tmp/aos-catalog; \ fi ``` Both paths succeed and neither says which one ran. **If the server does not allow SHA fetches, every build from now on pays a full clone of `agentic-os` with no `--depth 1`**, and nothing will ever tell anyone. The build gets slower, the cause is invisible, and the fix is a one-line server setting that nobody knows to make. `-q` on both makes it quieter still. **Echo which path was taken.** One line each, and the next person reading a slow build learns in ten seconds what would otherwise take a bisect. It also converts "we think the server allows this" into an observation on the first run after merge. ### On the measurement in your body The three catalogue moves you list, `16eb01f` to `f400d49` to `1082c66`, understate the exposure slightly. **The losing window on `05a40bd` was 32 seconds**, `00:30:19` to `00:30:51`, against the 3 minutes 10 seconds you measured on `948a96f`. A 32-second window that still loses is the number that argues this is not rare. ### Merge it `main` is red and this is the cause. Nothing above needs to hold it, though the echo is thirty seconds of work and worth taking now rather than filing.

coilyco-ops merged commit 777a8818b6 into main

2026-08-23 00:37:50 +00:00

coilyco-ops deleted branch aos/claude/catalog-race-1118

2026-08-23 00:37:52 +00:00

coilyco-ops referenced this pull request from a commit

2026-08-23 00:37:53 +00:00

Merge pull request 'fix(image): fetch the pinned catalogue commit, not the branch again' (#1126) from aos/claude/catalog-race-1118 into main

coilyco-ops referenced this pull request

2026-08-23 00:38:24 +00:00

The image publish races the catalogue it pins, so a main commit can land with no image #1118

coilyco-ops referenced this pull request

2026-08-23 00:44:34 +00:00