coilyco-flight-deck/agentic-os

Watch

The release image's PYTHONPATH shadows the pinned hook rev in CI #971

New issue

Closed

opened 2026-08-07 20:05:20 +00:00 by coilyco-ops · 2 comments

coilyco-ops commented

2026-08-07 20:05:20 +00:00

Member

Symptom

A consumer repo can pass pre-commit run --all-files on a developer host and fail the identical command in CI, with no difference in the tree or in .pre-commit-config.yaml.

Found while unbreaking coilyco-gaming/factory-game-v3 CI (that repo's issue #87). docs/FEATURES.md at 5933 chars passed locally and failed in CI against a 4000-char cap.

Cause

agentic-os:release sets PYTHONPATH=/opt/agentic-os/python. Pre-commit runs each hook from a per-repo venv, and PYTHONPATH lands ahead of that venv's site-packages on sys.path, so the image's agentic_os wins over the one pre-commit installed from the pinned rev.

Observed inside the image, from the hook's own interpreter:

hook venv python: /root/.cache/pre-commit/repo56i9jckq/py_env-python3/bin/python
resolved agentic_os -> /opt/agentic-os/python/agentic_os/__init__.py
sys.path[0:3] -> ['', '/opt/agentic-os/python', '.../python313.zip']

The two copies genuinely disagree. At the pinned v0.221.0, docs/FEATURES.md takes TRIFECTA_MAX_CHARS = 12_500. The image's copy adds a dedicated FEATURES_MAX_CHARS = 4_000 and applies it to that path. Same config, same file, two verdicts.

Why it matters

rev: is the contract that says which validator a repo is checked against. Right now it selects the code on a developer host and is ignored in CI, so every consumer's pin is advisory there. The failure mode is quiet: the hook reports a real violation, just not one the pinned rev would have flagged, and the repo owner has no local way to reproduce it.

It also cuts the other way. A repo could pass CI against a newer, laxer validator while the pinned rev would have failed it.

Possible directions

Scope the interpreter injection so it does not reach pre-commit hook venvs, for example dropping PYTHONPATH for the hook subprocess or shipping the CLI surface as an installed console script instead.
Or make the shadowing explicit and intended: have the image assert that a consumer's pinned rev matches the image's version and fail loudly on drift, rather than silently substituting.

The first keeps rev meaningful. The second at least makes the substitution visible.

Acceptance

A consumer repo pinned to rev X is checked against rev X's validators in the release image, or is told clearly that it is not.
The local and CI verdicts for the same tree and config agree.

Provenance

Found while landing coilyco-gaming/factory-game-v3#87. That repo took the trim rather than a cap override, since the 4000-char cap is the intended one for docs/FEATURES.md. This issue is the cross-repo half.

## Symptom A consumer repo can pass `pre-commit run --all-files` on a developer host and fail the identical command in CI, with no difference in the tree or in `.pre-commit-config.yaml`. Found while unbreaking `coilyco-gaming/factory-game-v3` CI (that repo's issue #87). `docs/FEATURES.md` at 5933 chars passed locally and failed in CI against a 4000-char cap. ## Cause `agentic-os:release` sets `PYTHONPATH=/opt/agentic-os/python`. Pre-commit runs each hook from a per-repo venv, and PYTHONPATH lands ahead of that venv's site-packages on `sys.path`, so the image's `agentic_os` wins over the one pre-commit installed from the pinned `rev`. Observed inside the image, from the hook's own interpreter: ``` hook venv python: /root/.cache/pre-commit/repo56i9jckq/py_env-python3/bin/python resolved agentic_os -> /opt/agentic-os/python/agentic_os/__init__.py sys.path[0:3] -> ['', '/opt/agentic-os/python', '.../python313.zip'] ``` The two copies genuinely disagree. At the pinned `v0.221.0`, `docs/FEATURES.md` takes `TRIFECTA_MAX_CHARS = 12_500`. The image's copy adds a dedicated `FEATURES_MAX_CHARS = 4_000` and applies it to that path. Same config, same file, two verdicts. ## Why it matters `rev:` is the contract that says which validator a repo is checked against. Right now it selects the code on a developer host and is ignored in CI, so every consumer's pin is advisory there. The failure mode is quiet: the hook reports a real violation, just not one the pinned rev would have flagged, and the repo owner has no local way to reproduce it. It also cuts the other way. A repo could pass CI against a newer, laxer validator while the pinned rev would have failed it. ## Possible directions - Scope the interpreter injection so it does not reach pre-commit hook venvs, for example dropping `PYTHONPATH` for the hook subprocess or shipping the CLI surface as an installed console script instead. - Or make the shadowing explicit and intended: have the image assert that a consumer's pinned `rev` matches the image's version and fail loudly on drift, rather than silently substituting. The first keeps `rev` meaningful. The second at least makes the substitution visible. ## Acceptance - A consumer repo pinned to rev X is checked against rev X's validators in the release image, or is told clearly that it is not. - The local and CI verdicts for the same tree and config agree. ## Provenance Found while landing coilyco-gaming/factory-game-v3#87. That repo took the trim rather than a cap override, since the 4000-char cap is the intended one for `docs/FEATURES.md`. This issue is the cross-repo half.

coilyco-ops referenced this issue from coilyco-gaming/factory-game-v3

2026-08-07 20:08:55 +00:00

Both CI workflows are red on main #87

coilyco-ops commented

2026-08-07 20:39:18 +00:00

Author

Member

Second case found, and it sharpens the problem: the shadowing does not just override the pinned rev, it strands fixes that have already landed here.

3a6fe4ad fixed #961 on 2026-08-06 at 20:31Z. It is on main and is genuinely correct - the affected tree reports code-comments check: OK against it. But it cannot reach any consumer's CI:

The newest plain tag v0.265.0 points at a commit from 19:34Z, before the fix.
The release, latest, and v0.265.0 images were all published at 19:53:32Z, 38 minutes before the fix.

docker/dev-base/full/Dockerfile:119 copies agentic_os into /opt/agentic-os/python/agentic_os/, and dev-base-publish triggers on pushes to release touching docker/**. A change under agentic_os/ matches neither the path filter nor any image rebuild, so a pure-Python validator fix cannot reach CI on its own. It waits for an unrelated docker/** change to carry it in.

That is the part worth naming separately from the original report. The shadowing means the image is the real source of validator truth in CI, but the image's publish trigger is not wired to the validators. So the authoritative copy is the one with no path from a fix to a release.

Demonstrated in coilyco-gaming/factory-game-v3 (its #82): pinning aos-precommit-v0.15.0, which does carry 3a6fe4ad, still fails in the release image with the identical false positive. There is no rev a consumer can pin to get the fixed behavior. That issue is now blocked on an image rebuild rather than on any repo-side change.

This does not change the suggested directions, but it does raise the stakes on the first one. As long as the image shadows the pin, agentic_os/** needs to be part of whatever triggers an image rebuild, or the validators need to stop being baked into the image at all.

Second case found, and it sharpens the problem: the shadowing does not just override the pinned rev, it strands fixes that have already landed here. `3a6fe4ad` fixed #961 on 2026-08-06 at 20:31Z. It is on `main` and is genuinely correct - the affected tree reports `code-comments check: OK` against it. But it cannot reach any consumer's CI: - The newest plain tag `v0.265.0` points at a commit from 19:34Z, before the fix. - The `release`, `latest`, and `v0.265.0` images were all published at 19:53:32Z, 38 minutes before the fix. `docker/dev-base/full/Dockerfile:119` copies `agentic_os` into `/opt/agentic-os/python/agentic_os/`, and `dev-base-publish` triggers on pushes to `release` touching `docker/**`. A change under `agentic_os/` matches neither the path filter nor any image rebuild, so a pure-Python validator fix cannot reach CI on its own. It waits for an unrelated `docker/**` change to carry it in. That is the part worth naming separately from the original report. The shadowing means the image is the real source of validator truth in CI, but the image's publish trigger is not wired to the validators. So the authoritative copy is the one with no path from a fix to a release. Demonstrated in `coilyco-gaming/factory-game-v3` (its #82): pinning `aos-precommit-v0.15.0`, which does carry `3a6fe4ad`, still fails in the release image with the identical false positive. There is no rev a consumer can pin to get the fixed behavior. That issue is now blocked on an image rebuild rather than on any repo-side change. This does not change the suggested directions, but it does raise the stakes on the first one. As long as the image shadows the pin, `agentic_os/**` needs to be part of whatever triggers an image rebuild, or the validators need to stop being baked into the image at all.

coilyco-ops referenced this issue from a commit

2026-08-10 00:40:47 +00:00

fix(dev-base): scope the embedded agentic_os to the system interpreter

coilyco-ops closed this issue

2026-08-10 00:40:47 +00:00

coilyco-ops referenced this issue

2026-08-10 00:41:57 +00:00

Stop dev-base PYTHONPATH from overriding pinned Python tools #771

coilyco-ops commented

2026-08-10 00:42:12 +00:00

Author

Member

Closed by 479924cf, which fixes the shared root cause. The full writeup, including the behavior regression and the open verification step, is on #771.

Closed by `479924cf`, which fixes the shared root cause. The full writeup, including the behavior regression and the open verification step, is on #771.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

coilyco-flight-deck/agentic-os#971

No description provided.

Rows
Columns