CI red on main is environmental, not #169: sandbox seccomp tests + golangci-lint install both fail on the runner #170

New issue

Closed

opened 2026-06-25 21:39:41 +00:00 by coilysiren · 2 comments

coilysiren commented

2026-06-25 21:39:41 +00:00

Owner

TL;DR

cli-guard main CI has been structurally red since the canonical gate landed (#165), and it is not caused by #169 (the override engine). #169's code is clean: go build, go vet, and golangci-lint run ./... (v2.12.2) all pass at cc4f8f7. The red is two environmental failures that also failed on the pre-#169 runs 224 and 226:

test job - 3 cli/sandbox security tests fail because the Forgejo docker runner can't enforce the sandbox's jail + seccomp filter.
lint job - fails in ~14s, before a lint run could finish (and local lint = 0 issues), so it is the golangci-lint install step failing on the runner, not a code violation.

Diagnosed from a fresh clone at cc4f8f7; the 3 sandbox failures reproduce on any box that can't enforce seccomp.

Cause 1: `test` job - sandbox security tests can't enforce on the runner

go test -race ./... fails at cli/sandbox:

--- FAIL: TestSetupDenied  (sandbox_degrade_linux_test.go:43: precondition: Wrap should have jailed the cmd)
--- FAIL: TestSecurityClaim_GrandchildRoutesThroughGate (name/abspath grandchild not rerouted to the gate)
--- FAIL: TestSecurityClaim_SeccompDeniesPtrace (ptrace probe exited non-zero: seccomp denylist not enforced)

The sandbox (cli/sandbox/seccomp_linux.go, sandbox_linux.go) jails via unprivileged user namespaces + PR_SET_NO_NEW_PRIVS + a seccomp(2) filter + CAP_SYS_ADMIN-in-userns bind-mounts. The tests run past their existing t.Skipf("unprivileged namespaces unavailable") guards (so userns is available), but the default Docker seccomp profile on the job container blocks the unshare/mount/seccomp syscalls the jail needs, so the enforcement never bites and the security claims fail.

Fix (no dind, no sidecar)

Make the runner capable with one container option, plus a skip-guard safety net:

ci.yml test job: run its container with --security-opt seccomp=unconfined (via container.options) so the in-container test can create its userns, bind-mount its jail, and install its own seccomp filter. The tests then run for real and guard the boundary - the "capable runner" outcome, with zero nested Docker. (If a cap is also needed, --cap-add=SYS_ADMIN; evidence says the seccomp profile is the blocker, userns already works.)
Skip-guard the 3 tests: extend the existing t.Skipf precedent (sandbox_linux_test.go:102,133) so these three detect missing enforcement capability and skip (not fail) anywhere it is genuinely absent - local dev, a locked-down runner. Green everywhere, actually-tested on CI.

Dependency to confirm: the Forgejo Actions runner must permit a workflow to set container options/security-opt. If yes -> one-line ci.yml change. If the runner forbids it -> a small host-side runner-config change to allow it (still not dind/sidecar); until then the skip-guard keeps CI green (degraded: the sandbox claims skip rather than run). Do NOT reach for docker-in-docker or a privileged sidecar - the security-opt on the existing runner is the whole fix (Kai).

Cause 2: `lint` job - golangci-lint install fails on the runner

The lint job fails in ~14s (id=1972, run 228), far too fast for a real lint pass, and golangci-lint run ./... (v2.12.2, the pinned CI version) reports 0 issues locally at cc4f8f7. So the failure is the install step, not the code:

- name: Install golangci-lint
  run: curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b /usr/local/bin "${GOLANGCI_LINT_VERSION}"

The runner reaches go.dev (the test job's Go install succeeds) but the lint install pulls from raw.githubusercontent.com, which appears unreachable/blocked from the runner.

Fix

Install golangci-lint from a source the runner already reaches - e.g. go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@$(GOLANGCI_LINT_VERSION) via the module proxy (works locally, confirmed), or vendor the install via the Go toolchain / a mirror, with a retry. Confirm the exact failure from the lint job log (run 228, id 1972) before settling the source.

Why this matters beyond cli-guard

Every recent headless agent run (#169, ward#338, ward#347) retro-noted that golangci-lint / trufflehog / kdlfmt are absent in the dev-base container, so agents cannot self-run the lint gate and keep landing code the canonical CI must catch. That is an agentic-os dev-base image gap worth a sibling issue - the in-container pre-commit tooling should match the CI gate so an agent can verify before it pushes. (Out of scope for this issue; noting the upstream root cause.)

Scope / dispatch

Two independent fixes (sandbox test job + lint install) - splittable into two runs or one. Not auto-dispatched (Kai dispatches; this touches the security gate + CI infra). A fix run can verify the sandbox change in-container (tests skip/pass) and the lint change on push; the security-opt's real effect needs a push to the actual runner to confirm.

References

cli-guard#169 (the override engine wrongly suspected; it is clean), cli-guard#165 (the canonical CI gate this fixes), cli/sandbox/seccomp_linux.go + sandbox_linux.go (the jail), .forgejo/workflows/ci.yml (the gate). Diagnosed from a fresh clone at cc4f8f7, golangci-lint v2.12.2, reproduced sandbox failures locally.

## TL;DR cli-guard `main` CI has been **structurally red since the canonical gate landed (#165)**, and it is **not** caused by #169 (the override engine). #169's code is clean: `go build`, `go vet`, and `golangci-lint run ./...` (v2.12.2) all pass at `cc4f8f7`. The red is two **environmental** failures that also failed on the pre-#169 runs 224 and 226: - **`test` job** - 3 `cli/sandbox` security tests fail because the Forgejo `docker` runner can't enforce the sandbox's jail + seccomp filter. - **`lint` job** - fails in ~14s, before a lint run could finish (and local lint = 0 issues), so it is the **golangci-lint install step** failing on the runner, not a code violation. Diagnosed from a fresh clone at `cc4f8f7`; the 3 sandbox failures reproduce on any box that can't enforce seccomp. ## Cause 1: `test` job - sandbox security tests can't enforce on the runner `go test -race ./...` fails at `cli/sandbox`: ``` --- FAIL: TestSetupDenied (sandbox_degrade_linux_test.go:43: precondition: Wrap should have jailed the cmd) --- FAIL: TestSecurityClaim_GrandchildRoutesThroughGate (name/abspath grandchild not rerouted to the gate) --- FAIL: TestSecurityClaim_SeccompDeniesPtrace (ptrace probe exited non-zero: seccomp denylist not enforced) ``` The sandbox (`cli/sandbox/seccomp_linux.go`, `sandbox_linux.go`) jails via **unprivileged user namespaces + `PR_SET_NO_NEW_PRIVS` + a `seccomp(2)` filter + CAP_SYS_ADMIN-in-userns bind-mounts**. The tests run past their existing `t.Skipf("unprivileged namespaces unavailable")` guards (so userns is available), but the **default Docker seccomp profile** on the job container blocks the `unshare`/`mount`/`seccomp` syscalls the jail needs, so the enforcement never bites and the security claims fail. ### Fix (no dind, no sidecar) Make the runner capable with **one container option**, plus a skip-guard safety net: 1. **`ci.yml` test job**: run its container with `--security-opt seccomp=unconfined` (via `container.options`) so the in-container test can create its userns, bind-mount its jail, and install its own seccomp filter. The tests then run for real and guard the boundary - the "capable runner" outcome, with zero nested Docker. (If a cap is also needed, `--cap-add=SYS_ADMIN`; evidence says the seccomp profile is the blocker, userns already works.) 2. **Skip-guard the 3 tests**: extend the existing `t.Skipf` precedent (`sandbox_linux_test.go:102,133`) so these three **detect missing enforcement capability and skip** (not fail) anywhere it is genuinely absent - local dev, a locked-down runner. Green everywhere, actually-tested on CI. **Dependency to confirm:** the Forgejo Actions runner must permit a workflow to set container `options`/security-opt. If yes -> one-line `ci.yml` change. If the runner forbids it -> a small host-side runner-config change to allow it (still not dind/sidecar); until then the skip-guard keeps CI green (degraded: the sandbox claims skip rather than run). **Do NOT reach for docker-in-docker or a privileged sidecar - the security-opt on the existing runner is the whole fix (Kai).** ## Cause 2: `lint` job - golangci-lint install fails on the runner The `lint` job fails in ~14s (`id=1972`, run 228), far too fast for a real lint pass, and `golangci-lint run ./...` (v2.12.2, the pinned CI version) reports **0 issues** locally at `cc4f8f7`. So the failure is the **install step**, not the code: ```yaml - name: Install golangci-lint run: curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b /usr/local/bin "${GOLANGCI_LINT_VERSION}" ``` The runner reaches `go.dev` (the `test` job's Go install succeeds) but the lint install pulls from `raw.githubusercontent.com`, which appears unreachable/blocked from the runner. ### Fix Install golangci-lint from a source the runner already reaches - e.g. `go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@$(GOLANGCI_LINT_VERSION)` via the module proxy (works locally, confirmed), or vendor the install via the Go toolchain / a mirror, with a retry. Confirm the exact failure from the `lint` job log (run 228, id 1972) before settling the source. ## Why this matters beyond cli-guard Every recent headless agent run (#169, ward#338, ward#347) retro-noted that **golangci-lint / trufflehog / kdlfmt are absent in the dev-base container**, so agents cannot self-run the lint gate and keep landing code the canonical CI must catch. That is an agentic-os dev-base image gap worth a sibling issue - the in-container pre-commit tooling should match the CI gate so an agent can verify before it pushes. (Out of scope for this issue; noting the upstream root cause.) ## Scope / dispatch Two independent fixes (sandbox test job + lint install) - splittable into two runs or one. **Not auto-dispatched** (Kai dispatches; this touches the security gate + CI infra). A fix run can verify the sandbox change in-container (tests skip/pass) and the lint change on push; the security-opt's real effect needs a push to the actual runner to confirm. ## References cli-guard#169 (the override engine wrongly suspected; it is clean), cli-guard#165 (the canonical CI gate this fixes), `cli/sandbox/seccomp_linux.go` + `sandbox_linux.go` (the jail), `.forgejo/workflows/ci.yml` (the gate). Diagnosed from a fresh clone at `cc4f8f7`, golangci-lint v2.12.2, reproduced sandbox failures locally.

coilysiren commented

2026-06-25 21:44:22 +00:00

Author

Owner

Ordering: the sandbox-test fix here has a prerequisite in the infra repo - infrastructure#401 (let the forgejo runner honor a per-job --security-opt seccomp=unconfined). That lands FIRST (or confirms workflow options are already honored, in which case this ci.yml change suffices alone). Dispatch infrastructure#401 before this. Note: infrastructure Flux-auto-deploys to the live fleet runner, so that one wants a supervised dispatch, not fire-and-forget.

coilysiren commented

2026-06-26 00:23:56 +00:00

Author

Owner

Upstream root cause of the lint-job red filed: agentic-os#292 - bake golangci-lint/trufflehog/kdlfmt into the dev-base image so agents can self-run the gate (the lint job re-installs golangci-lint from githubusercontent precisely because the agent container never had it). Fixing #292 stops the recurring tax across every repo; this issue (#170) still fixes the cli-guard-side runner/test specifics.

coilysiren added the

consult

labels

2026-07-01 04:29:32 +00:00

coilysiren closed this issue

2026-07-06 20:35:56 +00:00