interactive: verify scoped Forgejo runner recycle #689

New issue

Open

opened 2026-07-29 12:56:47 +00:00 by coilyco-ops · 0 comments

coilyco-ops commented

2026-07-29 12:56:47 +00:00

Member

Live verification checkpoint

The repository change from infrastructure#587 is on main at 59acdd65ee8cb22e9b801ed4a0f6b4d99069b8f2. The sealed engineer verified the Flux render, exact RBAC targets, scratch-wipe contracts, digest-pinned OCI entrypoint, pre-commit suite, and Forgejo CI. Live cluster evidence remains operator-owned.

After confirming the kubectl write, an operator should:

Confirm Flux applies deploy/forgejo-runner-recycle.yml and the CronJob template names forgejo-runner-flight-deck, forgejo-runner-bridge, forgejo-runner-gaming, and forgejo-runner-build-flight-deck.
Record the current pod UIDs for those four StatefulSets, then create one manual Job from cronjob/forgejo-runner-recycle.
Wait for that Job to complete and for all four StatefulSet rollouts to return Ready within 15 minutes.
Confirm every replacement pod ran init-wipe-docker-lib successfully. The three general pools must receive new docker-lib emptyDir volumes. The image-build pool must remount its docker-lib PVC and log the wipe before DinD starts.
Confirm no Job pod reports StartError. Preserve the Job name, pod UIDs before and after, rollout results, wipe-init evidence, and the read-only disk report.
After the next 09:15 UTC schedule, repeat the Job, rollout, Ready, and no-StartError checks without creating another manual Job.

Do not delete PVCs or application data. The recycle intentionally retains /data, runner registration state younger than 30 days, Forgejo artifacts and logs, package-upload scratch, Forgejo data, and PostgreSQL data.

Signed: Codex

## Live verification checkpoint The repository change from infrastructure#587 is on `main` at `59acdd65ee8cb22e9b801ed4a0f6b4d99069b8f2`. The sealed engineer verified the Flux render, exact RBAC targets, scratch-wipe contracts, digest-pinned OCI entrypoint, pre-commit suite, and Forgejo CI. Live cluster evidence remains operator-owned. After confirming the kubectl write, an operator should: 1. Confirm Flux applies `deploy/forgejo-runner-recycle.yml` and the CronJob template names `forgejo-runner-flight-deck`, `forgejo-runner-bridge`, `forgejo-runner-gaming`, and `forgejo-runner-build-flight-deck`. 2. Record the current pod UIDs for those four StatefulSets, then create one manual Job from `cronjob/forgejo-runner-recycle`. 3. Wait for that Job to complete and for all four StatefulSet rollouts to return Ready within 15 minutes. 4. Confirm every replacement pod ran `init-wipe-docker-lib` successfully. The three general pools must receive new `docker-lib` `emptyDir` volumes. The image-build pool must remount its `docker-lib` PVC and log the wipe before DinD starts. 5. Confirm no Job pod reports `StartError`. Preserve the Job name, pod UIDs before and after, rollout results, wipe-init evidence, and the read-only disk report. 6. After the next 09:15 UTC schedule, repeat the Job, rollout, Ready, and no-`StartError` checks without creating another manual Job. Do not delete PVCs or application data. The recycle intentionally retains `/data`, runner registration state younger than 30 days, Forgejo artifacts and logs, package-upload scratch, Forgejo data, and PostgreSQL data. Related to coilyco-flight-deck/infrastructure#587. Signed: Codex

coilyco-ops added the

autonomy

live-collab

label

2026-07-29 12:56:47 +00:00

coilyco-ops referenced this issue

2026-07-29 12:57:57 +00:00

fix: make the Forgejo runner recycle CronJob executable #587

coilyco-ops added the

priority

label

2026-08-05 10:25:58 +00:00