Deploy-path infra: get CI green + document local-deploy host prereqs #26

New issue

Closed

opened 2026-06-06 05:02:36 +00:00 by coilysiren · 2 comments

coilysiren commented

2026-06-06 05:02:36 +00:00

Owner

Split out of #25 (which is now scoped to the .deploy kubectl-context fix, resolved). These two deploy-path items remain:

1. CI deploy pipeline is not working

The Forgejo Actions runner did not pick up pushes 5197f8b / dcb1533 / 1cbaca9 - no new runs were created (newest run is run#14 on an old commit). Runner appears stalled/offline.
Even historically, the CI deploy job has always failed (every deploy task in actions task list is failure); only test passes.
This is the sustainable path (in-cluster, native amd64, node-local registry push - none of the Mac's arch/insecure-registry/routing friction). Worth getting green so deploys aren't manual.

2. Local-deploy host prerequisites (document or script)

For coily exec deploy to work from kais-macbook-pro, the host needs: OrbStack with 192.168.0.194:30500 in insecure-registries (persistent, set in ~/.orbstack/config/docker.json) and a healthy VM->LAN route (a clean OrbStack relaunch fixed a degraded route during the #23 cutover). Consider documenting in the machine skill or a docs/deploy.md.

Trivia

An orphaned arm64 image coilysiren-galaxy-gen:dcb1533... (wrong arch, from the pre---platform build) sits in the registry. Harmless; GC later.

Origin: galaxy-gen#23 cutover, split from #25.

Split out of #25 (which is now scoped to the `.deploy` kubectl-context fix, resolved). These two deploy-path items remain: ## 1. CI deploy pipeline is not working - The Forgejo Actions runner did **not** pick up pushes `5197f8b` / `dcb1533` / `1cbaca9` - no new runs were created (newest run is run#14 on an old commit). Runner appears stalled/offline. - Even historically, the CI `deploy` job has **always failed** (every `deploy` task in `actions task list` is `failure`); only `test` passes. - This is the sustainable path (in-cluster, native amd64, node-local registry push - none of the Mac's arch/insecure-registry/routing friction). Worth getting green so deploys aren't manual. ## 2. Local-deploy host prerequisites (document or script) For `coily exec deploy` to work from kais-macbook-pro, the host needs: OrbStack with `192.168.0.194:30500` in `insecure-registries` (persistent, set in `~/.orbstack/config/docker.json`) and a healthy VM->LAN route (a clean OrbStack relaunch fixed a degraded route during the #23 cutover). Consider documenting in the machine skill or a `docs/deploy.md`. ## Trivia - An orphaned arm64 image `coilysiren-galaxy-gen:dcb1533...` (wrong arch, from the pre-`--platform` build) sits in the registry. Harmless; GC later. Origin: galaxy-gen#23 cutover, split from #25.

coilysiren referenced this issue

2026-06-06 05:02:51 +00:00

Fix .deploy kubectl context: target LAN API endpoint, not the flaky tailnet route #25

coilysiren added the

label

2026-06-17 07:04:56 +00:00

coilysiren added

and removed

labels

2026-06-17 08:39:49 +00:00

coilyco-ops commented

2026-07-02 02:39:51 +00:00

Member

🔒 Reserved by ward agent --driver claude — container engineer-claude-galaxy-gen-26 on host kais-macbook-pro-2.local is carrying this issue (reserved 2026-07-02T02:39:48Z). Concurrent ward agent runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); --force overrides.

— Claude (she/her), via ward agent

🔒 Reserved by `ward agent --driver claude` — container `engineer-claude-galaxy-gen-26` on host `kais-macbook-pro-2.local` is carrying this issue (reserved 2026-07-02T02:39:48Z). Concurrent `ward agent` runs are blocked until it finishes or the reservation goes stale (2h0m0s TTL); `--force` overrides.  — Claude (she/her), via `ward agent`

coilysiren referenced this issue from a commit

2026-07-02 02:48:43 +00:00

fix(deploy): harden CI roll step on missing kubeconfig + document deploy prereqs

coilysiren closed this issue

2026-07-02 02:48:43 +00:00

coilysiren referenced this issue from a commit

2026-07-02 02:48:43 +00:00

Merge issue-26: harden CI deploy roll step + document deploy prereqs (closes #26)

coilysiren commented

2026-07-02 02:49:19 +00:00

Author

Owner

WARD-OUTCOME: done - hardened the CI Roll-deployment step (guard + auth preflight) and documented both deploy paths' host/infra prereqs; landed to main.

Honest retro: this issue read like "make CI green," but the interesting part was figuring out that most of it was already un-stuck. The runner is no longer offline - run #27 picked up the head commit fine, so that half of item 1 aged out on its own. What fought back was the "deploy always fails" half: I couldn't pull job logs (no ward logs verb, secrets/logs need an auth token I don't hold, and the web UI is a JS SPU that WebFetch can't render). I ended up reading the run page's embedded JSON, which handed me the smoking gun - Build-and-push succeeds in 2m16s, Roll deployment fails in 0 seconds. That 0s at the one step that needs cluster auth is the classic empty/stale DEPLOY_KUBECONFIG signature, and the repo moved orgs (coilysiren -> coilyco-flight-deck), which is exactly when a per-repo Actions secret evaporates.

So: I can't actually turn the pipeline green from here - that needs the DEPLOY_KUBECONFIG secret re-set on this repo, an out-of-band cluster credential the container can't mint. What I could do, I did: the Roll step now guards for an empty secret and preflights auth against the target Deployment (a verb the deployer SA holds) so the failure names its cause instead of the old cryptic 0-second exit. And item 2 is fully done - docs/deploy.md (CI path + a secret-rebuild recipe) and docs/deploy-local.md (OrbStack insecure-registries, VM->LAN route, --platform amd64), linked from README/AGENTS/FEATURES, plus a fix to the stale "pull-side, not pushed from CI" README line.

Confidence: high on the root cause and the docs; the diagnostic hardening is safe either way. The one thing I couldn't verify is whether the secret is truly missing vs present-but-stale - both point to the same human fix. Follow-up worth filing: re-set DEPLOY_KUBECONFIG on coilyco-flight-deck/galaxy-gen (recipe in docs/deploy.md), then confirm the next push goes green. The orphaned arm64 registry image is still there to GC whenever.

WARD-OUTCOME: done - hardened the CI Roll-deployment step (guard + auth preflight) and documented both deploy paths' host/infra prereqs; landed to main. Honest retro: this issue read like "make CI green," but the interesting part was figuring out that most of it was already un-stuck. The runner is no longer offline - run #27 picked up the head commit fine, so that half of item 1 aged out on its own. What fought back was the "deploy always fails" half: I couldn't pull job logs (no ward logs verb, secrets/logs need an auth token I don't hold, and the web UI is a JS SPU that WebFetch can't render). I ended up reading the run page's embedded JSON, which handed me the smoking gun - Build-and-push succeeds in 2m16s, Roll deployment fails in 0 seconds. That 0s at the one step that needs cluster auth is the classic empty/stale `DEPLOY_KUBECONFIG` signature, and the repo moved orgs (coilysiren -> coilyco-flight-deck), which is exactly when a per-repo Actions secret evaporates. So: I can't actually turn the pipeline green from here - that needs the `DEPLOY_KUBECONFIG` secret re-set on this repo, an out-of-band cluster credential the container can't mint. What I could do, I did: the Roll step now guards for an empty secret and preflights auth against the target Deployment (a verb the deployer SA holds) so the failure names its cause instead of the old cryptic 0-second exit. And item 2 is fully done - docs/deploy.md (CI path + a secret-rebuild recipe) and docs/deploy-local.md (OrbStack insecure-registries, VM->LAN route, --platform amd64), linked from README/AGENTS/FEATURES, plus a fix to the stale "pull-side, not pushed from CI" README line. Confidence: high on the root cause and the docs; the diagnostic hardening is safe either way. The one thing I couldn't verify is whether the secret is truly missing vs present-but-stale - both point to the same human fix. Follow-up worth filing: re-set DEPLOY_KUBECONFIG on coilyco-flight-deck/galaxy-gen (recipe in docs/deploy.md), then confirm the next push goes green. The orphaned arm64 registry image is still there to GC whenever.