Flux GitOps spike over deploy/ (pull-based reconcile) #366

New issue

Open

opened 2026-06-17 22:55:54 +00:00 by coilysiren · 1 comment

coilysiren commented

2026-06-17 22:55:54 +00:00

Owner

Flux GitOps spike over `deploy/`

Move kai-server CD from push-only (a resource changes only when a Forgejo Actions
workflow fires) to pull-based reconcile, so a commit to main becomes the deploy
and drift on live resources gets reverted.

Status: Stage 1 is LIVE on kai-server (2026-06-17). Flux controllers
installed, apps Kustomization Ready: True at the live revision - the deploy
runner is now Flux-managed and the manual kubectl apply bootstrap is retired.
infrastructure ships suspended as designed.

Landed (Stage 1)

clusters/kai-server/ sync tree: read-only GitRepository against canonical
Forgejo + layered Kustomizations (infrastructure suspended, apps active).
deploy/_flux/<layer>/kustomization.yaml overlays listing existing flat
manifests by path (no file moves), validated under LoadRestrictionsNone.
Stage 1 apps adopts only forgejo-runner-deploy.yml with prune: false, so
the reconciler owns the CD runner and the manual kubectl apply bootstrap
retires.
scripts/flux-bootstrap.sh (on-box, idempotent) + docs/flux-gitops.md.
The flux-deploy token needs exactly the read:repository scope (git pull is a
pure read); the holding account also needs read access to the repo.

Follow-ups (staged rollout)

Mint a Forgejo read-only (read:repository) token, stash at SSM
/forgejo/flux-deploy-token, run scripts/flux-bootstrap.sh on kai-server.
Done - apps reconciled green.
Record the SSM key + token scope in agentic-os-kai/SSM.md.
Verify drift correction: hand-edit the live forgejo-runner-deploy
StatefulSet, confirm Flux reverts it within the interval.
Flip apps to prune: true once trusted.
Un-suspend the infrastructure layer (SecretStore, ExternalSecrets,
ExternalDNS, cert-manager issuers); add dependsOn: infrastructure to apps.
Widen deploy/_flux/apps to the remaining services one entry at a time.
Adopt gotk-components into git (controllers are flux install-managed for
now).
Adopt the Helm-installed operators (cert-manager, external-secrets) into
Flux HelmRelease objects.

Origin: infra automation review, this session.

## Flux GitOps spike over `deploy/` Move kai-server CD from push-only (a resource changes only when a Forgejo Actions workflow fires) to pull-based reconcile, so a commit to `main` becomes the deploy and drift on live resources gets reverted. **Status: Stage 1 is LIVE on kai-server (2026-06-17).** Flux controllers installed, `apps` Kustomization `Ready: True` at the live revision - the deploy runner is now Flux-managed and the manual `kubectl apply` bootstrap is retired. `infrastructure` ships suspended as designed. ### Landed (Stage 1) - `clusters/kai-server/` sync tree: read-only `GitRepository` against canonical Forgejo + layered `Kustomization`s (`infrastructure` suspended, `apps` active). - `deploy/_flux/<layer>/kustomization.yaml` overlays listing existing flat manifests by path (no file moves), validated under `LoadRestrictionsNone`. - Stage 1 `apps` adopts only `forgejo-runner-deploy.yml` with `prune: false`, so the reconciler owns the CD runner and the manual `kubectl apply` bootstrap retires. - `scripts/flux-bootstrap.sh` (on-box, idempotent) + `docs/flux-gitops.md`. - The flux-deploy token needs exactly the `read:repository` scope (git pull is a pure read); the holding account also needs read access to the repo. ### Follow-ups (staged rollout) - [x] Mint a Forgejo **read-only** (`read:repository`) token, stash at SSM `/forgejo/flux-deploy-token`, run `scripts/flux-bootstrap.sh` on kai-server. Done - `apps` reconciled green. - [ ] Record the SSM key + token scope in `agentic-os-kai/SSM.md`. - [ ] Verify drift correction: hand-edit the live `forgejo-runner-deploy` StatefulSet, confirm Flux reverts it within the interval. - [ ] Flip `apps` to `prune: true` once trusted. - [ ] Un-suspend the `infrastructure` layer (SecretStore, ExternalSecrets, ExternalDNS, cert-manager issuers); add `dependsOn: infrastructure` to apps. - [ ] Widen `deploy/_flux/apps` to the remaining services one entry at a time. - [ ] Adopt `gotk-components` into git (controllers are `flux install`-managed for now). - [ ] Adopt the Helm-installed operators (cert-manager, external-secrets) into Flux `HelmRelease` objects. Origin: infra automation review, this session.

coilysiren referenced this issue from a commit

2026-06-17 22:57:07 +00:00

feat(flux): GitOps spike over deploy/ (Stage 1, runner-only, dry)

coilysiren referenced this issue from a commit

2026-06-17 23:04:15 +00:00

fix(flux-bootstrap): derive checkout root from script path, not a hardcoded org dir

coilysiren commented

2026-06-17 23:07:25 +00:00

Author

Owner

Stage 1 live (2026-06-17). Bootstrapped on kai-server: flux 2.4.0 controllers installed and ready, flux-system git secret seeded from SSM /forgejo/flux-deploy-token, sync objects applied.

NAME              REVISION              SUSPENDED    READY    MESSAGE
apps              main@sha1:d3b8a76c    False        True     Applied revision: main@sha1:d3b8a76c
flux-system       main@sha1:d3b8a76c    False        True     Applied revision: main@sha1:d3b8a76c
infrastructure                          True         False    waiting to be reconciled (suspended, Stage 2)

The deploy runner (forgejo-runner-deploy.yml) is now Flux-managed; the manual kubectl apply bootstrap is retired. Bootstrap hit one snag - scripts/flux-bootstrap.sh hardcoded a coilysiren/ org-dir path while the checkout lives under coilyco-flight-deck/; fixed in d3b8a76 to derive the root from the script location.

Next: the drift test (hand-edit the live StatefulSet, watch Flux revert), then flip apps to prune: true and move to Stage 2.

**Stage 1 live (2026-06-17).** Bootstrapped on kai-server: flux 2.4.0 controllers installed and ready, `flux-system` git secret seeded from SSM `/forgejo/flux-deploy-token`, sync objects applied. ``` NAME REVISION SUSPENDED READY MESSAGE apps main@sha1:d3b8a76c False True Applied revision: main@sha1:d3b8a76c flux-system main@sha1:d3b8a76c False True Applied revision: main@sha1:d3b8a76c infrastructure True False waiting to be reconciled (suspended, Stage 2) ``` The deploy runner (`forgejo-runner-deploy.yml`) is now Flux-managed; the manual `kubectl apply` bootstrap is retired. Bootstrap hit one snag - `scripts/flux-bootstrap.sh` hardcoded a `coilysiren/` org-dir path while the checkout lives under `coilyco-flight-deck/`; fixed in d3b8a76 to derive the root from the script location. Next: the drift test (hand-edit the live StatefulSet, watch Flux revert), then flip `apps` to `prune: true` and move to Stage 2.

coilysiren added the

consult

labels

2026-06-18 06:22:34 +00:00