coilyco-flight-deck/infrastructure
2
0
Fork 0
Code Issues 118 Pull requests Projects Releases Packages Wiki Activity Actions 2

Design ser8 as the primary ops plane, not only the o11y host #506

New issue
Open
opened 2026-07-09 21:29:58 +00:00 by coilyco-ops · 2 comments
coilyco-ops commented 2026-07-09 21:29:58 +00:00
Member
Copy link

Kai called out that moving only the old kai-server observability namespace underuses ser8 dramatically. ser8 has roughly 1.86x kai-server's RAM and is already documented as the ops/o11y plane, but the current cleanup threads are still piecemeal.

Current context:

  • kai-server is the app plane: Forgejo, Eco, Open WebUI, MCPs, public app services, game-adjacent workloads.
  • ser8 is the independent k3s ops plane and warm standby: current docs say o11y belongs there.
  • coilyco-flight-deck/infrastructure#505 now tracks retiring the old kai-server observability namespace after ser8 coverage is verified.
  • coilyco-bridge/deploy#51 tracks migrating glitchtip/signoz/vmsingle deploy ownership.
  • coilyco-bridge/deploy#114 tracks splitting the old shared observability ownership bucket.

Please produce a practical placement plan for ser8 as the primary ops plane, beyond just moving the o11y namespace.

Classify current and near-term workloads into:

  • App plane stays on kai-server - user-facing apps and workloads that need kai-server locality.
  • Ops plane should move to ser8 - observability storage/query, alerting, Gatus/fleet reachability, CI/deploy runners where sensible, control-plane dashboards, issue/status mirrors, background fleet automation, backup/prune receivers, and other meta-services.
  • Keep split / external vantage - anything that must observe kai-server from outside it, or must stay local to a producer.
  • Retire instead of move - legacy services that should disappear rather than consume ser8.

For each candidate, include:

  • Why ser8 is or is not the right home.
  • Memory/disk/network implications.
  • Dependency or data migration needs.
  • Suggested issue follow-ups in the right repo (infrastructure vs deploy).
  • A safe order that improves reliability without making the ops plane a single fragile blob.

Acceptance: the result is an actionable roadmap with concrete follow-up issues, not just "move observability."

Kai called out that moving only the old kai-server `observability` namespace underuses ser8 dramatically. ser8 has roughly 1.86x kai-server's RAM and is already documented as the ops/o11y plane, but the current cleanup threads are still piecemeal. Current context: * kai-server is the app plane: Forgejo, Eco, Open WebUI, MCPs, public app services, game-adjacent workloads. * ser8 is the independent k3s ops plane and warm standby: current docs say o11y belongs there. * `coilyco-flight-deck/infrastructure#505` now tracks retiring the old kai-server observability namespace after ser8 coverage is verified. * `coilyco-bridge/deploy#51` tracks migrating glitchtip/signoz/vmsingle deploy ownership. * `coilyco-bridge/deploy#114` tracks splitting the old shared observability ownership bucket. Please produce a practical placement plan for ser8 as the primary ops plane, beyond just moving the o11y namespace. Classify current and near-term workloads into: * **App plane stays on kai-server** - user-facing apps and workloads that need kai-server locality. * **Ops plane should move to ser8** - observability storage/query, alerting, Gatus/fleet reachability, CI/deploy runners where sensible, control-plane dashboards, issue/status mirrors, background fleet automation, backup/prune receivers, and other meta-services. * **Keep split / external vantage** - anything that must observe kai-server from outside it, or must stay local to a producer. * **Retire instead of move** - legacy services that should disappear rather than consume ser8. For each candidate, include: * Why ser8 is or is not the right home. * Memory/disk/network implications. * Dependency or data migration needs. * Suggested issue follow-ups in the right repo (`infrastructure` vs `deploy`). * A safe order that improves reliability without making the ops plane a single fragile blob. Acceptance: the result is an actionable roadmap with concrete follow-up issues, not just "move observability."
coilyco-ops commented 2026-07-09 21:34:46 +00:00
Author
Member
Copy link

🔎 ward agent advisor

ward agent advisor --harness codex ran a one-shot standard research pass on this question:

Read the issue title, body, and comment thread below as the research brief. Identify the decisions, options, open questions, and risks, then give a clear advisory recommendation that can be posted back on the issue.

Recommendation

Treat ser8 as the primary ops plane, but do it as a layered placement program, not a single "move observability" event.

The repo evidence already points that way:

  • docs/FEATURES.md describes ser8 as the independent ops-plane cluster and warm standby.
  • coilyco-bridge/deploy/docs/o11y.md says new observability signals belong on ser8, and kai-server's observability namespace is the retiring stack.
  • Infra already places several read-only ops timers on ser8: git-state-watch, flux-readiness-watch, fleet-converge, agentic-os-pin-converge, plus the host-level restic receiver/prune flow.
  • infrastructure#314 already tracks moving CI runners off kai-server, which is the other major ops-plane workload.

The right design is:

  • kai-server stays the app plane for user-facing apps, game workloads, and producer-local collectors.
  • ser8 becomes the default home for shared operator services: observability backends, alerting, CI/deploy runners, issue/status mirrors, and background fleet automation.
  • Some things must stay split so the ops plane can still observe kai-server from outside it.
  • Some legacy services should be removed, not re-homed.

Classification

App plane stays on kai-server

  • Public and user-facing apps - Forgejo, backend/public apps, Open WebUI, game workloads, and any service whose value is serving end users from the current app cluster should stay on kai-server. Why: they are the app plane by definition, and moving them would dilute the plane split. Implications: no direct ser8 RAM win, but it keeps failure domains clean. Follow-up: none from this issue.
  • Producer-local host services - thermal-heartbeat, process-memory-heartbeat, agent-health-heartbeat, and the kai-server-side node-stats-mcp surface should stay local to the producer host. Why: they read local /proc, sensors, journals, GPU state, or host namespaces. Implications: tiny RAM/disk, but they must be repointed to ser8 sinks where applicable. Follow-up: coilyco-flight-deck/infrastructure issue to repoint every local producer to ser8 endpoints and document any intentional drops.
  • Local-only operational conveniences that read kai-server state directly - personal-dashboard is closer to "host-local dashboard" than shared ops plane. Keep it on kai-server unless a replacement dashboard is deliberately designed on ser8. Implications: low resource cost, but do not expand it as a ser8 dependency by accident. Follow-up: none now.

Ops plane should move to ser8

  • VictoriaMetrics / vmagent / vmalert / SigNoz - This is the clearest move. Why: deploy docs already define ser8 as the canonical metrics, traces, logs, and alerting plane. Implications: largest RAM and disk win on kai-server, plus ser8 already has sizing for it (57G RAM class, 857G free disk noted in ser8-k3s.md). Dependencies: producer repoints first, preserve-or-waive historical data explicitly, then remove kai-server replicas. Follow-up: continue coilyco-flight-deck/infrastructure#505 plus the deploy-owned per-stack work under coilyco-bridge/deploy#51.
  • GlitchTip - Should move to ser8 as the error pane, but not in the same step as VM/SigNoz retirement unless the storage and DSN cutover are ready. Why: deploy/docs/o11y.md explicitly says GlitchTip is the destination and tracks the ser8 move separately. Implications: moderate RAM, persistent Postgres/Valkey disk, and backup planning matter more than CPU. Dependencies: CloudNativePG/operator prerequisites on ser8, backup plan, app DSN cutover sequencing. Follow-up: coilyco-bridge/deploy issue for the service move if deploy#51 does not already cover GlitchTip runtime placement in enough detail.
  • Forgejo CI/build/deploy runners - forgejo-runner, forgejo-runner-deploy, and forgejo-runner-tap-writer belong on ser8. Why: infrastructure#314 already identifies them as ops-plane workloads and moving them removes kai-server restart blast radius from CI. Implications: moderate RAM/ephemeral disk use, plus cross-cluster registry/network design needs care. Dependencies: resolve runner-to-registry path and any host-executor assumptions before moving. Follow-up: coilyco-flight-deck/infrastructure#314 is the right umbrella.
  • Issue/status mirrors and operator-facing readers - forgejo-issues and likely signoz-mcp should live on ser8. Why: they are meta-services over the fleet, not product apps. forgejo-issues is a pure ops surface (Windmill + Postgres + NocoDB), and signoz-mcp reads ser8 SigNoz anyway, so keeping it on kai-server adds an unnecessary hop and ties ops visibility to the app plane. Implications: light-to-moderate RAM, persistent disk for forgejo-issues, negligible disk for signoz-mcp, mostly tailnet/internal traffic. Dependencies: service manifests live in coilyco-bridge/deploy, so this is deploy-owned placement work. Follow-up: coilyco-bridge/deploy issues for forgejo-issues placement on ser8 and signoz-mcp placement on ser8.
  • Background fleet automation and bot services - coilyco-ops-summon should move to ser8, alongside existing ser8 timers like git-state-watch, fleet-converge, flux-readiness-watch, and agentic-os-pin-converge. Why: these are operator workflows, and several already use the "run on ser8, probe kai-server" pattern. Implications: low RAM/disk, mostly API/network traffic. Dependencies: none beyond systemd/ansible rollout. Follow-up: coilyco-flight-deck/infrastructure issue to move remaining bot/automation timers from kai-server to the ser8 ops host.
  • Issue-corpus runtime pieces - the repo already treats issue-corpus as an ops-plane anchor on ser8, and the unresolved runtime runner should follow the same rule. Why: it is discovery/automation infra, not an app workload. Implications: mostly runner/tooling cost, not service cost. Dependencies: dedicated runner provisioning from docs/issue-corpus-rollout-runner.md. Follow-up: coilyco-flight-deck/infrastructure issue if the dormant runner work is revived.
  • Secret-management support for long-lived ser8 services - not a workload, but likely a prerequisite. ser8 currently lacks external-secrets, so more moved services means more imperative secrets. Recommendation: if more than one or two persistent services move, add a tracked decision on whether ser8 should also run external-secrets. Follow-up: coilyco-flight-deck/infrastructure design issue.

Keep split / external vantage

  • Fleet reachability / Gatus - keep the current per-cluster and per-host split. Why: reachability must be measured from multiple vantage points, not centralized into one cluster. Implications: low RAM, almost no disk, valuable cross-host network signal. Dependencies: none. Follow-up: continue infrastructure#232 / current rollout path, but do not "move Gatus to ser8 only".
  • ser8 watchtower - keep it host-level on ser8, outside k3s. Why: it is explicitly there so one sick cluster can still observe the other. Implications: tiny cost, high resilience value. Follow-up: none.
  • Node-local MCPs - node-stats-mcp should stay per-node (kai-server and ser8) because it reads node-local state. Why: centralizing it would lose the host-local contract. Implications: tiny footprint, but keep both surfaces. Follow-up: none.
  • Hosted Sentry during the cutover - stays external until each app swaps its DSN to GlitchTip. Why: it is the current error sink for some apps. Implications: no homelab resource cost, but clear migration bookkeeping is required. Follow-up: coilyco-flight-deck/infrastructure#395 remains the right tracker.

Retire instead of move

  • kai-server Grafana / old shared observability residue - docs already call Grafana retired on the ser8 side and kai-server observability legacy. Do not spend design energy relocating dead panes. Follow-up: bundle the final deletion with infrastructure#505 once verified absent or unused.
  • Legacy kai-server VM/SigNoz surfaces after cutover - tailscale proxies, NodePorts, and ward verbs that recreate the old stack should be deleted once producers are repointed. Why: leaving them around makes accidental re-creation easy. Follow-up: coilyco-flight-deck/infrastructure#505.
  • Old mirror timers rather than porting them to ser8 - docs/FEATURES.md already marks the current Forgejo<->GitHub push --mirror timers as being replaced by the safer mirror-sync path. Recommendation: do not migrate the old timers to ser8 as a long-term pattern. Retire them once the safer replacement is live. Follow-up: coilyco-flight-deck/infrastructure under the existing mirror-sync work, not a new ser8-move issue.
  • coily-audit-dashboard - explicitly called out as kai-server coily-retention only. Recommendation: retire with the remaining kai-server coily dependency rather than moving it. Follow-up: tie to the existing coily-retention/gaming retirement work.

Safe order

Use this order to improve reliability without turning ser8 into one fragile blob:

  1. Keep the independent vantage points intact first. Do not centralize Gatus, watchtower, or node-local collectors. Those are the guardrails for everything else.
  2. Finish the ser8 observability sinks and producer repoints. That means ser8 VM/SigNoz are authoritative, and every kai-server local producer either points there or is explicitly dropped.
  3. Evacuate runners from kai-server to ser8. This reduces restart blast radius and matches the ops-plane role, but it should happen only once the registry/network path is proven.
  4. Move lightweight stateless operator services next. coilyco-ops-summon, signoz-mcp, and similar readers/bots are low-risk wins that reinforce ser8 as the operator home.
  5. Move moderate stateful operator services after that. forgejo-issues is a good example: useful on the ops plane, but it introduces its own Postgres/Windmill persistence and should follow the earlier stateless moves.
  6. Move GlitchTip last among the big panes. It is stateful and coupled to DSN cutover and backup policy, so it should not be bundled into the first retirement wave.
  7. Then delete the old kai-server observability residue and retire legacy timers instead of re-homing them.

Suggested follow-up issues

These look like the concrete child tracks the roadmap needs:

  • coilyco-flight-deck/infrastructure - "Repoint kai-server local telemetry producers to ser8 and delete kai-server observability bridge surfaces". Scope: thermal-heartbeat, process-memory-heartbeat, agent-health-heartbeat, NodePorts, tailscale proxies, ward verbs, docs.
  • coilyco-flight-deck/infrastructure - "Evacuate Forgejo runners from kai-server to the ser8 cluster". This already exists as #314; it should be treated as part of the primary-ops-plane program.
  • coilyco-flight-deck/infrastructure - "Move remaining bot/automation timers from kai-server to the ser8 ops host". Scope: coilyco-ops-summon first, plus any other read-only background automation still living on kai-server.
  • coilyco-flight-deck/infrastructure - "Decide whether ser8 needs external-secrets for long-lived ops services". This becomes more important as more persistent services move.
  • coilyco-bridge/deploy - "Place signoz-mcp on ser8 instead of kai-server". It is an ops reader for ser8's trace/log plane.
  • coilyco-bridge/deploy - "Place forgejo-issues on ser8 as an ops-plane mirror". Scope: node placement, storage, tailnet access, and any secret-handling implications.
  • coilyco-bridge/deploy - continue the existing per-stack observability ownership and placement work under deploy#51, with GlitchTip explicitly called out as the last heavy stateful move.

Open questions

  • Runner network path - if runners move to ser8, is the current in-cluster registry on kai-server still the right image path, or does that become a cross-cluster bottleneck?
  • ser8 secret strategy - does ser8 stay on imperative secrets, or is external-secrets worth installing there once more ops services land?
  • GlitchTip storage and backups - what is the explicit backup/restore plan once its Postgres/Valkey move with the error pane?
  • Which control-plane UIs are worth preserving at all - personal-dashboard may be worth keeping local, while coily-audit-dashboard looks like pure retirement material.

Risks

  • Making ser8 the only vantage point would be a mistake. Keep Gatus split and keep watchtower outside k3s.
  • Moving all stateful services at once would create a large shared failure window. VM/SigNoz, runners, forgejo-issues, and GlitchTip should be separate waves.
  • Leaving kai-server bridge surfaces behind creates drift and accidental resurrection risk.
  • Secret-management toil on ser8 grows quickly if more stateful services move there without a deliberate secret strategy.
  • Runner move before registry/network design is settled could trade one blast radius problem for a cross-cluster CI outage.

Bottom line

The actionable plan is:

  • Make ser8 the default home for shared ops services.
  • Keep user-facing apps and producer-local collectors on kai-server.
  • Keep external vantage and per-node observers split.
  • Retire dead legacy panes and old mirror timers instead of porting them.
  • Sequence the work as repoint sinks -> move runners -> move lightweight ops readers/bots -> move heavier stateful ops services -> delete old kai-server residue.

That is the practical shape that uses ser8 as the primary ops plane without collapsing everything into one new single point of failure.

Researched and posted automatically by ward agent advisor --harness codex (ward#179). This is one-shot research, not a carried change - verify before acting on it.

— Codex, via ward agent

### 🔎 ward agent advisor `ward agent advisor --harness codex` ran a one-shot **standard** research pass on this question: > Read the issue title, body, and comment thread below as the research brief. Identify the decisions, options, open questions, and risks, then give a clear advisory recommendation that can be posted back on the issue. --- ## Recommendation Treat `ser8` as the **primary ops plane**, but do it as a **layered placement program**, not a single "move observability" event. The repo evidence already points that way: - `docs/FEATURES.md` describes `ser8` as the independent ops-plane cluster and warm standby. - `coilyco-bridge/deploy/docs/o11y.md` says new observability signals belong on `ser8`, and kai-server's `observability` namespace is the retiring stack. - Infra already places several read-only ops timers on `ser8`: `git-state-watch`, `flux-readiness-watch`, `fleet-converge`, `agentic-os-pin-converge`, plus the host-level `restic` receiver/prune flow. - `infrastructure#314` already tracks moving CI runners off kai-server, which is the other major ops-plane workload. The right design is: - **kai-server stays the app plane** for user-facing apps, game workloads, and producer-local collectors. - **ser8 becomes the default home for shared operator services**: observability backends, alerting, CI/deploy runners, issue/status mirrors, and background fleet automation. - **Some things must stay split** so the ops plane can still observe kai-server from outside it. - **Some legacy services should be removed, not re-homed**. ## Classification ### App plane stays on kai-server - **Public and user-facing apps** - Forgejo, backend/public apps, Open WebUI, game workloads, and any service whose value is serving end users from the current app cluster should stay on kai-server. Why: they are the app plane by definition, and moving them would dilute the plane split. Implications: no direct ser8 RAM win, but it keeps failure domains clean. Follow-up: none from this issue. - **Producer-local host services** - `thermal-heartbeat`, `process-memory-heartbeat`, `agent-health-heartbeat`, and the kai-server-side `node-stats-mcp` surface should stay local to the producer host. Why: they read local `/proc`, sensors, journals, GPU state, or host namespaces. Implications: tiny RAM/disk, but they must be **repointed** to ser8 sinks where applicable. Follow-up: `coilyco-flight-deck/infrastructure` issue to repoint every local producer to ser8 endpoints and document any intentional drops. - **Local-only operational conveniences that read kai-server state directly** - `personal-dashboard` is closer to "host-local dashboard" than shared ops plane. Keep it on kai-server unless a replacement dashboard is deliberately designed on ser8. Implications: low resource cost, but do not expand it as a ser8 dependency by accident. Follow-up: none now. ### Ops plane should move to ser8 - **VictoriaMetrics / vmagent / vmalert / SigNoz** - This is the clearest move. Why: deploy docs already define ser8 as the canonical metrics, traces, logs, and alerting plane. Implications: largest RAM and disk win on kai-server, plus ser8 already has sizing for it (`57G` RAM class, `857G` free disk noted in `ser8-k3s.md`). Dependencies: producer repoints first, preserve-or-waive historical data explicitly, then remove kai-server replicas. Follow-up: continue `coilyco-flight-deck/infrastructure#505` plus the deploy-owned per-stack work under `coilyco-bridge/deploy#51`. - **GlitchTip** - Should move to ser8 as the error pane, but not in the same step as VM/SigNoz retirement unless the storage and DSN cutover are ready. Why: `deploy/docs/o11y.md` explicitly says GlitchTip is the destination and tracks the ser8 move separately. Implications: moderate RAM, persistent Postgres/Valkey disk, and backup planning matter more than CPU. Dependencies: CloudNativePG/operator prerequisites on ser8, backup plan, app DSN cutover sequencing. Follow-up: `coilyco-bridge/deploy` issue for the service move if `deploy#51` does not already cover GlitchTip runtime placement in enough detail. - **Forgejo CI/build/deploy runners** - `forgejo-runner`, `forgejo-runner-deploy`, and `forgejo-runner-tap-writer` belong on ser8. Why: `infrastructure#314` already identifies them as ops-plane workloads and moving them removes kai-server restart blast radius from CI. Implications: moderate RAM/ephemeral disk use, plus cross-cluster registry/network design needs care. Dependencies: resolve runner-to-registry path and any host-executor assumptions before moving. Follow-up: `coilyco-flight-deck/infrastructure#314` is the right umbrella. - **Issue/status mirrors and operator-facing readers** - `forgejo-issues` and likely `signoz-mcp` should live on ser8. Why: they are meta-services over the fleet, not product apps. `forgejo-issues` is a pure ops surface (Windmill + Postgres + NocoDB), and `signoz-mcp` reads ser8 SigNoz anyway, so keeping it on kai-server adds an unnecessary hop and ties ops visibility to the app plane. Implications: light-to-moderate RAM, persistent disk for `forgejo-issues`, negligible disk for `signoz-mcp`, mostly tailnet/internal traffic. Dependencies: service manifests live in `coilyco-bridge/deploy`, so this is deploy-owned placement work. Follow-up: `coilyco-bridge/deploy` issues for `forgejo-issues` placement on ser8 and `signoz-mcp` placement on ser8. - **Background fleet automation and bot services** - `coilyco-ops-summon` should move to ser8, alongside existing ser8 timers like `git-state-watch`, `fleet-converge`, `flux-readiness-watch`, and `agentic-os-pin-converge`. Why: these are operator workflows, and several already use the "run on ser8, probe kai-server" pattern. Implications: low RAM/disk, mostly API/network traffic. Dependencies: none beyond systemd/ansible rollout. Follow-up: `coilyco-flight-deck/infrastructure` issue to move remaining bot/automation timers from kai-server to the ser8 ops host. - **Issue-corpus runtime pieces** - the repo already treats issue-corpus as an ops-plane anchor on ser8, and the unresolved runtime runner should follow the same rule. Why: it is discovery/automation infra, not an app workload. Implications: mostly runner/tooling cost, not service cost. Dependencies: dedicated runner provisioning from `docs/issue-corpus-rollout-runner.md`. Follow-up: `coilyco-flight-deck/infrastructure` issue if the dormant runner work is revived. - **Secret-management support for long-lived ser8 services** - not a workload, but likely a prerequisite. ser8 currently lacks external-secrets, so more moved services means more imperative secrets. Recommendation: if more than one or two persistent services move, add a tracked decision on whether ser8 should also run external-secrets. Follow-up: `coilyco-flight-deck/infrastructure` design issue. ### Keep split / external vantage - **Fleet reachability / Gatus** - keep the current per-cluster and per-host split. Why: reachability must be measured from multiple vantage points, not centralized into one cluster. Implications: low RAM, almost no disk, valuable cross-host network signal. Dependencies: none. Follow-up: continue `infrastructure#232` / current rollout path, but do not "move Gatus to ser8 only". - **ser8 watchtower** - keep it host-level on ser8, outside k3s. Why: it is explicitly there so one sick cluster can still observe the other. Implications: tiny cost, high resilience value. Follow-up: none. - **Node-local MCPs** - `node-stats-mcp` should stay per-node (`kai-server` and `ser8`) because it reads node-local state. Why: centralizing it would lose the host-local contract. Implications: tiny footprint, but keep both surfaces. Follow-up: none. - **Hosted Sentry during the cutover** - stays external until each app swaps its DSN to GlitchTip. Why: it is the current error sink for some apps. Implications: no homelab resource cost, but clear migration bookkeeping is required. Follow-up: `coilyco-flight-deck/infrastructure#395` remains the right tracker. ### Retire instead of move - **kai-server Grafana / old shared observability residue** - docs already call Grafana retired on the ser8 side and kai-server `observability` legacy. Do not spend design energy relocating dead panes. Follow-up: bundle the final deletion with `infrastructure#505` once verified absent or unused. - **Legacy kai-server VM/SigNoz surfaces after cutover** - tailscale proxies, NodePorts, and ward verbs that recreate the old stack should be deleted once producers are repointed. Why: leaving them around makes accidental re-creation easy. Follow-up: `coilyco-flight-deck/infrastructure#505`. - **Old mirror timers rather than porting them to ser8** - `docs/FEATURES.md` already marks the current Forgejo<->GitHub `push --mirror` timers as being replaced by the safer mirror-sync path. Recommendation: do not migrate the old timers to ser8 as a long-term pattern. Retire them once the safer replacement is live. Follow-up: `coilyco-flight-deck/infrastructure` under the existing mirror-sync work, not a new ser8-move issue. - **`coily-audit-dashboard`** - explicitly called out as kai-server coily-retention only. Recommendation: retire with the remaining kai-server coily dependency rather than moving it. Follow-up: tie to the existing coily-retention/gaming retirement work. ## Safe order Use this order to improve reliability without turning ser8 into one fragile blob: 1. **Keep the independent vantage points intact first.** Do not centralize Gatus, watchtower, or node-local collectors. Those are the guardrails for everything else. 2. **Finish the ser8 observability sinks and producer repoints.** That means ser8 VM/SigNoz are authoritative, and every kai-server local producer either points there or is explicitly dropped. 3. **Evacuate runners from kai-server to ser8.** This reduces restart blast radius and matches the ops-plane role, but it should happen only once the registry/network path is proven. 4. **Move lightweight stateless operator services next.** `coilyco-ops-summon`, `signoz-mcp`, and similar readers/bots are low-risk wins that reinforce ser8 as the operator home. 5. **Move moderate stateful operator services after that.** `forgejo-issues` is a good example: useful on the ops plane, but it introduces its own Postgres/Windmill persistence and should follow the earlier stateless moves. 6. **Move GlitchTip last among the big panes.** It is stateful and coupled to DSN cutover and backup policy, so it should not be bundled into the first retirement wave. 7. **Then delete the old kai-server observability residue and retire legacy timers instead of re-homing them.** ## Suggested follow-up issues These look like the concrete child tracks the roadmap needs: - **`coilyco-flight-deck/infrastructure`** - "Repoint kai-server local telemetry producers to ser8 and delete kai-server observability bridge surfaces". Scope: `thermal-heartbeat`, `process-memory-heartbeat`, `agent-health-heartbeat`, NodePorts, tailscale proxies, ward verbs, docs. - **`coilyco-flight-deck/infrastructure`** - "Evacuate Forgejo runners from kai-server to the ser8 cluster". This already exists as `#314`; it should be treated as part of the primary-ops-plane program. - **`coilyco-flight-deck/infrastructure`** - "Move remaining bot/automation timers from kai-server to the ser8 ops host". Scope: `coilyco-ops-summon` first, plus any other read-only background automation still living on kai-server. - **`coilyco-flight-deck/infrastructure`** - "Decide whether ser8 needs external-secrets for long-lived ops services". This becomes more important as more persistent services move. - **`coilyco-bridge/deploy`** - "Place `signoz-mcp` on ser8 instead of kai-server". It is an ops reader for ser8's trace/log plane. - **`coilyco-bridge/deploy`** - "Place `forgejo-issues` on ser8 as an ops-plane mirror". Scope: node placement, storage, tailnet access, and any secret-handling implications. - **`coilyco-bridge/deploy`** - continue the existing per-stack observability ownership and placement work under `deploy#51`, with GlitchTip explicitly called out as the last heavy stateful move. ## Open questions - **Runner network path** - if runners move to ser8, is the current in-cluster registry on kai-server still the right image path, or does that become a cross-cluster bottleneck? - **ser8 secret strategy** - does ser8 stay on imperative secrets, or is external-secrets worth installing there once more ops services land? - **GlitchTip storage and backups** - what is the explicit backup/restore plan once its Postgres/Valkey move with the error pane? - **Which control-plane UIs are worth preserving at all** - `personal-dashboard` may be worth keeping local, while `coily-audit-dashboard` looks like pure retirement material. ## Risks - **Making ser8 the only vantage point** would be a mistake. Keep Gatus split and keep watchtower outside k3s. - **Moving all stateful services at once** would create a large shared failure window. VM/SigNoz, runners, `forgejo-issues`, and GlitchTip should be separate waves. - **Leaving kai-server bridge surfaces behind** creates drift and accidental resurrection risk. - **Secret-management toil on ser8** grows quickly if more stateful services move there without a deliberate secret strategy. - **Runner move before registry/network design is settled** could trade one blast radius problem for a cross-cluster CI outage. ## Bottom line The actionable plan is: - Make `ser8` the **default home for shared ops services**. - Keep **user-facing apps and producer-local collectors** on kai-server. - Keep **external vantage and per-node observers split**. - **Retire** dead legacy panes and old mirror timers instead of porting them. - Sequence the work as **repoint sinks -> move runners -> move lightweight ops readers/bots -> move heavier stateful ops services -> delete old kai-server residue**. That is the practical shape that uses ser8 as the primary ops plane without collapsing everything into one new single point of failure. --- Researched and posted automatically by `ward agent advisor --harness codex` (ward#179). This is one-shot research, not a carried change - verify before acting on it. <!-- ward-agent-reply --> <!-- ward-agent-signature --> — Codex, via `ward agent`
coilyco-ops commented 2026-07-10 11:57:05 +00:00
Author
Member
Copy link

🔎 ward agent advisor

ward agent advisor --harness codex ran a one-shot standard research pass on this question:

Read the issue title, body, and comment thread below as the research brief. Identify the decisions, options, open questions, and risks, then give a clear advisory recommendation that can be posted back on the issue.

Recommendation

Treat ser8 as the default home for shared operator services, but keep the app plane and external vantage points deliberately split. The repo evidence supports this: deploy docs already call ser8 the canonical ops and o11y plane, kai-server the app plane, and the old kai-server observability namespace a retiring stack. The practical plan is not move everything to ser8. It is move shared ops state and readers there, keep producer-local probes local, keep reachability multi-vantage, and retire legacy surfaces once producers are repointed.

Decisions

  • Plane boundary - kai-server remains the app plane for public and user-facing services, app-local registries and ingress, game-adjacent workloads, Open WebUI, backend, Eco, ComfyUI front door, and most public or Authelia-gated MCP surfaces that are part of the app serving path.
  • Ops default - ser8 becomes the default for shared ops services: VictoriaMetrics, vmalert, SigNoz, GlitchTip, Gatus aggregation, issue/status mirrors, CI and deploy runners where the identity model fits, background fleet automation, backup receivers, and operator dashboards.
  • External vantage stays split - Gatus should remain per-node and per-cluster, the ser8 watchtower should stay host-level outside k3s, node-stats-mcp should stay per-node, and host-local SigNoz should remain loopback-only for high-detail local debugging.
  • Retirement is part of the move - old kai-server Grafana, old VM/SigNoz surfaces, legacy observability NodePorts, stale tailscale proxies, old mirror timers, and hosted Sentry after DSN cutover should disappear instead of being re-homed.

Classification

App Plane Stays On Kai-Server

  • Forgejo, registry, backend, Eco, Open WebUI, ComfyUI front door, app MCPs - These are user-facing or app-local services. Keeping them on kai-server preserves locality to app ingress, app PVCs, the in-cluster registry, and current deployment mechanics. Moving Forgejo itself would also make the ops plane depend on the forge it uses to recover and deploy.
  • App-local registry - The registry is tied to kai-server deployment and image pulls. If runners move to ser8, solve the runner-to-registry path explicitly instead of moving the registry as a side effect.
  • Producer-local collectors - thermal-heartbeat, process-memory-heartbeat, agent-health-heartbeat, GPU heartbeat, and node-local MCPs should stay on the host they observe. Their sinks should point at ser8 where applicable.
  • Host-local Signoz - The signoz ansible role is intentionally separate from shared ser8 SigNoz because it can carry full local request and response bodies. Keep it loopback-only and do not fold it into the shared ops plane.

Ops Plane Should Move Or Stay On Ser8

  • VictoriaMetrics, vmagent, vmalert, SigNoz, ClickHouse - This is already the documented direction. Resource impact is meaningful: metrics retention is 3 months and 50Gi, SigNoz brings ClickHouse and ZooKeeper, and ser8 has the larger memory and disk budget. Follow through with producer repoints, alert validation, and old namespace deletion under infrastructure#505, deploy#51, and deploy#114.
  • GlitchTip - Right home is ser8, with CloudNativePG and Valkey treated as stateful ops dependencies. Do it after backup and DSN cutover planning, not bundled with every other stateful move. Track with the existing GlitchTip and Sentry cutover work, especially infrastructure#395.
  • Forgejo runners - Build, deploy, and tap-writer runners are ops workloads, but the move is not a nodeSelector edit. Build runners need a proven cross-cluster registry path and DinD resource envelope. Deploy runners currently inherit kai-server in-cluster ServiceAccount identity, so either keep kai-server deploy runners for kai-server rollouts or design per-cluster runner labels and credentials. infrastructure#314 is the right umbrella.
  • coilyco-ops-summon and fleet automation - The repo already runs git-state-watch, fleet-converge, flux-readiness-watch, and agentic-os-pin-converge on ser8. Move remaining read-only bot and automation timers there using the same user-systemd and ansible-managed pattern.
  • signoz-mcp - It reads ser8 SigNoz but currently serves from kai-server. Move it to ser8 or document why it stays. The blocker is secret delivery because ser8 lacks external-secrets today.
  • forgejo-issues - This is an ops mirror, not an app. It should move to ser8 after secrets and backups are clear. Expect moderate persistent disk for Postgres plus Windmill and NocoDB, and steady Forgejo API traffic every 15 minutes.
  • Backup receiver and prune - These already belong on ser8 and should stay host-level. Add restore drills and make sure newly moved ser8 stateful services have a backup story that does not make ser8 the only copy.

Keep Split Or External Vantage

  • Gatus - Do not centralize this into ser8 only. The deploy repo models one Gatus per node, probing peers over the real kernel path with hostNetwork and NET_RAW. Keep ser8 and kai-server instances, then add tower and fleet rows as separate vantage points. Ser8 can own Tier-2 aggregation, but Tier-1 pages and Telegram pages should survive a ser8 o11y outage.
  • ser8 watchtower - Keep it outside k3s on the ser8 host. It watches kai-server even if either cluster is sick.
  • node-stats-mcp - Keep one per node. The kai-server surface can remain public auth-gated plus tailnet-direct, and ser8 should stay tailnet-direct only.
  • Hosted Sentry during cutover - Keep it external until each app has swapped DSNs to GlitchTip. Then retire it.
  • Service-local Tailscale sidecars - Keep these with their producers. They are part of service reachability, not shared ops state.

Retire Instead Of Move

  • kai-server Grafana and old observability residue - Grafana is retired and the old kai-server observability namespace should be deleted after ser8 coverage is verified.
  • Legacy VM/SigNoz endpoints on kai-server - Delete NodePorts, tailscale services, docs, and ward verbs that would recreate the old stack after producers are repointed.
  • old mirror timers - Prefer the safer mirror-sync path. Do not spend ser8 capacity preserving obsolete timers.
  • issue-corpus runner path - Recheck this against forgejo-issues, which now documents itself as the replacement for the old git-rendered issue corpus. Do not revive the old runner unless there is still a distinct offline-corpus requirement.
  • ntfy alert bridge - Retire after Telegram delivery is fully trusted.

Safe Order

  1. Preserve guardrails first - keep watchtower, Gatus split-vantage, node-stats per-node, and restic receiver/prune intact.
  2. Finish ser8 o11y authority - validate ser8 VM, SigNoz, vmalert, GlitchTip reachability, repoint producers, then delete old kai-server o11y surfaces.
  3. Decide ser8 secret strategy - either install external-secrets on ser8 or standardize imperative SSM-to-Secret procedures before moving more persistent services.
  4. Design runner migration - settle registry, PVC, DinD, and deploy identity. Move build and tap-writer first. Treat deploy runners separately because of in-cluster ServiceAccount assumptions.
  5. Move lightweight ops readers and bots - coilyco-ops-summon, signoz-mcp, and similar stateless readers are low-risk once secrets are solved.
  6. Move stateful meta-services - move forgejo-issues and any remaining GlitchTip state only after backup and restore paths are written down.
  7. Retire old surfaces - remove legacy namespace contents, stale docs, obsolete timers, and hosted Sentry once the replacement paths are proven.

Follow-Up Issues To De-Duplicate

  • coilyco-flight-deck/infrastructure - Repoint kai-server local telemetry producers to ser8 and delete legacy kai-server observability bridge surfaces. This extends infrastructure#505.
  • coilyco-flight-deck/infrastructure - Finish the Forgejo runner ser8 design under infrastructure#314, with explicit registry, PVC, DinD, and deploy-runner identity decisions.
  • coilyco-flight-deck/infrastructure - Decide whether ser8 should run external-secrets before more long-lived services move there.
  • coilyco-flight-deck/infrastructure - Move coilyco-ops-summon and any remaining read-only bot timers to the ser8 user-systemd pattern.
  • coilyco-flight-deck/infrastructure - Add backup and restore coverage for stateful ser8 ops services, especially GlitchTip and forgejo-issues.
  • coilyco-bridge/deploy - Move or explicitly pin signoz-mcp placement. It reads ser8 SigNoz, but current access is kai-server:30083.
  • coilyco-bridge/deploy - Move forgejo-issues to ser8 as the private ops mirror after secrets and backups are decided.
  • coilyco-bridge/deploy - Wire Gatus Tier-2 scraping into ser8 vmagent or document why annotations remain documentary.
  • coilyco-bridge/deploy - Continue deploy#51 and deploy#114 for observability ownership, with GlitchTip treated as the heavier stateful tail of the move.

Open Questions

  • Runner identity - Should ser8 host only build and tap-writer runners, while kai-server keeps deploy runners for kai-server rollouts, or should deploy runners use remote kubeconfigs or per-cluster labels?
  • Registry path - If build runners move to ser8, should they keep pushing to kai-server's NodePort registry, use a ser8 mirror, or publish somewhere else first?
  • Secret strategy - Is the lack of external-secrets on ser8 still intentional once signoz-mcp, Gatus, vmalert, GlitchTip, and forgejo-issues all need secrets?
  • Backup boundary - What backs up ser8-local state so the ops plane is not the only copy of its own history?
  • Issue corpus vs forgejo-issues - Is the old issue-corpus runner still needed, or has the Postgres-backed mirror fully replaced it?

Risks

  • Single fragile ops blob - Moving all status, alerting, runners, and mirrors at once would make ser8 a new shared failure point. Keep independent vantage and sequence stateful moves.
  • Broken CD during runner migration - The deploy runner is identity-sensitive. Moving it before designing the cluster access model can stop all app deploys.
  • Secret sprawl on ser8 - More imperative secrets without a policy will make recovery and rotation brittle.
  • Historical data ambiguity - VM, SigNoz, and GlitchTip moves need explicit keep, migrate, or waive decisions for old data.
  • Legacy resurrection - Leaving old ward verbs, NodePorts, or docs around makes the old kai-server stack easy to recreate by accident.

Bottom line: make ser8 the primary ops plane for shared operator state and automation, but keep kai-server as the app plane and keep independent observation paths alive. The roadmap should improve reliability wave by wave: guardrails, o11y repoints, runner design, lightweight ops readers, stateful mirrors, then retirement cleanup.

Researched and posted automatically by ward agent advisor --harness codex (ward#179). This is one-shot research, not a carried change - verify before acting on it.

— Codex, via ward agent

### 🔎 ward agent advisor `ward agent advisor --harness codex` ran a one-shot **standard** research pass on this question: > Read the issue title, body, and comment thread below as the research brief. Identify the decisions, options, open questions, and risks, then give a clear advisory recommendation that can be posted back on the issue. --- ## Recommendation Treat `ser8` as the default home for shared operator services, but keep the app plane and external vantage points deliberately split. The repo evidence supports this: deploy docs already call `ser8` the canonical ops and o11y plane, `kai-server` the app plane, and the old kai-server `observability` namespace a retiring stack. The practical plan is not move everything to ser8. It is move shared ops state and readers there, keep producer-local probes local, keep reachability multi-vantage, and retire legacy surfaces once producers are repointed. ## Decisions * **Plane boundary** - `kai-server` remains the app plane for public and user-facing services, app-local registries and ingress, game-adjacent workloads, Open WebUI, backend, Eco, ComfyUI front door, and most public or Authelia-gated MCP surfaces that are part of the app serving path. * **Ops default** - `ser8` becomes the default for shared ops services: VictoriaMetrics, vmalert, SigNoz, GlitchTip, Gatus aggregation, issue/status mirrors, CI and deploy runners where the identity model fits, background fleet automation, backup receivers, and operator dashboards. * **External vantage stays split** - Gatus should remain per-node and per-cluster, the ser8 watchtower should stay host-level outside k3s, `node-stats-mcp` should stay per-node, and host-local SigNoz should remain loopback-only for high-detail local debugging. * **Retirement is part of the move** - old kai-server Grafana, old VM/SigNoz surfaces, legacy observability NodePorts, stale tailscale proxies, old mirror timers, and hosted Sentry after DSN cutover should disappear instead of being re-homed. ## Classification ### App Plane Stays On Kai-Server * **Forgejo, registry, backend, Eco, Open WebUI, ComfyUI front door, app MCPs** - These are user-facing or app-local services. Keeping them on kai-server preserves locality to app ingress, app PVCs, the in-cluster registry, and current deployment mechanics. Moving Forgejo itself would also make the ops plane depend on the forge it uses to recover and deploy. * **App-local registry** - The registry is tied to kai-server deployment and image pulls. If runners move to ser8, solve the runner-to-registry path explicitly instead of moving the registry as a side effect. * **Producer-local collectors** - `thermal-heartbeat`, `process-memory-heartbeat`, `agent-health-heartbeat`, GPU heartbeat, and node-local MCPs should stay on the host they observe. Their sinks should point at ser8 where applicable. * **Host-local Signoz** - The `signoz` ansible role is intentionally separate from shared ser8 SigNoz because it can carry full local request and response bodies. Keep it loopback-only and do not fold it into the shared ops plane. ### Ops Plane Should Move Or Stay On Ser8 * **VictoriaMetrics, vmagent, vmalert, SigNoz, ClickHouse** - This is already the documented direction. Resource impact is meaningful: metrics retention is 3 months and 50Gi, SigNoz brings ClickHouse and ZooKeeper, and ser8 has the larger memory and disk budget. Follow through with producer repoints, alert validation, and old namespace deletion under `infrastructure#505`, `deploy#51`, and `deploy#114`. * **GlitchTip** - Right home is ser8, with CloudNativePG and Valkey treated as stateful ops dependencies. Do it after backup and DSN cutover planning, not bundled with every other stateful move. Track with the existing GlitchTip and Sentry cutover work, especially `infrastructure#395`. * **Forgejo runners** - Build, deploy, and tap-writer runners are ops workloads, but the move is not a nodeSelector edit. Build runners need a proven cross-cluster registry path and DinD resource envelope. Deploy runners currently inherit kai-server in-cluster ServiceAccount identity, so either keep kai-server deploy runners for kai-server rollouts or design per-cluster runner labels and credentials. `infrastructure#314` is the right umbrella. * **`coilyco-ops-summon` and fleet automation** - The repo already runs `git-state-watch`, `fleet-converge`, `flux-readiness-watch`, and `agentic-os-pin-converge` on ser8. Move remaining read-only bot and automation timers there using the same user-systemd and ansible-managed pattern. * **`signoz-mcp`** - It reads ser8 SigNoz but currently serves from kai-server. Move it to ser8 or document why it stays. The blocker is secret delivery because ser8 lacks external-secrets today. * **`forgejo-issues`** - This is an ops mirror, not an app. It should move to ser8 after secrets and backups are clear. Expect moderate persistent disk for Postgres plus Windmill and NocoDB, and steady Forgejo API traffic every 15 minutes. * **Backup receiver and prune** - These already belong on ser8 and should stay host-level. Add restore drills and make sure newly moved ser8 stateful services have a backup story that does not make ser8 the only copy. ### Keep Split Or External Vantage * **Gatus** - Do not centralize this into ser8 only. The deploy repo models one Gatus per node, probing peers over the real kernel path with hostNetwork and `NET_RAW`. Keep ser8 and kai-server instances, then add tower and fleet rows as separate vantage points. Ser8 can own Tier-2 aggregation, but Tier-1 pages and Telegram pages should survive a ser8 o11y outage. * **ser8 watchtower** - Keep it outside k3s on the ser8 host. It watches kai-server even if either cluster is sick. * **`node-stats-mcp`** - Keep one per node. The kai-server surface can remain public auth-gated plus tailnet-direct, and ser8 should stay tailnet-direct only. * **Hosted Sentry during cutover** - Keep it external until each app has swapped DSNs to GlitchTip. Then retire it. * **Service-local Tailscale sidecars** - Keep these with their producers. They are part of service reachability, not shared ops state. ### Retire Instead Of Move * **kai-server Grafana and old observability residue** - Grafana is retired and the old kai-server observability namespace should be deleted after ser8 coverage is verified. * **Legacy VM/SigNoz endpoints on kai-server** - Delete NodePorts, tailscale services, docs, and ward verbs that would recreate the old stack after producers are repointed. * **old mirror timers** - Prefer the safer mirror-sync path. Do not spend ser8 capacity preserving obsolete timers. * **issue-corpus runner path** - Recheck this against `forgejo-issues`, which now documents itself as the replacement for the old git-rendered issue corpus. Do not revive the old runner unless there is still a distinct offline-corpus requirement. * **ntfy alert bridge** - Retire after Telegram delivery is fully trusted. ## Safe Order 1. **Preserve guardrails first** - keep watchtower, Gatus split-vantage, node-stats per-node, and restic receiver/prune intact. 2. **Finish ser8 o11y authority** - validate ser8 VM, SigNoz, vmalert, GlitchTip reachability, repoint producers, then delete old kai-server o11y surfaces. 3. **Decide ser8 secret strategy** - either install external-secrets on ser8 or standardize imperative SSM-to-Secret procedures before moving more persistent services. 4. **Design runner migration** - settle registry, PVC, DinD, and deploy identity. Move build and tap-writer first. Treat deploy runners separately because of in-cluster ServiceAccount assumptions. 5. **Move lightweight ops readers and bots** - `coilyco-ops-summon`, `signoz-mcp`, and similar stateless readers are low-risk once secrets are solved. 6. **Move stateful meta-services** - move `forgejo-issues` and any remaining GlitchTip state only after backup and restore paths are written down. 7. **Retire old surfaces** - remove legacy namespace contents, stale docs, obsolete timers, and hosted Sentry once the replacement paths are proven. ## Follow-Up Issues To De-Duplicate * **`coilyco-flight-deck/infrastructure`** - Repoint kai-server local telemetry producers to ser8 and delete legacy kai-server observability bridge surfaces. This extends `infrastructure#505`. * **`coilyco-flight-deck/infrastructure`** - Finish the Forgejo runner ser8 design under `infrastructure#314`, with explicit registry, PVC, DinD, and deploy-runner identity decisions. * **`coilyco-flight-deck/infrastructure`** - Decide whether ser8 should run external-secrets before more long-lived services move there. * **`coilyco-flight-deck/infrastructure`** - Move `coilyco-ops-summon` and any remaining read-only bot timers to the ser8 user-systemd pattern. * **`coilyco-flight-deck/infrastructure`** - Add backup and restore coverage for stateful ser8 ops services, especially GlitchTip and `forgejo-issues`. * **`coilyco-bridge/deploy`** - Move or explicitly pin `signoz-mcp` placement. It reads ser8 SigNoz, but current access is `kai-server:30083`. * **`coilyco-bridge/deploy`** - Move `forgejo-issues` to ser8 as the private ops mirror after secrets and backups are decided. * **`coilyco-bridge/deploy`** - Wire Gatus Tier-2 scraping into ser8 vmagent or document why annotations remain documentary. * **`coilyco-bridge/deploy`** - Continue `deploy#51` and `deploy#114` for observability ownership, with GlitchTip treated as the heavier stateful tail of the move. ## Open Questions * **Runner identity** - Should ser8 host only build and tap-writer runners, while kai-server keeps deploy runners for kai-server rollouts, or should deploy runners use remote kubeconfigs or per-cluster labels? * **Registry path** - If build runners move to ser8, should they keep pushing to kai-server's NodePort registry, use a ser8 mirror, or publish somewhere else first? * **Secret strategy** - Is the lack of external-secrets on ser8 still intentional once `signoz-mcp`, Gatus, vmalert, GlitchTip, and `forgejo-issues` all need secrets? * **Backup boundary** - What backs up ser8-local state so the ops plane is not the only copy of its own history? * **Issue corpus vs `forgejo-issues`** - Is the old issue-corpus runner still needed, or has the Postgres-backed mirror fully replaced it? ## Risks * **Single fragile ops blob** - Moving all status, alerting, runners, and mirrors at once would make ser8 a new shared failure point. Keep independent vantage and sequence stateful moves. * **Broken CD during runner migration** - The deploy runner is identity-sensitive. Moving it before designing the cluster access model can stop all app deploys. * **Secret sprawl on ser8** - More imperative secrets without a policy will make recovery and rotation brittle. * **Historical data ambiguity** - VM, SigNoz, and GlitchTip moves need explicit keep, migrate, or waive decisions for old data. * **Legacy resurrection** - Leaving old ward verbs, NodePorts, or docs around makes the old kai-server stack easy to recreate by accident. Bottom line: make ser8 the primary ops plane for shared operator state and automation, but keep kai-server as the app plane and keep independent observation paths alive. The roadmap should improve reliability wave by wave: guardrails, o11y repoints, runner design, lightweight ops readers, stateful mirrors, then retirement cleanup. --- Researched and posted automatically by `ward agent advisor --harness codex` (ward#179). This is one-shot research, not a carried change - verify before acting on it. <!-- ward-agent-reply --> <!-- ward-agent-signature --> — Codex, via `ward agent`
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
issue-515
issue-486
issue-529
issue-500
issue-483
No results found.
Labels
Clear labels   
burndown-2026-06
Backlog burndown June 2026

  
coherence-core
Core review set for the warded control plane coherence milestone. These issues form the release spine; adjacent milestone issues are stretch or supporting work.

  
consult
A human decision, design, or external action must happen first. Fail-closed default for unlabeled/low-confidence.

  
headless
Agent can take it from open issue to merged change with no human in the loop. The auto-burndown queue.

  
interactive
Agent does the work but pauses at a human checkpoint mid-flight (design choice, destructive step, human-only verification).

  
P0
now / blocking / urgent

  
P1
next / important

  
P2
backlog / will do

  
P3
someday / low

  
P4
icebox / frozen / reopen if it becomes real

No labels
burndown-2026-06
coherence-core
consult
headless
interactive
P0
P1
P2
P3
P4
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
coilyco-flight-deck/infrastructure#506
No description provided.