chore(observability): delete Grafana live resources (manual followup to #388) #391

New issue

Open

opened 2026-06-23 08:38:13 +00:00 by coilyco-ops · 1 comment

coilyco-ops commented

2026-06-23 08:38:13 +00:00

Member

The tracked-code Grafana teardown landed in #388 (merged as 15b0f98). This issue tracks the live-resource deletion, which needs kubectl/terraform against kai-server's k3s and is Kai's manual followup - not a headless-agent task.

Checklist

helm uninstall grafana -n observability
Delete the Grafana PVC in the observability namespace (the 2 GiB data volume).
Confirm the Grafana admin-password ExternalSecret/Secret is gone (its manifest was removed in #388; delete any lingering live object).
terraform apply in aws-inventory to drop the grafana.coilysiren.me Route53 A record (the record was removed from tracked code in #388, the apply deletes the live record).
Remove the issued TLS cert for grafana.coilysiren.me.
Tear down the terraform/grafana remote state (the module dir was deleted in #388).

Notes

Keep VictoriaMetrics, vmagent, vmsingle, node-exporter running - only Grafana goes. VM stays as the headless metrics store (the eco-telemetry agent-loop reads it via a VM MCP).
Order does not matter much, but drop the DNS record + cert last so nothing is briefly pointing at a torn-down service longer than needed.
After this, metrics live in VM (headless) and dashboards/glancing move to SigNoz - see the metrics-to-SigNoz migration (#389 covers the agent-health slice).

Refs

Tracked-code teardown: #388 (closed).
Heartbeat to SigNoz: #389.

The tracked-code Grafana teardown landed in #388 (merged as `15b0f98`). This issue tracks the **live-resource deletion**, which needs `kubectl`/`terraform` against kai-server's k3s and is Kai's manual followup - not a headless-agent task. ## Checklist - [ ] `helm uninstall grafana -n observability` - [ ] Delete the Grafana PVC in the `observability` namespace (the 2 GiB data volume). - [ ] Confirm the Grafana admin-password ExternalSecret/Secret is gone (its manifest was removed in #388; delete any lingering live object). - [ ] `terraform` apply in `aws-inventory` to drop the `grafana.coilysiren.me` Route53 A record (the record was removed from tracked code in #388, the apply deletes the live record). - [ ] Remove the issued TLS cert for `grafana.coilysiren.me`. - [ ] Tear down the `terraform/grafana` remote state (the module dir was deleted in #388). ## Notes - **Keep VictoriaMetrics, vmagent, vmsingle, node-exporter running** - only Grafana goes. VM stays as the headless metrics store (the eco-telemetry agent-loop reads it via a VM MCP). - Order does not matter much, but drop the DNS record + cert last so nothing is briefly pointing at a torn-down service longer than needed. - After this, metrics live in VM (headless) and dashboards/glancing move to SigNoz - see the metrics-to-SigNoz migration (#389 covers the agent-health slice). ## Refs - Tracked-code teardown: #388 (closed). - Heartbeat to SigNoz: #389.

coilyco-ops added the

interactive

labels

2026-06-24 05:38:16 +00:00

coilyco-ops commented

2026-06-24 05:38:17 +00:00

Author

Member

Triage: P3 / interactive - Delete live Grafana resources; body states it is a manual destructive kubectl/terraform followup, not a headless task.

(goose-style pass; claude as the judgment engine, applied 2026-06-23)

**Triage:** `P3` / `interactive` - Delete live Grafana resources; body states it is a manual destructive kubectl/terraform followup, not a headless task. (goose-style pass; claude as the judgment engine, applied 2026-06-23)

coilyco-ops added

consult

and removed

interactive

labels

2026-07-10 09:00:04 +00:00