Add read-only k3s pod/container attribution to node-stats MCP #6

New issue

Closed

opened 2026-07-09 20:52:42 +00:00 by coilyco-ops · 2 comments

coilyco-ops commented

2026-07-09 20:52:42 +00:00

Member

node-stats-mcp is now useful for kai-server host inventory, but its process memory output is too low-level for k3s workloads. A live inventory saw service processes like reddit-mcp, but Open WebUI only appeared as generic uvicorn/python names. The director surface also lacked a working kubectl binary, so pod/container memory attribution could not be recovered from kubectl top.

Please teach node-stats-mcp a general-purpose read-only k3s view so agents can answer host-to-pod questions through MCP without shelling out to kubectl.

Suggested v1 surface:

get_k3s_pods - namespace, pod, phase, node, restart count, container names/images, pod IP, age if available.
get_k3s_container_memory - namespace/pod/container memory working set/RSS where available, sorted by memory.
get_k3s_process_attribution - map the existing top host processes to owning pod/container when cgroup/container metadata allows it.
Keep it strictly read-only. No exec, logs, secret reads, deletes, patches, rollout actions, or arbitrary kubectl proxy.

Implementation direction:

Prefer direct Python reads/API over invoking kubectl.
For metadata, use the in-cluster Kubernetes API with a minimal ServiceAccount/Role if needed.
For memory attribution, prefer cgroup/containerd data already visible from hostPID + hostPath, or metrics-server API if available. Make missing metrics graceful.
Preserve the existing node-local tools and public/tailnet MCP behavior.
Update the deploy surface if new RBAC/serviceaccount/env/volume mounts are needed.

Acceptance:

From kai-server:9110/mcp, an agent can identify reddit-mcp and open-webui by namespace/pod/container and see approximate memory use without needing kubectl in the caller container.
Existing host memory/process tools still work.
Tests cover the parser/API fallback path enough that generic process names do not silently become unattributed when cgroup data is present.

node-stats-mcp is now useful for kai-server host inventory, but its process memory output is too low-level for k3s workloads. A live inventory saw service processes like `reddit-mcp`, but Open WebUI only appeared as generic `uvicorn`/`python` names. The director surface also lacked a working `kubectl` binary, so pod/container memory attribution could not be recovered from `kubectl top`. Please teach node-stats-mcp a general-purpose read-only k3s view so agents can answer host-to-pod questions through MCP without shelling out to kubectl. Suggested v1 surface: * `get_k3s_pods` - namespace, pod, phase, node, restart count, container names/images, pod IP, age if available. * `get_k3s_container_memory` - namespace/pod/container memory working set/RSS where available, sorted by memory. * `get_k3s_process_attribution` - map the existing top host processes to owning pod/container when cgroup/container metadata allows it. * Keep it strictly read-only. No exec, logs, secret reads, deletes, patches, rollout actions, or arbitrary kubectl proxy. Implementation direction: * Prefer direct Python reads/API over invoking `kubectl`. * For metadata, use the in-cluster Kubernetes API with a minimal ServiceAccount/Role if needed. * For memory attribution, prefer cgroup/containerd data already visible from hostPID + hostPath, or metrics-server API if available. Make missing metrics graceful. * Preserve the existing node-local tools and public/tailnet MCP behavior. * Update the deploy surface if new RBAC/serviceaccount/env/volume mounts are needed. Acceptance: * From `kai-server:9110/mcp`, an agent can identify `reddit-mcp` and `open-webui` by namespace/pod/container and see approximate memory use without needing `kubectl` in the caller container. * Existing host memory/process tools still work. * Tests cover the parser/API fallback path enough that generic process names do not silently become unattributed when cgroup data is present.

coilyco-ops commented

2026-07-09 20:52:49 +00:00

Author

Member

WARD-RESERVATION: held 🔒

reservation details

Holder: container engineer-codex-node-stats-mcp-6 on host kais-macbook-pro-2.local.

Reserved by ward agent --harness codex (reserved 2026-07-09T20:52:47Z). Concurrent ward agent runs are blocked until it finishes or the reservation goes stale (1h TTL). --force overrides.

Do not comment on or edit this issue to steer the run while it is reserved. The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a new issue, dispatched fresh. That is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494).

run seed context — what this run is carrying (ward#609)

Resolved: coilyco-flight-deck/node-stats-mcp#6 · branch issue-6 · harness codex · workflow direct-to-main
Run: engineer-codex-node-stats-mcp-6 · ward v0.501.0 · dispatched 2026-07-09T20:52:47Z
Comment thread: 0 included in the pre-flight read, 0 stripped (ward's own automated comments).

Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.501.0).

— Codex, via ward agent

WARD-RESERVATION: held 🔒 <details><summary>reservation details</summary> Holder: container `engineer-codex-node-stats-mcp-6` on host `kais-macbook-pro-2.local`. Reserved by `ward agent --harness codex` (reserved 2026-07-09T20:52:47Z). Concurrent `ward agent` runs are blocked until it finishes or the reservation goes stale (1h TTL). `--force` overrides. **Do not comment on or edit this issue to steer the run while it is reserved.** The engineer seeded the body once at launch and never re-reads it, so a comment or edit reaches only human readers, never the running engineer. A correction goes to a **new issue, dispatched fresh**. That is the only channel that reaches a run in flight. Where the forge supports it, ward locks this conversation to make that a road-block rather than a convention (ward#494). <details><summary>run seed context — what this run is carrying (ward#609)</summary> - **Resolved:** `coilyco-flight-deck/node-stats-mcp#6` · branch `issue-6` · harness `codex` · workflow `direct-to-main` - **Run:** `engineer-codex-node-stats-mcp-6` · ward `v0.501.0` · dispatched `2026-07-09T20:52:47Z` - **Comment thread:** 0 included in the pre-flight read, 0 stripped (ward's own automated comments). Static container doctrine and seed boilerplate are identical every run and omitted here (they ride ward v0.501.0). </details> </details>  — Codex, via `ward agent`

coilyco-ops referenced this issue from a commit

2026-07-09 21:06:38 +00:00

feat: add read-only k3s inventory tools

coilyco-ops closed this issue

2026-07-09 21:06:38 +00:00

coilyco-ops commented

2026-07-09 21:07:21 +00:00

Author

Member

WARD-OUTCOME: done ✅

details

workflow: direct-to-main; review summary: skipped by ~/.ward/config.yaml default
Implemented read-only k3s pod/container inventory, cgroup-backed process attribution, and RSS fallback.
felt: a bit heavier than the surface suggested, mostly around kubeconfig and typing.
confidence: high.
surprise: host-mounted k3s kubeconfig was enough, so no deploy-repo edit was needed here.
follow-ups: set NODE_STATS_KUBECONFIG if the host path changes.

WARD-OUTCOME: done ✅ <details><summary>details</summary> workflow: direct-to-main; review summary: skipped by ~/.ward/config.yaml default Implemented read-only k3s pod/container inventory, cgroup-backed process attribution, and RSS fallback. felt: a bit heavier than the surface suggested, mostly around kubeconfig and typing. confidence: high. surprise: host-mounted k3s kubeconfig was enough, so no deploy-repo edit was needed here. follow-ups: set NODE_STATS_KUBECONFIG if the host path changes. </details>