ops kubectl: self-elevate for mutating verbs the same way systemctl does #23

New issue

Open

opened 2026-05-23 20:53:58 +00:00 by coilysiren · 0 comments

coilysiren commented

2026-05-23 20:53:58 +00:00

Owner

Originally filed by @coilysiren on 2026-05-19T09:22:46Z - https://github.com/coilysiren/coily/issues/254

Problem - coily ops kubectl ... on kai-server fails with error loading config file "/etc/rancher/k3s/k3s.yaml": open /etc/rancher/k3s/k3s.yaml: permission denied. The k3s kubeconfig is root-owned on the host (--write-kubeconfig-mode default), so any kubectl invocation needs to run as root to read it.

Bare coily ssh kai-server -- sudo k3s kubectl apply ... doesn't work either: coily ssh wraps the remote argv into coily <args> on the destination, so it lands as coily sudo k3s kubectl ... which trips the policy gate ("flag provided but not defined: -f").

This is the same shape as coily#203 - coily systemctl <mutating-verb> had the same "needs sudo, no audited path through coily" gap. The fix landed in coily#244 (forward outer --commit-scope) and coily#245 (forward outer --cwd): when the verb runs as non-root for a mutating subcommand, re-exec the coily binary under sudo --non-interactive <coily-path> ops kubectl <verb> <args...>, relying on the broad (ALL) NOPASSWD: <coily-path> sudoers grant on hosts that have it.

Proposed shape - When coily ops kubectl is invoked as non-root AND the verb is a mutating one (apply, delete, create, replace, patch, scale, rollout, set, cordon, uncordon, drain, taint, label, annotate, edit):

Build the re-exec argv: ["sudo", "--non-interactive", coilyPath, "--cwd=" + outerCwd, "ops", "kubectl", verb, args...]. Forward --cwd so the inner can resolve scope from the same git toplevel the outer is bound to (mirrors coily#245).
Re-exec via r.Runner.Exec(ctx, argv[0], argv[1:]...). No inner sudo.
Audit row is written twice (outer + inner), the same way coily systemctl produces two rows on a self-elevated invocation.

Read-only verbs (get, describe, logs, top, explain, api-resources, api-versions, config view) stay unprivileged. On hosts with --write-kubeconfig-mode 0644 they work as-is; on stricter hosts they trip the same permission-denied as the mutating verbs, but the right fix there is chmod 0644 on the kubeconfig (or a per-user copy at ~/.kube/config), not self-elevation of every read.

Cross-links

coily#203 - the design intent for self-elevation. "coily is the security boundary, the broad NOPASSWD: grant is the trusted shape."
coily#244 - propagate outer --commit-scope so the inner doesn't trip the strict-mode scope gate.
coily#245 - propagate outer --cwd so the inner resolves the same git toplevel.
coily#249 - the parallel gap for brew services mutating verbs. Closed by the coily brew services wrapper (commit 61272f3).

Test plan - Extend cmd/coily/ops_kubectl_test.go (or add it if it doesn't exist) with a TestBuildOpsKubectlSelfElevateArgv mirroring the systemctl test shape: each mutating verb produces the right argv (outer sudo, coily binary, --cwd=<toplevel>, ops kubectl, verb, args); read-only verbs produce no sudo prefix; an inner-sudo at argv[1+] trips a security-claim test the same way TestSecurityClaim_SystemctlSelfElevatesNotInnerSudo does.

Blocks

The manual coily ssh kai-server -- coily ops kubectl apply -f deploy/repo-recall.yml first-apply step that brings the k3s deploy up. Right now the user has to dictate sudo k3s kubectl apply ... directly on kai-server.
The future ship job in coilysiren/repo-recall/docker.yml once the deploy-user issue (infrastructure#TBD) is resolved - kubectl set image runs over tailscale ssh today, but a coily ops kubectl path with self-elevation would let the homelab's audit trail capture the deploy directly.

_Originally filed by @coilysiren on 2026-05-19T09:22:46Z - [https://github.com/coilysiren/coily/issues/254](https://github.com/coilysiren/coily/issues/254)_ **Problem** - `coily ops kubectl ...` on kai-server fails with `error loading config file "/etc/rancher/k3s/k3s.yaml": open /etc/rancher/k3s/k3s.yaml: permission denied`. The k3s kubeconfig is root-owned on the host (`--write-kubeconfig-mode` default), so any kubectl invocation needs to run as root to read it. Bare `coily ssh kai-server -- sudo k3s kubectl apply ...` doesn't work either: coily ssh wraps the remote argv into `coily <args>` on the destination, so it lands as `coily sudo k3s kubectl ...` which trips the policy gate ("flag provided but not defined: -f"). **This is the same shape as coily#203** - `coily systemctl <mutating-verb>` had the same "needs sudo, no audited path through coily" gap. The fix landed in coily#244 (forward outer `--commit-scope`) and coily#245 (forward outer `--cwd`): when the verb runs as non-root for a mutating subcommand, re-exec the coily binary under `sudo --non-interactive <coily-path> ops kubectl <verb> <args...>`, relying on the broad `(ALL) NOPASSWD: <coily-path>` sudoers grant on hosts that have it. **Proposed shape** - When `coily ops kubectl` is invoked as non-root AND the verb is a mutating one (`apply`, `delete`, `create`, `replace`, `patch`, `scale`, `rollout`, `set`, `cordon`, `uncordon`, `drain`, `taint`, `label`, `annotate`, `edit`): 1. Build the re-exec argv: `["sudo", "--non-interactive", coilyPath, "--cwd=" + outerCwd, "ops", "kubectl", verb, args...]`. Forward `--cwd` so the inner can resolve scope from the same git toplevel the outer is bound to (mirrors coily#245). 2. Re-exec via `r.Runner.Exec(ctx, argv[0], argv[1:]...)`. No inner sudo. 3. Audit row is written twice (outer + inner), the same way `coily systemctl` produces two rows on a self-elevated invocation. Read-only verbs (`get`, `describe`, `logs`, `top`, `explain`, `api-resources`, `api-versions`, `config view`) stay unprivileged. On hosts with `--write-kubeconfig-mode 0644` they work as-is; on stricter hosts they trip the same permission-denied as the mutating verbs, but the right fix there is `chmod 0644` on the kubeconfig (or a per-user copy at `~/.kube/config`), not self-elevation of every read. **Cross-links** - coily#203 - the design intent for self-elevation. "coily is the security boundary, the broad NOPASSWD: <coily-path> grant is the trusted shape." - coily#244 - propagate outer `--commit-scope` so the inner doesn't trip the strict-mode scope gate. - coily#245 - propagate outer `--cwd` so the inner resolves the same git toplevel. - coily#249 - the parallel gap for `brew services` mutating verbs. Closed by the `coily brew services` wrapper (commit 61272f3). **Test plan** - Extend `cmd/coily/ops_kubectl_test.go` (or add it if it doesn't exist) with a `TestBuildOpsKubectlSelfElevateArgv` mirroring the systemctl test shape: each mutating verb produces the right argv (outer sudo, coily binary, `--cwd=<toplevel>`, `ops kubectl`, verb, args); read-only verbs produce no sudo prefix; an inner-sudo at argv[1+] trips a security-claim test the same way `TestSecurityClaim_SystemctlSelfElevatesNotInnerSudo` does. **Blocks** - The manual `coily ssh kai-server -- coily ops kubectl apply -f deploy/repo-recall.yml` first-apply step that brings the k3s deploy up. Right now the user has to dictate `sudo k3s kubectl apply ...` directly on kai-server. - The future `ship` job in coilysiren/repo-recall/docker.yml once the deploy-user issue (infrastructure#TBD) is resolved - `kubectl set image` runs over tailscale ssh today, but a `coily ops kubectl` path with self-elevation would let the homelab's audit trail capture the deploy directly.

coilysiren referenced this issue

2026-05-23 20:54:03 +00:00

off-host audit-log shadow: needs, not implementation #56

coilysiren added the

label

2026-05-31 01:52:50 +00:00

coilysiren added

and removed

labels

2026-05-31 06:59:53 +00:00