Always-on hardening for kai-desktop-tower-wsl k3s node (WSL lifecycle) #242

New issue

Closed

opened 2026-06-06 05:11:24 +00:00 by coilysiren · 1 comment

coilysiren commented

2026-06-06 05:11:24 +00:00

Owner

kai-desktop-tower-wsl is a k3s agent running inside WSL2 on the Windows desktop tower. WSL's desktop-oriented lifecycle makes it go NotReady in ways a normal Linux box never would - it caused a 2-day silent CI outage (coilysiren/inbox#65, root cause). CI no longer depends on this node (runners unpinned, infrastructure#241), but if we want it back as a reliable worker it needs four layers of always-on hardening. All steps run on the Windows tower itself.

Hardening checklist

Defeat idle shutdown - %UserProfile%\.wslconfig: [wsl2] vmIdleTimeout=-1 AND [general] instanceIdleTimeout=-1 (both required; needs WSL >= 2.4.4). wsl --shutdown to apply.
systemd as init - /etc/wsl.conf: [boot] systemd=true. Run k3s-agent (and tailscaled) as systemd units so they are supervised and restart on distro boot.
Start WSL at Windows boot, no login - Task Scheduler task: trigger At startup, "Run whether user is logged on or not," highest privileges, action wsl.exe -d <distro> -u root -e dbus-launch true. Boots the VM headless so systemd brings up k3s without a human shell.
Stop host sleep - powercfg /change standby-timeout-ac 0, powercfg /change hibernate-timeout-ac 0, powercfg /hibernate off. BIOS "restore on AC power loss -> On". Disable Windows fast startup.
Networking - node already registers on its Tailscale IP (100.100.229.108), which sidesteps WSL NAT-IP churn. Ensure tailscaled runs under systemd. (networkingMode=mirrored not needed given Tailscale-as-node-IP.)

Sturdier alternative to evaluate

Running k3s in a Hyper-V Linux VM on the tower (real autostart, no desktop-lifecycle coupling) instead of WSL. More setup, far less fragility. Decide WSL-hardening vs Hyper-V-VM before investing in the checklist.

When resolved, fold the working recipe into docs/k3s-deploy-notes.md §7.

Design notes captured in chat 2026-06-05.

`kai-desktop-tower-wsl` is a k3s agent running inside WSL2 on the Windows desktop tower. WSL's desktop-oriented lifecycle makes it go `NotReady` in ways a normal Linux box never would - it caused a 2-day silent CI outage (coilysiren/inbox#65, root cause). CI no longer depends on this node (runners unpinned, infrastructure#241), but if we want it back as a reliable worker it needs four layers of always-on hardening. All steps run on the Windows tower itself. ## Hardening checklist - [ ] **Defeat idle shutdown** - `%UserProfile%\.wslconfig`: `[wsl2] vmIdleTimeout=-1` AND `[general] instanceIdleTimeout=-1` (both required; needs WSL >= 2.4.4). `wsl --shutdown` to apply. - [ ] **systemd as init** - `/etc/wsl.conf`: `[boot] systemd=true`. Run k3s-agent (and tailscaled) as systemd units so they are supervised and restart on distro boot. - [ ] **Start WSL at Windows boot, no login** - Task Scheduler task: trigger At startup, "Run whether user is logged on or not," highest privileges, action `wsl.exe -d <distro> -u root -e dbus-launch true`. Boots the VM headless so systemd brings up k3s without a human shell. - [ ] **Stop host sleep** - `powercfg /change standby-timeout-ac 0`, `powercfg /change hibernate-timeout-ac 0`, `powercfg /hibernate off`. BIOS "restore on AC power loss -> On". Disable Windows fast startup. - [ ] **Networking** - node already registers on its Tailscale IP (100.100.229.108), which sidesteps WSL NAT-IP churn. Ensure tailscaled runs under systemd. (`networkingMode=mirrored` not needed given Tailscale-as-node-IP.) ## Sturdier alternative to evaluate Running k3s in a Hyper-V Linux VM on the tower (real autostart, no desktop-lifecycle coupling) instead of WSL. More setup, far less fragility. Decide WSL-hardening vs Hyper-V-VM before investing in the checklist. When resolved, fold the working recipe into `docs/k3s-deploy-notes.md` §7. Design notes captured in chat 2026-06-05.

coilysiren referenced this issue from a commit

2026-06-06 05:12:09 +00:00

fix(forgejo): unpin runners from kai-desktop-tower-wsl

coilysiren referenced this issue

2026-06-07 05:00:12 +00:00

kai-desktop-tower: 2:50am Windows maintenance reboot drops both tailnet nodes, no remote recovery (~19h offline) #257

coilysiren added the

label

2026-06-17 07:05:27 +00:00

coilysiren commented

2026-06-17 08:22:36 +00:00

Author

Owner

Backlog burndown 2026-06-17: closing low-priority (P3/P4) to bring the open count to a manageable level. Nothing lost — reopen if this resurfaces. Batch tag: burndown-2026-06.

Backlog burndown 2026-06-17: closing low-priority (P3/P4) to bring the open count to a manageable level. Nothing lost — reopen if this resurfaces. Batch tag: `burndown-2026-06`.

coilysiren

2026-06-17 08:22:36 +00:00