ansible(mac): drop the tailscale formula, keep only the tailscale-app cask (dual install wedges the tun) #253

New issue

Closed

opened 2026-06-07 02:08:47 +00:00 by coilysiren · 0 comments

coilysiren commented

2026-06-07 02:08:47 +00:00

Owner

Problem

The Mac group installs two competing Tailscale stacks from brew, which fight over the tun and wedge the kernel data path:

tailscale formula in homebrew_installed_packages (ansible/inventory/group_vars/mac.yml:108) - ships the tailscaled daemon + CLI, loaded as a launchd service.
tailscale-app cask in homebrew_cask_apps (ansible/inventory/group_vars/mac.yml:152) - the GUI app + its macsys system extension, which owns the supported macOS data path.

On macOS only the cask app should run. Running the formula tailscaled alongside it produces two daemons / two utuns / split routes: disco + userspace (tailscale ping) still works, but kernel TCP/ICMP to the tailnet black-holes (nc/ssh/ping time out).

How it surfaced

Hit live on kais-macbook-pro during the kais-macbook-pro -> kai-server o2r debug session (channel ZHQK). kai-server was healthy; the macbook could disco-ping it (7ms via LAN) but every kernel-path probe (nc :22, nc :6443, ssh) timed out. Neither tailscale down/up, a brew service restart, nor a full reboot fixed it - because the fault is two installs, not one stale daemon. Manual remediation required launchctl bootout system/homebrew.mxcl.tailscale (sudo) to stop the root-domain formula daemon and leave the cask app as the sole stack. A persistent client 1.98.5 != tailscaled 1.98.2 version skew was the tell (CLI from the formula, daemon socket from the cask's 1.98.2 extension).

Asked fix (ansible)

Force the proper, single Tailscale install on Macs so this can't recur on convergence/reboot:

Remove tailscale (the formula) from homebrew_installed_packages for the mac group - keep only the tailscale-app cask.
Converge existing machines off the formula: stop + unload the formula's launchd service (user agent and the root /Library/LaunchDaemons/homebrew.mxcl.tailscale.plist) and brew uninstall tailscale so already-provisioned Macs self-heal, not just fresh ones.
If the tailscale CLI on PATH is wanted, confirm the cask app provides it (it does, in the app bundle) before dropping the formula - so coily tailscale keeps resolving.
Add a short note in mac.yml near the cask entry: "do NOT also add the tailscale formula - dual install wedges the tun (see this issue)."

Acceptance

A converged Mac shows exactly one Tailscale stack (the cask app + macsys extension), no formula tailscaled process, no version skew warning from tailscale ip.
Kernel-path probes to a tailnet peer succeed (nc -z <peer> 22 / ssh <peer> connect).

## Problem The Mac group installs **two competing Tailscale stacks** from brew, which fight over the tun and wedge the kernel data path: - `tailscale` **formula** in `homebrew_installed_packages` (`ansible/inventory/group_vars/mac.yml:108`) - ships the `tailscaled` daemon + CLI, loaded as a launchd service. - `tailscale-app` **cask** in `homebrew_cask_apps` (`ansible/inventory/group_vars/mac.yml:152`) - the GUI app + its `macsys` system extension, which owns the supported macOS data path. On macOS only the cask app should run. Running the formula `tailscaled` alongside it produces two daemons / two utuns / split routes: disco + userspace (`tailscale ping`) still works, but kernel TCP/ICMP to the tailnet black-holes (`nc`/`ssh`/`ping` time out). ## How it surfaced Hit live on `kais-macbook-pro` during the `kais-macbook-pro -> kai-server` o2r debug session (channel ZHQK). kai-server was healthy; the macbook could disco-ping it (7ms via LAN) but every kernel-path probe (`nc :22`, `nc :6443`, `ssh`) timed out. Neither `tailscale down/up`, a brew service restart, nor a full reboot fixed it - because the fault is two installs, not one stale daemon. Manual remediation required `launchctl bootout system/homebrew.mxcl.tailscale` (sudo) to stop the root-domain formula daemon and leave the cask app as the sole stack. A persistent `client 1.98.5 != tailscaled 1.98.2` version skew was the tell (CLI from the formula, daemon socket from the cask's 1.98.2 extension). ## Asked fix (ansible) Force the proper, single Tailscale install on Macs so this can't recur on convergence/reboot: 1. **Remove** `tailscale` (the formula) from `homebrew_installed_packages` for the mac group - keep only the `tailscale-app` cask. 2. **Converge existing machines off the formula**: stop + unload the formula's launchd service (user agent *and* the root `/Library/LaunchDaemons/homebrew.mxcl.tailscale.plist`) and `brew uninstall tailscale` so already-provisioned Macs self-heal, not just fresh ones. 3. If the `tailscale` CLI on PATH is wanted, confirm the cask app provides it (it does, in the app bundle) before dropping the formula - so `coily tailscale` keeps resolving. 4. Add a short note in `mac.yml` near the cask entry: "do NOT also add the `tailscale` formula - dual install wedges the tun (see this issue)." ## Acceptance - A converged Mac shows exactly one Tailscale stack (the cask app + macsys extension), no formula `tailscaled` process, no version skew warning from `tailscale ip`. - Kernel-path probes to a tailnet peer succeed (`nc -z <peer> 22` / `ssh <peer>` connect).

coilysiren referenced this issue

2026-06-07 03:25:08 +00:00

Full-mesh reachability healthcheck (N×N connectivity matrix) across the fleet #255

coilysiren added the

priority

label

2026-06-17 07:05:33 +00:00