Batch the native startup liveness check into one ps call #964

Closed
opened 2026-08-06 20:56:58 +00:00 by coilyco-ops · 0 comments
Member

What

cleanDeadNativeSessions calls nativeLeaseIsLive once per lease file, and on unix processStartIdentity shells out to ps -o lstart= -p <pid> (aos-cli/platform_unix.go:27). That is one process spawn per lease, on every native launch, inside the startup lock.

Measured cost

On a host carrying 125 lease files:

  • 125 ps spawns: 0.38s
  • 1081 artifact symlink resolutions, for contrast: 0.02s

So the liveness check is roughly 0.35s of avoidable per-launch startup latency, and it scales with the lease count rather than with the work actually being done.

Why it is worth doing

The lease count is normal accumulation, not a leak. Of the 125 above, 108 were dead but inside the 24h grace (the cheap path), 9 were past grace holding 20 artifacts, and 12 were live. No retention change is needed. The ps fan-out is the only real inefficiency in that loop.

Suggested shape

Collect the candidate PIDs first, then resolve all start identities in one ps -o pid=,lstart= -p <pid1>,<pid2>,... call and look each lease up in that map. Keep the current per-PID path as the fallback for a PID that the batch call does not return, so a missing process still reads as dead. processStartIdentity stays as-is for its single-PID callers (resolveNativeRuntime, the startup lock owner check).

Windows already uses a direct API call per PID (platform_windows.go:22) and does not need this.

Not urgent

The dominant startup cost is the fleet pass, roughly 19 repositories at ~0.8s per git fetch --prune origin, gated to once per ten minutes. This is a smaller, cheaper win.

Startup phase timings are now visible on stderr via the narration added in e109fea2, so the before and after are directly measurable: compare the reclaim finished sessions line.

## What `cleanDeadNativeSessions` calls `nativeLeaseIsLive` once per lease file, and on unix `processStartIdentity` shells out to `ps -o lstart= -p <pid>` (`aos-cli/platform_unix.go:27`). That is one process spawn per lease, on every native launch, inside the startup lock. ## Measured cost On a host carrying 125 lease files: * 125 `ps` spawns: **0.38s** * 1081 artifact symlink resolutions, for contrast: **0.02s** So the liveness check is roughly 0.35s of avoidable per-launch startup latency, and it scales with the lease count rather than with the work actually being done. ## Why it is worth doing The lease count is normal accumulation, not a leak. Of the 125 above, 108 were dead but inside the 24h grace (the cheap path), 9 were past grace holding 20 artifacts, and 12 were live. No retention change is needed. The `ps` fan-out is the only real inefficiency in that loop. ## Suggested shape Collect the candidate PIDs first, then resolve all start identities in one `ps -o pid=,lstart= -p <pid1>,<pid2>,...` call and look each lease up in that map. Keep the current per-PID path as the fallback for a PID that the batch call does not return, so a missing process still reads as dead. `processStartIdentity` stays as-is for its single-PID callers (`resolveNativeRuntime`, the startup lock owner check). Windows already uses a direct API call per PID (`platform_windows.go:22`) and does not need this. ## Not urgent The dominant startup cost is the fleet pass, roughly 19 repositories at ~0.8s per `git fetch --prune origin`, gated to once per ten minutes. This is a smaller, cheaper win. Startup phase timings are now visible on stderr via the narration added in e109fea2, so the before and after are directly measurable: compare the `reclaim finished sessions` line.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
coilyco-flight-deck/agentic-os#964
No description provided.