Pin OLLAMA_KEEP_ALIVE so the model stays warm (30m default) #279

New issue

Closed

opened 2026-06-09 04:23:50 +00:00 by coilysiren · 0 comments

coilysiren commented

2026-06-09 04:23:50 +00:00

Owner

The ollama role should pin OLLAMA_KEEP_ALIVE so a loaded model stays resident longer than the 5-minute default.

Why

During the OpenCode/Qwen-3 eval (#278), the first build-agent step took ~64s, much of it cold model load. Keeping qwen3.5:9b-mlx warm between requests removes that cold-start on back-to-back agent turns. Per-request keep_alive from the client would reset to the server default each call, so the durable fix is the server-side OLLAMA_KEEP_ALIVE env default.

Change

New ollama_keep_alive default (30m) + ollama_plist path in the role defaults.
Role pins OLLAMA_KEEP_ALIVE into the launchd plist EnvironmentVariables via PlistBuddy (additive - leaves OLLAMA_FLASH_ATTENTION/OLLAMA_KV_CACHE_TYPE alone) and launchctl bootout/bootstrap-reloads only on drift. Idempotent: re-converge reports changed=0.

The ollama role should pin `OLLAMA_KEEP_ALIVE` so a loaded model stays resident longer than the 5-minute default. ## Why During the OpenCode/Qwen-3 eval (https://forgejo.coilysiren.me/coilyco-flight-deck/infrastructure/issues/278), the first build-agent step took ~64s, much of it cold model load. Keeping `qwen3.5:9b-mlx` warm between requests removes that cold-start on back-to-back agent turns. Per-request `keep_alive` from the client would reset to the server default each call, so the durable fix is the server-side `OLLAMA_KEEP_ALIVE` env default. ## Change - New `ollama_keep_alive` default (`30m`) + `ollama_plist` path in the role defaults. - Role pins `OLLAMA_KEEP_ALIVE` into the launchd plist `EnvironmentVariables` via `PlistBuddy` (additive - leaves `OLLAMA_FLASH_ATTENTION`/`OLLAMA_KV_CACHE_TYPE` alone) and `launchctl bootout`/`bootstrap`-reloads only on drift. Idempotent: re-converge reports `changed=0`.

coilysiren referenced this issue from a commit

2026-06-09 04:24:36 +00:00

feat(ollama): keep the loaded model warm for 30m via OLLAMA_KEEP_ALIVE

coilysiren closed this issue

2026-06-09 04:24:36 +00:00

coilysiren referenced this issue

2026-06-09 05:01:01 +00:00

Managed OLLAMA_DEBUG toggle for per-request context tracing #280