Turn disk-pressure and Forgejo-storage runbooks into an Ops skill family #703

Closed
opened 2026-07-31 08:53:40 +00:00 by coilyco-ops · 0 comments
Owner

Outcome

Make disk consumption, root filesystem pressure, Kubernetes DiskPressure, and Forgejo storage incidents directly discoverable from one operator skill family instead of requiring prior knowledge of several docs filenames.

Depends on the canonical provider layout in #702

Source material

The family must reconcile the operational guidance currently spread across:

  • docs/kai-disk-pressure-watch.md
  • docs/kai-disk-pressure-drill.md
  • docs/kai-disk-pressure-forgejo-pods.md
  • docs/forgejo-pvc-retention-audit.md
  • docs/forgejo-pvc-retention-checklist.md
  • docs/k3s-deploy-notes-traps-forgejo-storage.md
  • docs/ansible-kai-disk-guardrails-role.md
  • skills/ops-investigation-k3s-pod-eviction/SKILL.md

Skill shape

  • Add a focused disk-pressure investigation skill with aliases for disk usage, disk full, rootfs pressure, inode pressure, Kubernetes DiskPressure, ephemeral storage, ImageGCFailed, FreeDiskSpaceFailed, local-path PVC growth, containerd growth, runner scratch, and Forgejo storage.
  • Keep the SKILL.md body short. It should establish the read-only classification order, the safety boundary, and route to focused references.
  • Consolidate branching runbook content under the skill references where that removes duplicate truth.
  • Keep feature and convergence explanation as docs when it is not an operator workflow. Link it from the skill instead of copying it.
  • Route pod-eviction cases cleanly between this family and the existing k3s pod-eviction skill.
  • Preserve the safe relief order: measure first, distinguish bytes from inodes, attribute ownership, protect durable PVC and game data, and leave cleanup attended.
  • Make rollback and after-state verification explicit for every mutating relief path.
  • Update docs navigation and FEATURES without retaining duplicated procedural copies.

Acceptance

  • A prompt about ordinary disk usage or an 85 percent rootfs alert discovers the new skill without mentioning Kubernetes or Forgejo.
  • DiskPressure and pod-eviction prompts route to the correct peer skill without conflicting instructions.
  • The references cover host logs, journald, containerd, runner scratch, local-path PVCs, Forgejo data, and game saves.
  • The skill never recommends arbitrary deletion or truncation.
  • Read-only diagnosis is clearly separated from attended cleanup.
  • Every retained docs page has one clear purpose and no second copy of the same procedure.
  • Existing disk-pressure watcher, guardrail, and telemetry behavior remains accurately documented.
  • Skill validation, dead-link checks, and pre-commit run --all-files pass.
## Outcome Make disk consumption, root filesystem pressure, Kubernetes DiskPressure, and Forgejo storage incidents directly discoverable from one operator skill family instead of requiring prior knowledge of several docs filenames. Depends on the canonical provider layout in https://forgejo.coilysiren.me/coilyco-flight-deck/infrastructure/issues/702 ## Source material The family must reconcile the operational guidance currently spread across: * `docs/kai-disk-pressure-watch.md` * `docs/kai-disk-pressure-drill.md` * `docs/kai-disk-pressure-forgejo-pods.md` * `docs/forgejo-pvc-retention-audit.md` * `docs/forgejo-pvc-retention-checklist.md` * `docs/k3s-deploy-notes-traps-forgejo-storage.md` * `docs/ansible-kai-disk-guardrails-role.md` * `skills/ops-investigation-k3s-pod-eviction/SKILL.md` ## Skill shape * Add a focused disk-pressure investigation skill with aliases for disk usage, disk full, rootfs pressure, inode pressure, Kubernetes DiskPressure, ephemeral storage, ImageGCFailed, FreeDiskSpaceFailed, local-path PVC growth, containerd growth, runner scratch, and Forgejo storage. * Keep the SKILL.md body short. It should establish the read-only classification order, the safety boundary, and route to focused references. * Consolidate branching runbook content under the skill references where that removes duplicate truth. * Keep feature and convergence explanation as docs when it is not an operator workflow. Link it from the skill instead of copying it. * Route pod-eviction cases cleanly between this family and the existing k3s pod-eviction skill. * Preserve the safe relief order: measure first, distinguish bytes from inodes, attribute ownership, protect durable PVC and game data, and leave cleanup attended. * Make rollback and after-state verification explicit for every mutating relief path. * Update docs navigation and FEATURES without retaining duplicated procedural copies. ## Acceptance * A prompt about ordinary disk usage or an 85 percent rootfs alert discovers the new skill without mentioning Kubernetes or Forgejo. * DiskPressure and pod-eviction prompts route to the correct peer skill without conflicting instructions. * The references cover host logs, journald, containerd, runner scratch, local-path PVCs, Forgejo data, and game saves. * The skill never recommends arbitrary deletion or truncation. * Read-only diagnosis is clearly separated from attended cleanup. * Every retained docs page has one clear purpose and no second copy of the same procedure. * Existing disk-pressure watcher, guardrail, and telemetry behavior remains accurately documented. * Skill validation, dead-link checks, and pre-commit run --all-files pass.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
coilyco-flight-deck/infrastructure#703
No description provided.