Converge the restic backup plane on kai-server and ser8 #729

New issue

Closed

opened 2026-08-01 17:59:07 +00:00 by coilyco-ops · 1 comment

coilyco-ops commented

2026-08-01 17:59:07 +00:00

Member

Context

The live 2026-08-01 disk-pressure investigation confirmed that restic-backup.service and its timer are absent on kai-server, while restic-rest-server.service is absent on ser8. The repository already owns scripts, units, and a documented topology, but Ansible does not install or enable them. This blocks the backup gate for the scoped Forgejo package retention run. Parent recovery and restore-drill work remains #599.

Work

Add an idempotent Ansible role for the existing restic server, prune timer, backup client, and backup timer topology.
Install restic from the distro package on both hosts and a pinned, checksummed upstream rest-server binary on ser8.
Stage the SSM-backed repository password without exposing it in logs.
Initialize the repository only when absent.
Keep the client backup service manual during convergence so the first large snapshot is an observed operator step.
Document the rollout and validation commands.

Done when

Focused tests and repository validation pass.
The role lands on main.
An attended check/apply can partition ser8 before kai-server.

The live first snapshot, freshness monitoring, and isolated restore drill remain on #599.

## Context The live 2026-08-01 disk-pressure investigation confirmed that restic-backup.service and its timer are absent on kai-server, while restic-rest-server.service is absent on ser8. The repository already owns scripts, units, and a documented topology, but Ansible does not install or enable them. This blocks the backup gate for the scoped Forgejo package retention run. Parent recovery and restore-drill work remains #599. ## Work * Add an idempotent Ansible role for the existing restic server, prune timer, backup client, and backup timer topology. * Install restic from the distro package on both hosts and a pinned, checksummed upstream rest-server binary on ser8. * Stage the SSM-backed repository password without exposing it in logs. * Initialize the repository only when absent. * Keep the client backup service manual during convergence so the first large snapshot is an observed operator step. * Document the rollout and validation commands. ## Done when * Focused tests and repository validation pass. * The role lands on main. * An attended check/apply can partition ser8 before kai-server. The live first snapshot, freshness monitoring, and isolated restore drill remain on #599.

coilyco-ops added the

autonomy

live-collab

label

2026-08-01 17:59:07 +00:00

coilyco-ops referenced this issue from a commit

2026-08-01 18:09:07 +00:00

fix(backup): converge the restic backup plane

coilyco-ops commented

2026-08-01 18:09:30 +00:00

Author

Member

Landed on main as 60967a2. Focused backup tests, Ansible production-profile lint, the full pre-commit suite, and the push-triggered Forgejo Actions checks passed. The live first snapshot and restore drill remain on #599.

Landed on main as 60967a2. Focused backup tests, Ansible production-profile lint, the full pre-commit suite, and the push-triggered Forgejo Actions checks passed. The live first snapshot and restore drill remain on #599.

coilyco-ops closed this issue

2026-08-01 18:09:36 +00:00

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

coilyco-flight-deck/infrastructure#729

No description provided.

Rows
Columns