Daily DB backup cache with hard byte-cap eviction #7

New issue

Open

opened 2026-05-23 20:54:26 +00:00 by coilysiren · 0 comments

coilysiren commented

2026-05-23 20:54:26 +00:00

Owner

Originally filed by @coilysiren on 2026-05-22T20:33:07Z - https://github.com/coilysiren/infrastructure/issues/284

Goal - stand up a poor-girl's daily database backup cache with a hard total-size cap on the /backups folder. Three databases, one folder, a moving window of backups bounded by total bytes - oldest files evicted once the folder exceeds the budget.

Plan - hard byte cap

Three backup cron jobs (one per DB) write timestamped dumps into /backups. A fourth cron step runs an eviction loop that deletes the oldest files until the folder is back under the byte budget:

cd /backups || exit 1
while [ "$(du -sb . | cut -f1)" -gt $((5 * 1024 * 1024 * 1024)) ]; do
    oldest=$(ls -t | tail -1)
    [ -z "$oldest" ] && break
    rm -- "$oldest"
done

That is the whole "poor-girl's cache eviction" - LRU-by-mtime against a fixed total-size budget. Wrap it as one systemd service plus a timer, or fold it into the same unit that fires the backups so ordering is guaranteed (backups first, eviction last).

Considered and declined - logrotate

Logrotate was the first instinct, since "rotate a folder of backups" sounds like exactly its job. It is not. Logrotate prunes by count (rotate N) or age (maxage D), both per-file. It has no total-folder-size budget anywhere in its config language. size / maxsize only gate when a single file rotates, not a directory total.

Logrotate could give "keep 7 days per DB", but that is not the requested semantic. The ask is a hard byte cap, and logrotate cannot express it - so we would be hand-rolling the eviction logic regardless. At that point the du/ls/rm loop is simpler than bending logrotate around a feature it does not have. Declined.

Decision at implementation time

Pick the byte budget (placeholder above is 5 GiB) and where the eviction step lives - its own timer, or appended to the backup unit. Prefer appending to the backup unit so backups always land before eviction runs.

Part of the Infrastructure Reboot - no rush, file-and-defer.

_Originally filed by @coilysiren on 2026-05-22T20:33:07Z - [https://github.com/coilysiren/infrastructure/issues/284](https://github.com/coilysiren/infrastructure/issues/284)_ **Goal** - stand up a poor-girl's daily database backup cache with a **hard total-size cap** on the `/backups` folder. Three databases, one folder, a moving window of backups bounded by total bytes - oldest files evicted once the folder exceeds the budget. ## Plan - hard byte cap Three backup cron jobs (one per DB) write timestamped dumps into `/backups`. A fourth cron step runs an eviction loop that deletes the oldest files until the folder is back under the byte budget: ```sh cd /backups || exit 1 while [ "$(du -sb . | cut -f1)" -gt $((5 * 1024 * 1024 * 1024)) ]; do oldest=$(ls -t | tail -1) [ -z "$oldest" ] && break rm -- "$oldest" done ``` That is the whole "poor-girl's cache eviction" - LRU-by-mtime against a fixed total-size budget. Wrap it as one systemd service plus a timer, or fold it into the same unit that fires the backups so ordering is guaranteed (backups first, eviction last). ## Considered and declined - logrotate Logrotate was the first instinct, since "rotate a folder of backups" sounds like exactly its job. It is not. Logrotate prunes by **count** (`rotate N`) or **age** (`maxage D`), both per-file. It has **no total-folder-size budget** anywhere in its config language. `size` / `maxsize` only gate when a single file rotates, not a directory total. Logrotate could give "keep 7 days per DB", but that is not the requested semantic. The ask is a hard byte cap, and logrotate cannot express it - so we would be hand-rolling the eviction logic regardless. At that point the `du`/`ls`/`rm` loop is simpler than bending logrotate around a feature it does not have. Declined. ## Decision at implementation time Pick the byte budget (placeholder above is 5 GiB) and where the eviction step lives - its own timer, or appended to the backup unit. Prefer appending to the backup unit so backups always land before eviction runs. Part of the Infrastructure Reboot - no rush, file-and-defer.

coilysiren referenced this issue

2026-05-23 20:54:36 +00:00

factorio: build Sirens-authored mod stack (mid/long-term) #68

coilysiren added the

label

2026-05-31 01:54:51 +00:00

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

coilyco-flight-deck/infrastructure#7

No description provided.

Rows
Columns