coilyco-flight-deck/infrastructure
1
0
Fork 0
Code Issues 101 Pull requests Projects Releases Packages Wiki Activity Actions 1

Self-hosted Backstage on kai-server #94

New issue
Closed
opened 2026-05-23 20:54:41 +00:00 by coilysiren · 2 comments
coilysiren commented 2026-05-23 20:54:41 +00:00
Owner
Copy link

Originally filed by @coilysiren on 2026-04-14T09:22:54Z - https://github.com/coilysiren/infrastructure/issues/63

Self-hosted Backstage on kai-server

Goal

Stand up a personal Backstage instance at backstage.coilysiren.me to serve as a dev portal + TechDocs host for ~79 public coilysiren repos. First step toward getting acquainted with Backstage (and later, Dagger) as upstream OSS contribution targets.

Topology

  • Runtime: existing single-node k3s on kai-server
  • Network: public, TLS via existing cert-manager + Route53 DNS-01 + Traefik
  • Database: Postgres on the host via systemd (apt install postgresql-16), not in-cluster
  • TechDocs storage: local filesystem on the k3s node (hostPath or PVC on local-path-provisioner) — TechDocs is a cache, source of truth is git, rebuild is cheap
  • Auth: GitHub OAuth, single-user allowlist (coilysiren)
  • GitHub API credential: existing PAT stored in AWS SSM, pulled into the cluster by external-secrets
  • Catalog strategy: GitHub org discovery against github.com/coilysiren, bootstrapped by a batch PR adding catalog-info.yaml stubs to all repos so the portal is non-empty on day one
  • v1 plugin scope: core catalog, GitHub discovery/integration, TechDocs, GitHub OAuth, Software Templates (plugin enabled, no templates yet). Explicitly not in v1: Kubernetes plugin, observability integrations, any custom plugin.

Repo layout

Two repos touched. One new, one existing.

coilysiren/backstage (new)

Self-contained, mirrors the coilysiren/backend pattern exactly.

  • Output of npx @backstage/create-app at the root
  • Dockerfile — standard multi-stage Backstage build
  • Makefile with .build-docker and .deploy targets
  • app-config.yaml (committed, non-secret) + app-config.production.yaml (references env vars)
  • deploy/ — raw k8s manifests:
    • namespace.yaml
    • deployment.yaml (references ghcr.io/coilysiren/backstage:\$SHA)
    • service.yaml
    • ingressroute.yaml (Traefik) for backstage.coilysiren.me
    • certificate.yaml (cert-manager)
    • externalsecret.yaml — pulls GitHub PAT, GitHub OAuth client ID/secret, Postgres password from SSM
    • endpoints.yaml — headless Service + manual Endpoints pointing at the host IP so Backstage reaches host Postgres via a stable DNS name (postgres.backstage.svc.cluster.local)
  • .github/workflows/build-and-publish.yml — copy-pasted from coilysiren/backend:
    • Triggers on push to main
    • Builds Docker image, tags with commit SHA, pushes to public GHCR
    • Tailscale OAuth → install kubectl → build kubeconfig from secrets → make .deploy

coilysiren/infrastructure (this repo — minimal addition)

Two things added. Nothing else touched.

  1. Host Postgres install + bootstrap — new systemd unit and/or invoke task that:
    • Installs postgresql-16 via apt (idempotent)
    • Creates a backstage role and backstage database
    • Pulls the role password from SSM and sets it via ALTER ROLE
    • Configures pg_hba.conf / postgresql.conf to accept connections from the k3s pod CIDR on the host interface
  2. Invoke task: inv backstage.db-init (idempotent) that runs the above. Run once by hand from kai-server. No automation for v1.

Backups deferred. Not in scope for v1. Revisit once the portal has any state worth preserving beyond the re-ingestable catalog.

Secrets

All runtime secrets live in AWS SSM and are pulled into the cluster via the existing external-secrets operator into the backstage namespace:

  • github-pat — existing PAT, used for catalog discovery, TechDocs source fetching, rate-limit-friendly API access
  • github-oauth-client-id / github-oauth-client-secret — new GitHub OAuth app for single-user login
  • postgres-password — shared between the host Postgres role and the Backstage pod's DATABASE_PASSWORD env var

GitHub Actions secrets (for the deploy job, matching backend): KUBE_SERVER, KUBE_CA, KUBE_CLIENT_CERT, KUBE_CLIENT_KEY, TS_OAUTH_CLIENT_ID, TS_OAUTH_SECRET.

Catalog bootstrap

One-time scripted batch PR across the coilysiren org:

  • Script iterates public repos, generates a minimal catalog-info.yaml per repo (name, description from repo metadata, owner coilysiren, type inferred from primary language — library, service, website)
  • Opens a PR per repo (or commits directly since it's a personal org)
  • Org discovery picks them up automatically

Good candidate for being packaged as a Dagger module afterward — reusable, generates blog content, and doubles as the first concrete "Backstage + Dagger working together" artifact.

Out of scope for v1

  • Postgres backups
  • Any custom Backstage plugin
  • Kubernetes plugin / cluster visibility inside Backstage
  • Software template content (plugin enabled, zero templates)
  • Observability integrations
  • Multi-user auth or RBAC
  • S3 TechDocs (reconsider only if filesystem storage becomes a problem)
  • Changes to coilysiren/infrastructure beyond the Postgres bootstrap task

Definition of done

  • backstage.coilysiren.me resolves and serves Backstage over TLS
  • GitHub OAuth login works for user coilysiren
  • Catalog page shows >50 entities from org discovery
  • TechDocs renders mkdocs content from at least one repo
  • git push to coilysiren/backstage main deploys automatically via the existing backend-style workflow
  • Host Postgres is reachable from the cluster and survives a k3s restart unchanged
  • At least one blog post on coilysiren.me links to a TechDocs page and the link resolves
_Originally filed by @coilysiren on 2026-04-14T09:22:54Z - [https://github.com/coilysiren/infrastructure/issues/63](https://github.com/coilysiren/infrastructure/issues/63)_ # Self-hosted Backstage on kai-server ## Goal Stand up a personal Backstage instance at `backstage.coilysiren.me` to serve as a dev portal + TechDocs host for ~79 public `coilysiren` repos. First step toward getting acquainted with Backstage (and later, Dagger) as upstream OSS contribution targets. ## Topology - **Runtime:** existing single-node k3s on kai-server - **Network:** public, TLS via existing cert-manager + Route53 DNS-01 + Traefik - **Database:** Postgres on the host via systemd (`apt install postgresql-16`), *not* in-cluster - **TechDocs storage:** local filesystem on the k3s node (hostPath or PVC on local-path-provisioner) — TechDocs is a cache, source of truth is git, rebuild is cheap - **Auth:** GitHub OAuth, single-user allowlist (`coilysiren`) - **GitHub API credential:** existing PAT stored in AWS SSM, pulled into the cluster by external-secrets - **Catalog strategy:** GitHub org discovery against `github.com/coilysiren`, bootstrapped by a batch PR adding `catalog-info.yaml` stubs to all repos so the portal is non-empty on day one - **v1 plugin scope:** core catalog, GitHub discovery/integration, TechDocs, GitHub OAuth, Software Templates (plugin enabled, no templates yet). Explicitly *not* in v1: Kubernetes plugin, observability integrations, any custom plugin. ## Repo layout Two repos touched. One new, one existing. ### `coilysiren/backstage` (new) Self-contained, mirrors the `coilysiren/backend` pattern exactly. - Output of `npx @backstage/create-app` at the root - `Dockerfile` — standard multi-stage Backstage build - `Makefile` with `.build-docker` and `.deploy` targets - `app-config.yaml` (committed, non-secret) + `app-config.production.yaml` (references env vars) - `deploy/` — raw k8s manifests: - `namespace.yaml` - `deployment.yaml` (references `ghcr.io/coilysiren/backstage:\$SHA`) - `service.yaml` - `ingressroute.yaml` (Traefik) for `backstage.coilysiren.me` - `certificate.yaml` (cert-manager) - `externalsecret.yaml` — pulls GitHub PAT, GitHub OAuth client ID/secret, Postgres password from SSM - `endpoints.yaml` — headless Service + manual Endpoints pointing at the host IP so Backstage reaches host Postgres via a stable DNS name (`postgres.backstage.svc.cluster.local`) - `.github/workflows/build-and-publish.yml` — copy-pasted from `coilysiren/backend`: - Triggers on push to `main` - Builds Docker image, tags with commit SHA, pushes to public GHCR - Tailscale OAuth → install kubectl → build kubeconfig from secrets → `make .deploy` ### `coilysiren/infrastructure` (this repo — minimal addition) Two things added. Nothing else touched. 1. **Host Postgres install + bootstrap** — new systemd unit and/or invoke task that: - Installs `postgresql-16` via apt (idempotent) - Creates a `backstage` role and `backstage` database - Pulls the role password from SSM and sets it via `ALTER ROLE` - Configures `pg_hba.conf` / `postgresql.conf` to accept connections from the k3s pod CIDR on the host interface 2. **Invoke task:** `inv backstage.db-init` (idempotent) that runs the above. Run once by hand from kai-server. No automation for v1. **Backups deferred.** Not in scope for v1. Revisit once the portal has any state worth preserving beyond the re-ingestable catalog. ## Secrets All runtime secrets live in AWS SSM and are pulled into the cluster via the existing external-secrets operator into the `backstage` namespace: - `github-pat` — existing PAT, used for catalog discovery, TechDocs source fetching, rate-limit-friendly API access - `github-oauth-client-id` / `github-oauth-client-secret` — new GitHub OAuth app for single-user login - `postgres-password` — shared between the host Postgres role and the Backstage pod's `DATABASE_PASSWORD` env var GitHub Actions secrets (for the deploy job, matching `backend`): `KUBE_SERVER`, `KUBE_CA`, `KUBE_CLIENT_CERT`, `KUBE_CLIENT_KEY`, `TS_OAUTH_CLIENT_ID`, `TS_OAUTH_SECRET`. ## Catalog bootstrap One-time scripted batch PR across the coilysiren org: - Script iterates public repos, generates a minimal `catalog-info.yaml` per repo (name, description from repo metadata, owner `coilysiren`, type inferred from primary language — `library`, `service`, `website`) - Opens a PR per repo (or commits directly since it's a personal org) - Org discovery picks them up automatically Good candidate for being packaged as a Dagger module afterward — reusable, generates blog content, and doubles as the first concrete "Backstage + Dagger working together" artifact. ## Out of scope for v1 - Postgres backups - Any custom Backstage plugin - Kubernetes plugin / cluster visibility inside Backstage - Software template content (plugin enabled, zero templates) - Observability integrations - Multi-user auth or RBAC - S3 TechDocs (reconsider only if filesystem storage becomes a problem) - Changes to `coilysiren/infrastructure` beyond the Postgres bootstrap task ## Definition of done - [ ] `backstage.coilysiren.me` resolves and serves Backstage over TLS - [ ] GitHub OAuth login works for user `coilysiren` - [ ] Catalog page shows >50 entities from org discovery - [ ] TechDocs renders mkdocs content from at least one repo - [ ] `git push` to `coilysiren/backstage` main deploys automatically via the existing `backend`-style workflow - [ ] Host Postgres is reachable from the cluster and survives a `k3s` restart unchanged - [ ] At least one blog post on `coilysiren.me` links to a TechDocs page and the link resolves
coilysiren commented 2026-05-30 05:43:18 +00:00
Author
Owner
Copy link

Iceboxed in the 2026-05-29 backlog burn-down: self-host Backstage dev portal, speculative play. Reopen anytime if it becomes real.

Iceboxed in the 2026-05-29 backlog burn-down: self-host Backstage dev portal, speculative play. Reopen anytime if it becomes real.
coilysiren commented 2026-05-30 05:43:29 +00:00
Author
Owner
Copy link

Iceboxed in the 2026-05-29 backlog burn-down: duplicate of #48 backstage note; speculative dev portal. Reopen anytime if it becomes real.

Iceboxed in the 2026-05-29 backlog burn-down: duplicate of #48 backstage note; speculative dev portal. Reopen anytime if it becomes real.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
dispatch/issue-294
tangled-knot
dispatch/issue-216
No results found.
Labels
Clear labels
  
P0
now / blocking / urgent

  
P1
next / important

  
P2
backlog / will do

  
P3
someday / low

  
P4
icebox / frozen / reopen if it becomes real

No labels
P0
P1
P2
P3
P4
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
coilyco-flight-deck/infrastructure#94
No description provided.