fix: protect Authentik server startup after database recovery #1

Merged

morten-olsen merged 3 commits from fix/3f88f077-startup-probe into main

2026-10-24 20:58:07 +02:00

morten-olsen commented

2026-10-24 09:09:51 +02:00

Owner

Summary

Give Authentik server ten minutes to start after PostgreSQL and its migration worker recover from a reboot. Kubernetes gates the existing liveness/readiness probes on an HTTP startup probe: /-/health/live/, port 9000, 60 × 10 seconds, five-second timeout.

server start → startup succeeds → normal liveness/readiness
             ↳ up to 600s before restart

The main release workflow publishes the explicitly reviewed chart version 0.1.2 and refuses an existing tag. This avoids deriving the already published 0.1.0 from stale v0.0.15 tag history. Release/fixture documentation changes in the same commits as their behavior.

Part of HOMEIT-368. Deploy postgres ownership repair first; base documentation/pointer follows this PR.

Evidence

Before: live server has no startupProbe; the reconciliation regression fails with “server needs an HTTP startup probe”. After: creation and repeated reconciliation tests pass.
CI is green at a8c93fc: unit tests, lint, Kind E2E. All four E2E specs pass; the run builds/starts the operator image, verifies the generated startup budget and reconciles a real server/client.
E2E fixes: fixtures pin the homelab's existing 2026.8.3 image instead of an unavailable latest tag, check the actual HTTPS URL contract, and remove CRs before the operator so finalizers can finish. Cleanup waits are bounded.
Full local make test/build and compatible Go 1.25 lint pass. Helm lint/render, chart API server dry-run and proposed server probe dry-run pass. Release workflow actionlint, Bash syntax/execution and version extraction (0.1.2) pass.
No Woodpecker registration exists; existing mirror CI performed the Docker/Kind run because Docker is unavailable locally. Nothing has been deployed to the homelab and no host reboot was exercised.

Merge danger

Door: one-way publication. The mirror's main workflow automatically publishes v0.1.2, multiarchitecture images, a release, install manifest and Helm chart. Morten's release approval is required before merge. Publication does not change core's live pin.

Blast radius: household. Deploying the new operator reconciles Authentik and rolls its server, which provides login to NetBird and Coder. The workload change is reversible.

Rollback and verification

Roll back: restore core's previous operator chart/image (0.1.0), then revert this code for later releases. Published artifacts remain.
Verify after rollout: actual operator image digest, server startupProbe and ready replicas; no database lock queues; Authentik live/ready and actual NetBird OIDC discovery HTTP 200; NetBird management/Coder available.
Reviewers: self-review; no auth policy, RBAC, exposure, secret or image-source expansion. No Designer sign-off requested.

## Summary Give Authentik server ten minutes to start after PostgreSQL and its migration worker recover from a reboot. Kubernetes gates the existing liveness/readiness probes on an HTTP startup probe: `/-/health/live/`, port 9000, 60 × 10 seconds, five-second timeout. ```text server start → startup succeeds → normal liveness/readiness ↳ up to 600s before restart ``` The main release workflow publishes the explicitly reviewed chart version **0.1.2** and refuses an existing tag. This avoids deriving the already published 0.1.0 from stale v0.0.15 tag history. Release/fixture documentation changes in the same commits as their behavior. Part of HOMEIT-368. Deploy [postgres ownership repair](https://code.olsen.cloud/homelab/postgres-operator/pulls/1) first; [base documentation/pointer](https://code.olsen.cloud/homelab/base/pulls/57) follows this PR. ## Evidence - Before: live server has no startupProbe; the reconciliation regression fails with “server needs an HTTP startup probe”. After: creation and repeated reconciliation tests pass. - CI is green at **a8c93fc**: [unit tests](https://github.com/morten-olsen/homelab-authentik-operator/actions/runs/40117312983), [lint](https://github.com/morten-olsen/homelab-authentik-operator/actions/runs/40117313040), [Kind E2E](https://github.com/morten-olsen/homelab-authentik-operator/actions/runs/40117312981). All four E2E specs pass; the run builds/starts the operator image, verifies the generated startup budget and reconciles a real server/client. - E2E fixes: fixtures pin the homelab's existing 2026.8.3 image instead of an unavailable latest tag, check the actual HTTPS URL contract, and remove CRs before the operator so finalizers can finish. Cleanup waits are bounded. - Full local make test/build and compatible Go 1.25 lint pass. Helm lint/render, chart API server dry-run and proposed server probe dry-run pass. Release workflow actionlint, Bash syntax/execution and version extraction (0.1.2) pass. - No Woodpecker registration exists; existing mirror CI performed the Docker/Kind run because Docker is unavailable locally. Nothing has been deployed to the homelab and no host reboot was exercised. ## Merge danger **Door:** one-way publication. The mirror's main workflow automatically publishes v0.1.2, multiarchitecture images, a release, install manifest and Helm chart. Morten's release approval is required before merge. Publication does not change core's live pin. **Blast radius:** household. Deploying the new operator reconciles Authentik and rolls its server, which provides login to NetBird and Coder. The workload change is reversible. ## Rollback and verification - **Roll back:** restore core's previous operator chart/image (0.1.0), then revert this code for later releases. Published artifacts remain. - **Verify after rollout:** actual operator image digest, server startupProbe and ready replicas; no database lock queues; Authentik live/ready and actual NetBird OIDC discovery HTTP 200; NetBird management/Coder available. - **Reviewers:** self-review; no auth policy, RBAC, exposure, secret or image-source expansion. No Designer sign-off requested.

morten-olsen added 1 commit

2026-10-24 09:09:51 +02:00