feat(ci): automatically deploy published Mission Control releases #37
No reviewers
Labels
No labels
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
ai/mission-control!37
Loading…
Reference in a new issue
No description provided.
Delete branch "mc/93f5be9d-af49-40ca-bae6-2176088b7802"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Publishing a stable Mission Control release updates the server and agent digest pins together in Forgejo homelab/apps main, removing the manual deployment bump. Both image workflows must succeed for the same resolved source SHA. The updater validates published release metadata, tag/main ancestry and immutable linux/amd64 digests, renders the fresh chart, then writes only image.tag and agentImage in one commit.
A normal non-force Git push guards the entire rendered main revision. Any competing main commit, including globals/template changes with an unchanged values blob, causes a fresh read and render. Retries are bounded to three writes; older releases skip, identical pairs are no-ops, and same-version changed digests fail.
The authorized existing Renovate token is injected only into trusted updater/access steps. Fixed-origin metadata rejects redirects; only the fixed Git push receives a URL-scoped authentication header through its environment. Credentials stay out of arguments, URLs, files and output. New steps have bounded CPU, memory and ephemeral storage.
Evidence
b44086e: CI188/190 green. DevOps/Security found no blocking implementation findings; SRE identified the revision race, now fixed ind1f4c7f. Fresh same-head CI and all three reviews are required before merge.Merge danger
Two-way for the workflow: revert disables future updates. Already-produced apps commits require their own Git revert; published releases are irreversible. Future qualifying releases restart the existing single SQLite server using Recreate. No migration, storage, exposure or credential change.
Verification and rollback
After merge: green main build and access-verified, followed by an approved real release. Verify both same-SHA image publications, updater success, exact two-field apps commit, mirror, ArgoCD expected revision Synced/Healthy, actual server/agent digests, readiness/health and a newly launched agent. Live acceptance remains pending.
Rollback: pause/drain updater runs, revert only the apps pair preserving unrelated commits, follow mirror/ArgoCD/workload back to known-good digests; revert this PR to disable the updater. No automatic restore. Software Engineer retains implementation and rollout ownership.
Security re-review pinned to
d1f4c7fd02, increment b44086e..d1f4c7f and full origin/main...HEAD. Reviewed confirmed spec aefe6183-b67c-4604-8694-67759f92855f, change-review (including smell baseline), Homelab AGENTS.md/security-operations/security-perimeter and apps AGENTS.md. Mission Control has no applicable root AGENTS.md/CONTRIBUTING/CODING_STANDARDS in this checkout. Separate review/security-d1f4c7f branch; author checkout untouched.Standards
No blocking findings.
Spec
No blocking findings.
Evidence and limits
Independent anonymous CI API read: GET https://woodpecker.olsen.cloud/api/repos/32/pipelines/195 -> number195, status success, commit
d1f4c7fd02; docs/image-agent/image/quality all success. Owner parent evidence a795f7be-bab9-44cf-89b7-5c15067a3f1d records 80 files/685 tests and focused20/strict workflow checks in handoff. Local focused commandpnpm test scripts/deploy-release.test.ts scripts/deploy-release-git.test.tsnot rerun: pnpm and installed Vitest absent in this isolated checkout; exact-head CI success was independently seen. Static code/test review plus independent CI/live baseline, not live-release acceptance.Independent
kubectl get deployment mission-control -n prod -o jsonpath='{.status.readyReplicas}/{.spec.replicas}'-> 1/1.kubectl get application mission-control -n argocd -o jsonpath='{.status.sync.revision} {.status.sync.status} {.status.health.status}'-> f1f2b38909bc9d643f0f1be23c11d4382788f419 Synced Healthy. No mutation, logs, Secrets or ConfigMaps read. Git diff --check passed; checkout clean.Required live acceptance and rollback
Software Engineer retains parent. After other required validators finish, verify green trusted-main access step; obtain Morten's explicit first migration/release approval. Follow one published eligible release: both successful same-source image workflows and registry digests, updater status/revision, exact two-field commit, preserved unrelated changes, mirror, ArgoCD expected revision, actual paired image digests, readiness/health and newly launched agent. Report failures without raw credential-bearing subprocess output.
Rollback: pause/drain updater runs, normal GitOps revert of just the pair preserving intervening commits, follow mirror/ArgoCD and workload back to known-good compatible digests; revert workflow to disable future updates if needed. Data/schema restore or migration choices stay with Morten.
Standards: 0 blocking, 2 non-blocking (worst HIGH approved authority context). Spec: 0 blocking, 1 non-blocking (worst UNKNOWN live acceptance).
Review by the Security Engineer role, an agent in Mission Control, for the task Improve mission control deployment.
DevOps re-review of PR37 at
d1f4c7fd02, specifically b44086e..d1f4c7f; also reconfirmed the full release workflow/policy against confirmed specification aefe6183-b67c-4604-8694-67759f92855f. Own clean checkout: review/devops-d1f4c7f. Read Homelab AGENTS.md (Git and deployment/boundaries), current apps origin/main AGENTS.md, change-review (including smells), delivery-path and shared CI skill. Mission Control has no root AGENTS.md or CONTRIBUTING.md.Standards
No blocking findings. The new updater runner is digest pinned with resource bounds; workflow/token documentation changed with transport. Push is fixed to https://code.olsen.cloud/homelab/apps.git and HEAD:refs/heads/main, with no force, redirect following, credential helper or hooks. Authenticated /api/v1/user supplies the actual PAT login for Basic Git auth and commit identity. Ordinary children strip DEPLOY_TOKEN and inherited GIT_*; push alone receives a URL-scoped header via environment, with captured/sanitized errors. Only the values file is staged and committed. No GitHub remote/write, chart/sync-wave/prune change or cluster credentials added. Existing latest-tag image build plugins are unchanged by this increment; their reproducibility remains the existing platform limitation, not a newly introduced transport defect.
Validation: independently ran woodpecker-cli lint --strict .woodpecker/*.yaml: all 11 valid. HEAD verified
d1f4c7f; git status clean. Static review plus owner-confirmed build: parent comment a795f7be-bab9-44cf-89b7-5c15067a3f1d confirms CI195 PR SUCCESS (quality/docs/server image/agent image) and CI194 branch SUCCESS at this exact SHA, 80 files/685 tests, plus 20 focused tests/local quality and 55 event renderings. I could not independently fetch CI195: curl failed to connect to woodpecker.olsen.cloud:443. Focused local test command mise exec -- pnpm exec vitest run scripts/deploy-release.test.ts scripts/deploy-release-git.test.ts did not execute tests: pnpm unavailable and project node_modules absent. Do not represent these as independently run/passed. Owner must reconfirm unchanged PR head and green CI immediately before merge.Spec
No blocking findings. Commit checks both precommit HEAD==snapshot.revision and postcommit HEAD^==snapshot.revision; ordinary non-force push rejects an intervening main revision even when the values blob is unchanged. Fresh fetch/reset/replan/render on rejection preserves unrelated commits. Real bare-Git test introduces exactly that globals-only race and checks rerender/new parent, retained competitor and no credential in config/log/arguments. Synthetic denied/uncertain transport tests complement policy tests covering three-write bound, final reread, already-current/superseded recovery and visible exhaustion.
Reconfirmed unchanged gates: deploy depends on both tag image workflows in the same pipeline/SHA; canonical stable tag, published non-draft/non-prerelease release, exact release/tag/CI source and main ancestry checks, recheck before each write, linux/amd64 manifest/config digest checks, forward-only numeric ordering, identical no-op and same-version changed-digest failure. Only image.tag/agentImage change, with preserved formatting and render validation using the exact target revision's globals. Binary assets do not gate deployment.
Non-blocking acceptance clarification: trusted-main --check-access proves API-reported homelab/apps push permission, not successful Git HTTP authentication or branch-policy acceptance. Code now obtains the real identity, but synthetic transport regressions cannot prove the production token/Forgejo combination. Require the trusted-main access workflow to succeed before publishing the first release; record actual Git transport success during the approved real release. No secret/permission widening or bypass is authorized.
Required live acceptance (Software Engineer owns): obtain Morten's explicit first-release and migration026 approval (main independently advanced to
2b87a06); quote both tag publication results and updater outcome, resolved digest pair, exact apps commit/parent/two-key diff, Forgejo push-mirror target revision/error and GitHub matching revision, ArgoCD expected revision plus actual Deployment server image/MC_AGENT_IMAGE, readiness/health and newly launched agent digest. Verify no manual Renovate/values edit was needed. Owner baseline remains server0.13.0 ready1/1, ArgoCD f1f2b389 Synced Healthy; no live release acceptance has occurred, and I have no private cluster access. Healthy alone is insufficient. The root Helm release's effective globals must match the renderer's Git globals for acceptance; apps/root changes require their separately controlled Helm rollout per delivery-path.Rollback: pause/drain updater runs; revert only the image pair to recorded known-good digests through normal Forgejo GitOps, preserving intervening commits; follow mirror/ArgoCD and inspect actual fields/health. Disable faulty updater separately if needed; reverting workflow does not undo an existing apps commit. Resume only after delayed newer runs cannot reverse rollback. Migration/data compatibility or restore decisions stay with Morten.
Standards: 0 blocking findings; existing plugin pinning limitation noted. Spec: 0 blocking findings, 1 non-blocking acceptance clarification (API access verification is not actual Git push proof).
Review by the DevOps Engineer role, an agent in Mission Control, for the task Improve mission control deployment.