fix: cleanup orphaned deploys when services are renamed or removed from config#140
Open
vigneshrajsb wants to merge 2 commits intomainfrom
Open
fix: cleanup orphaned deploys when services are renamed or removed from config#140vigneshrajsb wants to merge 2 commits intomainfrom
vigneshrajsb wants to merge 2 commits intomainfrom
Conversation
…om config Adds a DEPLOY_CLEANUP async queue and worker that tears down Kubernetes/Helm resources and marks DB deploy records inactive for services no longer present in lifecycle.yaml. Orphan detection runs after every upsertDeployablesWithDatabase call so renames and removals are caught on the next config push. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… state after patch When a service rename is reverted, an existing TORN_DOWN deploy was being reused without resetting its status, causing it to be skipped during re-deployment. Reset status to PENDING and active to true when a TORN_DOWN deploy is matched to an active deployable. After patching an existing deployable, refresh the in-memory object via $query() so the returned deployables array reflects the latest DB state rather than the pre-patch snapshot. Also add jest fallback env vars so tests don't fail at module load time, and expand deployCleanup test suite with mocks for nativeBuild and LogArchivalService. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When a service is renamed or removed from
lifecycle.yaml, the old service's Kubernetes resources (deployments, services, ingress) and DB deploy records were left behind indefinitely. This caused:Fix
Orphan detection is now run after every
upsertDeployablesWithDatabase()call. AnyDeployablepresent in the database for the current build that is no longer referenced in the incoming config is identified as orphaned and enqueued for async teardown.Changes
src/shared/config.ts— AddedDEPLOY_CLEANUPqueue name constantsrc/server/services/deploy.ts— AddeddeployCleanupQueue,enqueueDeployCleanup(), andprocessDeployCleanupQueue()toDeployService. The processor deletes Kubernetes resources viakubectl delete(for Docker/GitHub deploys) orhelm uninstall(for Helm deploys), then marks the deploy record inactive in the DBsrc/server/jobs/index.ts— Registered theDEPLOY_CLEANUPBullMQ workersrc/server/services/deployable.ts— AddedcleanupOrphanedDeploys()which queries existing deploys for the build, diffs against the current set of upserted deployables, and callsenqueueDeployCleanup()for each orphan. WhenfilterGithubRepositoryIdis set (push webhook for a specific repo), the orphan query is scoped to that repo only — preventing services from other repos being falsely flagged as orphans on a filtered runsrc/server/services/__tests__/deployCleanup.test.ts— Unit tests covering orphan detection logic and the cleanup worker (skipping already-torn-down deploys, Helm uninstall path, kubectl delete path, DB record deactivation, and filtered repo scoping)Bug Fix (same PR)
Orphan cleanup incorrectly tore down valid deploys on filtered builds. When a push webhook fires for a single repo,
upsertDeployablesonly processes services for that repo (remote services from other repos are intentionally skipped). The originalcleanupOrphanedDeployscompared against all deployables in the DB, causing services from non-triggering repos to appear as orphans. Fixed by scoping the orphan check torepositoryId = filterGithubRepositoryIdwhen the filter is active.Testing
Unit tests added in
deployCleanup.test.ts. Live DB/Redis integration tests were intentionally skipped per project convention for this PR — the cleanup path can be exercised manually by renaming a service inlifecycle.yamland pushing a new commit against an active ephemeral environment.🤖 Generated with Claude Code