Repository navigation
fix(ci): deploy the merged commit's own image, not whatever :latest points at (CCP-5824) - #254
Merged
Merged
Conversation
…oints at (CCP-5824) diff_branch defaulted to main, so a push to main diffed main against main — always empty, so the build never triggered and the action retagged :latest onto :dev instead. :latest is whichever image was tagged most recently by any build, so the deployed image depended on timing. After #229 merged, DEV and TEST ran the #210 frontend and the run still went green. - merge.yml: diff_branch HEAD^ so only packages the merge touched rebuild, from the merge commit; tag_fallback dev so an untouched package keeps the image it had - pr-open.yml: same tag_fallback change — unchanged packages in a PR env were also being filled from :latest - Log per package whether the image was built or retagged, plus the digest Verified the HEAD^ diff against the last five merge commits: #245 frontend only, #242 backend+frontend, #244 all three.
NikhilMM89
force-pushed
the
fix/ccp-5824-merge-image-race
branch
from
October 2, 2026 17:59
4a3be5f to
aacac48
Compare
ContainerMemoryHigh fired on two prod frontend pods. The frontend container itself is nowhere near its limit (39-57Mi against 8Gi); it is the promtail sidecar enabled alongside it, sitting at 109Mi of a 128Mi limit - 85.2%, which is exactly the alert threshold. It is not log volume. access.log is 664KB after four days and promtail's positions file shows it fully consumed, so there is no backlog. 60-110Mi is simply promtail's resting Go footprint, and the chart's 128Mi left no headroom. Test shows the same spread, 55Mi and 103Mi. Nothing has OOMKilled yet. The reason to fix it ahead of that: the app container has its own limit, so an OOMKilled promtail costs no user-visible service - it just stops frontend logs reaching Loki and quietly blinds the frontend log alerts again. Requests go to 128Mi and limits to 256Mi, roughly 2.5x the observed peak. Costs 320Mi of requests across five prod replicas, against 7.4Gi of headroom.
barrfalk
approved these changes
Oct 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes CCP-5824.
diff_branchdefaulted tomain, so a push to main diffed main against main — always empty, so the build never triggered and the action retagged:latestonto:devinstead.:latestis whichever image was tagged most recently by any build, so the deployed image depended on timing.Sets
diff_branch: HEAD^andtag_fallback: dev, and logs which image each package actually deployed.Also raises the frontend promtail sidecar memory limit from 128Mi to 256Mi — it idles at 60-110Mi in prod and was tripping ContainerMemoryHigh at 85% of the old limit.
Thanks for the PR!
Deployments, as required, will be available below:
Please create PRs in draft mode. Mark as ready to enable:
After merge, new images are deployed in: