Update release workflow #6697

wileyj · 2025-11-18T22:01:39Z

Note: relies on composite PR stacks-network/actions#91 due to the changes in registry.

This change refactors the release workflow as follows:

changes the binary build process from using docker to building on the VM directly
docker images use the binary artifact archive, rather than assets from a published github release
multiple approval gates are removed in favor of a single one to start the build
docker images publish to ghcr (this will require some additional setup after merge)
workflows and inputs have been simplified to reduce duplication and remove variables that aren't needed
docker tags are simplified: stacks-blockchain is no longer tagged by default for ghcr
workflow filenames have been renamed slightly to better reflect what they do. release-*.yml etc.
building the ad-hoc docker image using the same artifact method, but is coped only for glibc x64/arm64
all archives and images are attested, with the docker images also being signed with cosign (https://github.com/sigstore/cosign)
dockerfiles used to create the images are re-organized based on target and purpose
macos builds are building on macos runners, with the only cross-compiliation being for cpu/c stdlib, which only involves updating linkers and other build-time flags.
removed the cleanup step when setting up docker. via composite recent runners don't benefit from this, and the storage is more than adequate now to build stacks-core.

Overall, the performance is much better than using docker - what was taking ~30 minutes (discounting timeouts and failures) is now reliably down to ~12 minutes per build (macos being the outlier, since it was tested on a 7GB machine with the release-lite profile).

a full test release workflow: https://github.com/wileyj/stacks-core/actions/runs/19481419227

and a specific run of just the release workflow to show the expected timing of the changes proposed here: https://github.com/wileyj/stacks-core/actions/runs/19475022815

and the ghcr images produced-
signer: https://github.com/wileyj/stacks-core/pkgs/container/stacks-signer/versions?filters%5Bversion_type%5D=tagged
stacks-core: https://github.com/wileyj/stacks-core/pkgs/container/stacks-core/versions?filters%5Bversion_type%5D=tagged

published test release: https://github.com/wileyj/stacks-core/releases/tag/4.0.0.0.6
published singer test release: https://github.com/wileyj/stacks-core/releases/tag/signer-4.0.0.0.6.0

finally - the builds are set to use the release profile, which willl require at least the macos-latest-large runner, with the linux runners currently set to use the default ubuntu-latest (this may be changed, but the timing on default runners was acceptable).

the binaries/images need more verification, but initial testing of produced artifacts looks good.

codecov · 2025-11-19T00:03:05Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79.78%. Comparing base (dd1b787) to head (eb9f283).
⚠️ Report is 63 commits behind head on develop.

Additional details and impacted files

@@             Coverage Diff              @@
##           develop    #6697       +/-   ##
============================================
+ Coverage    44.77%   79.78%   +35.01%     
============================================
  Files          577      577               
  Lines       357352   357795      +443     
============================================
+ Hits        159992   285475   +125483     
+ Misses      197360    72320   -125040

see 494 files with indirect coverage changes

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dd1b787...eb9f283. Read the comment docs.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

wileyj · 2025-11-19T04:29:17Z

moving to ghcr - one thing i don't like is how all the layers (tagged or not) are visible on the packages page (see link above).
i have a feeling it's a UI diff compared to something like dockerhub, but it may be confusing for users looking for what is the latest tagged release.

a few things to follow up on:

schedule a nightly build from develop. i think this should be easier than expected with the move to using artifacts built natively. a simple conditional tag if it's run via cron should be all that's needed (along with hardcoding to build nightly from develop, compared to any desired branch when run manually)
github release is created in draft mode, but i think we could also pre-populate most of the release message from the changelog(s) using a tool like https://github.com/sean0x42/markdown-extract

wileyj · 2025-11-19T15:45:00Z

there is also a question of if we should be re-running all the same tests (which should have been passing prior to release in order for the changes to merge), i.e. bitcoin tests.
on the other hand, there could be a case where a hotfix release breaks an existing test, and it woudn't be caught until the release ci.

wileyj · 2025-11-20T18:05:21Z

update - one thing i notice that i'm not sure about (but shoudln't be a blocker to merge) is that the artifacts are being attested 2x for the same archive. this may be a result of the the archive name being identical for stacks-core and the stacks-signer release.

Edit: it is indeed because of the archive file naming being a duplicate for the release. so stacks-core & stacks-signer both having the same archive name like linux-glibc-x64.zip will produce 2 quite similar attestations, even though the context of that archive is different.

fabrizio-stacks · 2025-11-24T11:35:17Z

update - one thing i notice that i'm not sure about (but shoudln't be a blocker to merge) is that the artifacts are being attested 2x for the same archive. this may be a result of the the archive name being identical for stacks-core and the stacks-signer release.

Edit: it is indeed because of the archive file naming being a duplicate for the release. so stacks-core & stacks-signer both having the same archive name like linux-glibc-x64.zip will produce 2 quite similar attestations, even though the context of that archive is different.

if you have a look to the sbtc repo I solved this problem!

.github/workflows/docker-image.yml

.github/workflows/release-github.yml

federico-stacks · 2025-11-25T11:26:09Z

there is also a question of if we should be re-running all the same tests (which should have been passing prior to release in order for the changes to merge), i.e. bitcoin tests. on the other hand, there could be a case where a hotfix release breaks an existing test, and it woudn't be caught until the release ci.

just to clear my understanding: does it means we don't run the "ci" tests for the hotfix release?

wileyj · 2025-11-25T20:28:20Z

there is also a question of if we should be re-running all the same tests (which should have been passing prior to release in order for the changes to merge), i.e. bitcoin tests. on the other hand, there could be a case where a hotfix release breaks an existing test, and it woudn't be caught until the release ci.

just to clear my understanding: does it means we don't run the "ci" tests for the hotfix release?

no - just a note that we currently re-run all tests that are run as part of the PR process when doing a release, and perhaps there's an option to not do that in the future (with the expectation that tests would have already run. there are some caveats today, but i think it's possible).

for a hotfix release, we'd still need to run all the normal required steps. but, i think there may be a case to be made where we can ensure all tests are passing before a release workflow is triggered - not addressed in this PR, but something to think about for a future change.
for hotfixes, the tests should likewise already be run by the time the workflow is triggered. it's a good question though, i think it's worth the effort to determine when and how we run tests.

…-test workflow - update ci.yml to no longer call atlas, slow workflows on release - remove test from sbtc and epoch workflows that no longer exist - moved last remaining test from slow_test to bitcoin_test workflow

.github/actions/dockerfiles/alpine/Dockerfile.release

.github/actions/dockerfiles/debian/Dockerfile.release

aaronb-stacks

This LGTM, just a comment on some of the dockerfiles' CMDs

.github/workflows/bitcoin-tests.yml

jacinta-stacks

LGTM.

.github/actions/dockerfiles/alpine/Dockerfile.release.signer

.github/workflows/docker-image.yml

francesco-stacks · 2025-12-05T07:11:23Z

.github/actions/dockerfiles/alpine/Dockerfile.release.signer

+
+FROM alpine
+COPY --from=builder /out/* /bin/
+CMD ["/bin/stacks-signer run --config /signer-config.toml"]


sanity check: shouldn't this be CMD ["/bin/stacks-signer", "run", "--config", "/signer-config.toml"] ?

both are correct, but the list is more correct.i'll adjust (same with later comment)

francesco-stacks · 2025-12-05T07:12:06Z

.github/actions/dockerfiles/debian/Dockerfile

+
+FROM debian:stable-slim
+COPY --from=builder /out/* /bin/
+CMD ["/bin/stacks-node mainnet"]


same q: CMD ["/bin/stacks-node", "mainnet"] ?

francesco-stacks · 2025-12-05T07:12:22Z

.github/actions/dockerfiles/debian/Dockerfile.release

+
+FROM debian:stable-slim
+COPY --from=builder /out/* /bin/
+CMD ["/bin/stacks-node mainnet"]


francesco-stacks · 2025-12-05T07:12:29Z

.github/actions/dockerfiles/debian/Dockerfile.release.signer

+
+FROM debian:stable-slim
+COPY --from=builder /out/* /bin/
+CMD ["/bin/stacks-signer run --config /signer-config.toml"]


francesco-stacks · 2025-12-05T07:12:57Z

.github/actions/dockerfiles/alpine/Dockerfile.release

+
+FROM alpine
+COPY --from=builder /out/* /bin/
+CMD ["/bin/stacks-node mainnet"]


francesco-stacks · 2025-12-05T07:31:57Z

.github/workflows/release-docker.yml

+              shell: bash
+              run: |
+                  var_docker_tag_rc=false
+                  if [[ "${docker_tag}" =~ -rc[0-9]*$ ]]; then


should the variable docker_tag be uppercase? if [[ "${DOCKER_TAG}" =~ -rc[0-9]*$ ]]; then

i treated this as a global/local syntax, where var_docker_tag_rc (and others only used in the step) are lowercase, and the global vars are set to uppercase. this may not be the same in older workflows though, but in this PR i tried to treat them this way consistently.

there's no clear guideline on what's acceptable in workflows that i could find, but if you think uppercase makes more sense i can adjust and push.

francesco-stacks · 2025-12-05T07:50:21Z

.github/workflows/docker-image.yml

+                  context: /tmp
+                  file: ./.github/actions/dockerfiles/debian/Dockerfile


sanity check: does the file relative path works even if context is pointing to /tmp instead of the current dir . ?

I have this doubt because the documentation says that the default value of file is relative to context. (default {context}/Dockerfile)

https://github.com/docker/build-push-action?tab=readme-ov-file#inputs

i read the same docs and had some really strange issues (i suspect the docs are al little obtuse).
when using a context of ., the dockerfile was not able to find the artifacts downloaded to ., which is why i chose a generic absolute of /tmp.

now, since we are setting both the context and the file inputs, i don't think the defaults in the docs apply here (and in practice, it works) - since we're setting the context specifically to /tmp, unless we also did not set the file input, it would look for /tmp/Dockerfile as a default.
but, since we're being verbose about which dockerfile to use, the default {context}/Dockerfile doesn't apply.

This reverts commit 56b5ee7.

wileyj added 4 commits November 18, 2025 11:46

Updating relase workflow to use artifacts and push images to ghcr

e191ab8

Cleanup and add some more comments

2e11be5

remove forked composite for docker

5416a51

use the release profile, not release-lite

764115a

wileyj mentioned this pull request Nov 18, 2025

Chore/6601 stacks-network/actions#91

Open

wileyj marked this pull request as ready for review November 19, 2025 04:24

wileyj requested review from a team November 19, 2025 04:24

wileyj added 2 commits November 19, 2025 09:02

disable PR creation with recent change to settings this will not work

b3662a8

fix inconsistent var formatting

542b898

wileyj mentioned this pull request Nov 19, 2025

Publish multi-platform Docker from develop #6556

Open

federico-stacks reviewed Nov 25, 2025

View reviewed changes

.github/workflows/docker-image.yml Outdated Show resolved Hide resolved

.github/workflows/docker-image.yml Show resolved Hide resolved

.github/workflows/release-github.yml Show resolved Hide resolved

.github/workflows/release-github.yml Outdated Show resolved Hide resolved

wileyj added 3 commits November 25, 2025 12:31

remove commented step

52d0ba3

addressing PR comments and adding consistency to env vars

c3fb53f

aaronb-stacks reviewed Dec 2, 2025

View reviewed changes

.github/actions/dockerfiles/alpine/Dockerfile.release Outdated Show resolved Hide resolved

aaronb-stacks reviewed Dec 2, 2025

View reviewed changes

.github/actions/dockerfiles/debian/Dockerfile.release Outdated Show resolved Hide resolved

aaronb-stacks reviewed Dec 2, 2025

View reviewed changes

jacinta-stacks reviewed Dec 2, 2025

View reviewed changes

.github/workflows/bitcoin-tests.yml Show resolved Hide resolved

jacinta-stacks previously approved these changes Dec 2, 2025

View reviewed changes

Merge branch 'develop' into chore/6601

83e43fe

wileyj requested a review from a team December 3, 2025 01:34

Merge branch 'develop' into chore/6601

1309bca

Updating default stacks-node CMD for release dockerfiles

52a4727

wileyj dismissed jacinta-stacks’s stale review via 52a4727 December 3, 2025 02:01

federico-stacks previously approved these changes Dec 3, 2025

View reviewed changes

aaronb-stacks previously approved these changes Dec 3, 2025

View reviewed changes

benjamin-stacks reviewed Dec 4, 2025

View reviewed changes

.github/actions/dockerfiles/alpine/Dockerfile.release.signer Show resolved Hide resolved

.github/workflows/docker-image.yml Show resolved Hide resolved

jacinta-stacks previously approved these changes Dec 4, 2025

View reviewed changes

francesco-stacks reviewed Dec 5, 2025

View reviewed changes

Update dockerfile CMD to use a list vs a string

56b5ee7

wileyj dismissed stale reviews from jacinta-stacks, aaronb-stacks, and federico-stacks via 56b5ee7 December 5, 2025 18:23

wileyj added 2 commits December 5, 2025 10:29

Revert "Update dockerfile CMD to use a list vs a string"

2f93b37

This reverts commit 56b5ee7.

Update dockerfile CMD to use a list vs a string

573990b

		context: /tmp
		file: ./.github/actions/dockerfiles/debian/Dockerfile

Update release workflow #6697

Are you sure you want to change the base?

Update release workflow #6697

Uh oh!

Conversation

wileyj commented Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Nov 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

wileyj commented Nov 19, 2025

Uh oh!

wileyj commented Nov 19, 2025

Uh oh!

wileyj commented Nov 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fabrizio-stacks commented Nov 24, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

federico-stacks commented Nov 25, 2025

Uh oh!

wileyj commented Nov 25, 2025

Uh oh!

Uh oh!

Uh oh!

aaronb-stacks left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jacinta-stacks left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

wileyj commented Nov 18, 2025 •

edited

Loading

codecov bot commented Nov 19, 2025 •

edited

Loading

wileyj commented Nov 20, 2025 •

edited

Loading