Repository navigation
Floresta distros: declarative, per-host build matrix - #66
Merged
Merged
Conversation
packageName becomes packageSet: a list of florestad / floresta-cli / libfloresta built in one cargo invocation, so the shared dependency graph (libbitcoinkernel included) compiles once. The default is florestad and floresta-cli; the package is named after its component when it has one, floresta otherwise, and runs as its first binary. The import-time patch arguments become mkFloresta options: extraEnvVars, buildPhase and installPhase (each replacing its cargo hook), and the Android SDK goes through extraBuildInputs. android-outputs.nix passes them per build; the examples follow the packageSet name.
A distro is floresta-build specialised for one place to run: one release, one package set, one Rust target. mkFlorestaDistro validates a distro spec (name, description, release, pkgs, static, rustTarget, toolchain, mkFloresta) with the module system and builds it through floresta-build. `pkgs` is the package set the build runs in (a pkgsCross set for cross builds); rustTarget defaults to its triple. Static distros are checked by an installCheck. The toolchain comes from rust-overlay: the release's rust-toolchain.toml, stable when absent, carrying rust-std for the target. mkFloresta takes floresta-build's options, and patch modules read the distro's pkgs and rustTarget as module arguments. mkFlorestaDistroForEachTag builds one distro from each release of a list, the spec a function of the release, named florestad-<distro>-v<release>. The flake's pkgs now carry the rust-overlay overlay, and lib.mkFlorestaDistro exposes the builder per system.
…ost) Replace the master/v0_9_x release outputs with distros: florestad and floresta-cli built for one place to run, from one release, as one host builds it. lib/targets.nix lists every package, per host, with mkFlorestaDistroForEachTag: packages.<host>.florestad-<distro>-v<release>. A host lists a distro only after that combination was built once. Releases are a list, oldest first, so releasesSince "0.10.0" is a range; 0.9.0, 0.9.1 and 0.10.0 (v0.10.0-preview) are fetched from jaoleal/FlorestaBA. Distros: x86_64-linux and aarch64-linux (native, dynamic, glibc), aarch64-darwin, and aarch64-android from 0.10.0 (x86_64-linux). A distro is dynamic unless its name says -static; only static binaries run outside Nix or NixOS. The static musl x86_64-linux-static distro (from x86_64-linux and aarch64-linux) is written out but commented: rustc links it as static-pie, and pkgsStatic's libstdc++.a, which libbitcoinkernel-sys links, is not built with -fPIE. lib/patches/android-patches.nix is the NDK build as a mkFloresta module, reading the distro's pkgs and rustTarget as module arguments. It replaces lib/android-outputs.nix, and with it the fenix input. `packages` holds the whole matrix for the host; the `master` release and the floresta-master input are gone. Attestation manifests move to legacyPackages.<host>.attestation-manifests."<version>", grouping the host's packages by release; `nix run .#attest` builds them from there. lib.targetsFor exposes the matrix per system.
They covered the old workspace builds, which are gone: the distro manifests of those releases no longer reproduce them.
The floresta-master input is gone, so there is nothing for the update-master workflow to bump, nor for dependabot to leave to it.
ci.yml evaluates packages.<host> for every host and builds each package on that host's runner, so a distro or release added to lib/targets.nix is built with no edit here. It absorbs android.yml. build-and-attest publishes one release, taken from the dispatch input or the pushed tag, building each host's packages of that release and naming artifacts <file>-<rustTarget> as lib/attestation.nix does, on x86_64-linux, aarch64-linux and aarch64-darwin. One host per distro: two copies of one artifact cannot both be published, so only one Linux host may build x86_64-linux-static once it is enabled.
`just build <distro> <release>` builds one package, and build-and-package-all builds every distro this host produces, of every release or only the one given, into artifacts/<release>/ named <file>-<triple>. An artifact already there is compared, never overwritten: the recipe fails if the build's bytes differ.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Until now, the flake exported one ad-hoc output per release (
.#master,.#v0_9_1, …), with Android bolted ontomasterby a separate module (android-outputs.nix) and its own toolchain (fenix). Adding a new target meant new wiring inflake.nix, CI and the attestation code. Nothing was shared between releases and targets except by accident.This PR adds tooling to declare Floresta distributions (distros). A distro is
florestad+floresta-clibuilt for one place to run, from one release, by one build host. Each distro spells out:packages.<host>lists it;pkgsor apkgsCross.*set) and the Rust triple;installCheck);lib/patches/android-patches.nix).Every distro goes through the same builder, so they share one dependency graph:
rust-toolchain.toml.florestad,floresta-cliandlibflorestabuild in one cargo invocation, solibbitcoinkernelcompiles once per build.Because of this, adding a distro or a release only adds entries: it reuses what is already built and cached instead of adding new wiring.
What changes
Build library
floresta-build:packageNamebecomespackageSet, a list of components built in one cargo run. The per-target patches (buildPhase,installPhase,extraEnvVars,extraBuildInputs) becomemkFlorestaoptions instead of import-time arguments.lib/mkFlorestaDistro.nix(new):mkFlorestaDistrovalidates a distro spec with the module system and builds it.mkFlorestaDistroForEachTag releases (release: { … })stamps out one package per release.lib.mkFlorestaDistro/lib.targetsFor.Outputs
lib/targets.nixis the single catalogue: the releases (an ordered list, soreleasesSince "0.10.0"is a range) and, per host, the distros it builds. Packages are namedflorestad-<distro>-v<release>, e.g.florestad-aarch64-darwin-v0_9_1.Current matrix:
x86_64-linux(dynamic, glibc)aarch64-linux(dynamic, glibc)aarch64-darwinaarch64-android(NDK)Naming convention: a distro is dynamic unless its name ends in
-static. Only static binaries run outside Nix/NixOS, because dynamic ones load their interpreter and libraries from/nix/store.CI and release
ci.ymlreads the matrix offnix eval .#packagesand builds each package on its host's runner, so a new distro or release needs no workflow edit. It absorbsandroid.yml.build-and-attest.ymlbuilds one release on every host (x86_64-linux, aarch64-linux, aarch64-darwin) and names artifacts<file>-<rustTarget>, matching the attestation manifests.just build <distro> <release>builds one package.just build-and-package-all [release]collects every distro (or one release) intoartifacts/<release>/, and fails if an existing artifact's bytes differ.Breaking changes
.#master,.#v0_9_0,.#v0_9_1and.#attestation-manifest-*are gone. Manifests now live atlegacyPackages.<host>.attestation-manifests."<version>".floresta-masterandfenixinputs are removed, along with theupdate-mastersync workflow.contrib/sigs/are dropped. They signed the old workspace builds, which the distro manifests no longer reproduce, so these releases need to be re-attested.x86_64-darwinno longer has packages (see below).Not in this PR / known issues
x86_64-linux-static) is written out but commented: rustc links it as static-pie, andpkgsStatic'slibstdc++.a(pulled in bylibbitcoinkernel-sys) isn't built with-fPIE.armv7a-android,x86_64-androidandx86_64-darwinare commented out until they've been built once.jaoleal/FlorestaBA, and0.10.0points atv0.10.0-preview. That tag'sCargo.tomlstill says0.9.0, so its store paths are named…-0.9.0. This is temporary until we have 0.10.0 on upstream floresta.Testing
nix flake checkandnix flake check ./examples --no-buildpass.drvPaths are unchanged by themkFlorestaDistroForEachTagrewrite.