Skip to content

xtask: package the binaries in a fixed order - #436

Open
rudycelekli wants to merge 1 commit into
pollen-robotics:mainfrom
rudycelekli:fix/xtask-package-sorted-binaries
Open

rudycelekli wants to merge 1 commit into
pollen-robotics:mainfrom
rudycelekli:fix/xtask-package-sorted-binaries

Conversation

@rudycelekli

Copy link
Copy Markdown

What's wrong

updater-design.md 搂16 says:

Artifacts are reproducible. Fixed mtimes in the tar mean the same inputs produce the same archive, so a rebuild can be compared against what shipped; a test asserts two packages of the same inputs hash identically.

xtask package fixes the mtime, but it appends bin/* in std::fs::read_dir order, and that order belongs to the filesystem. On ext4 (the CI runners) it is htree hash order, seeded per filesystem, so the same staged binaries packed on two runners can produce two archives with two different sha256s. The name list written into version.toml was already sorted. The archive order was not. Also, no test in xtask/tests packs the same inputs twice, so the test the design doc mentions does not exist.

Here is what packing eight stub binaries on main produced on APFS:

left:  ["bin/btd", "bin/zeta", "bin/updaterd", "bin/configd", "bin/robotd", "bin/mid", "bin/alpha", "bin/nfcd"]
right: ["bin/alpha", "bin/btd", "bin/configd", "bin/mid", "bin/nfcd", "bin/robotd", "bin/updaterd", "bin/zeta"]

The fix

package collects the bin_dir paths, sorts them, and then appends them. shipped is built in that order, so the separate shipped.sort() is no longer needed. --include entries keep their command-line order, which the workflow already fixes.

Test

binaries_are_packaged_in_a_fixed_order (in xtask/tests/artifact.rs) runs the real xtask package on eight stub binaries. It checks that the bin/ entries in the archive are in sorted order, and that packing the same files created in reverse order gives byte-identical archives. The order assertion is the one that catches the bug. Two directories on one filesystem usually enumerate the same way, so the byte comparison alone would pass on main. The test comment says this.

  • On main: fails with the order shown above.
  • With the fix: passes.
  • Mutation check: I restored xtask/src/main.rs from origin/main and the named test failed. Then I put the fix back.

Checks (macOS, aarch64)

  • cargo test -p xtask: 68 passed, 0 failed (41 + 6 + 18 + 3)
  • cargo clippy -p xtask --all-targets -- -D warnings: clean
  • cargo fmt -p xtask --check: clean

馃 Generated with Claude Code

`package` appended `bin/*` in `read_dir` order, which is whatever order
the filesystem keeps (on ext4 that is hash order with a per-filesystem
seed). The same binaries packed on two machines could therefore produce
two archives with two sha256s. That breaks the promise in
updater-design.md 搂16 that fixed mtimes make the same inputs produce the
same archive, "so a rebuild can be compared against what shipped". The
names were already sorted for version.toml. Now the archive order is
sorted too.

The test checks the entry order directly. Packing twice on one
filesystem usually enumerates both directories alike, so a byte
comparison alone would pass on the bug.

Assisted-by: Claude:claude-opus-5-5

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant