Skip to content

Test actual LRE in CI - #2321

Draft
palfrey wants to merge 5 commits into
TraceMachina:mainfrom
palfrey:lre-in-ci
Draft

Test actual LRE in CI#2321
palfrey wants to merge 5 commits into
TraceMachina:mainfrom
palfrey:lre-in-ci

Conversation

@palfrey

@palfrey palfrey commented May 11, 2026

Copy link
Copy Markdown
Member

Description

We previously had LRE testing, but it was a mix of trivial examples and buggy k8s-in-CI setups long commented out. This PR spins up a docker compose setup with the current branch's nix images and tries to build them, which will make it much easier to make improvements to those.

Type of change

Please delete options that aren't relevant.

  • New feature (non-breaking change which adds functionality)

How Has This Been Tested?

CI, bunch of local docker compose and bazel test similar to the CI steps.

Checklist

  • Updated documentation if needed
  • Tests added/amended
  • bazel test //... passes locally
  • PR is contained in a single commit, using git amend see some docs

This change is Reviewable


EXPOSE 50051/tcp 50052/tcp
CMD ["nativelink"]
ENTRYPOINT ["nativelink"]

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making it behave like the nix images

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using the newer config so I don't get the warnings every time I boot things

platform_properties: {
cpu_count: {
query_cmd: "nproc",
values: [],

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Path isn't present on the nix images

Comment thread nativelink-worker/src/running_actions_manager.rs
Comment thread nativelink-worker/src/running_actions_manager.rs
Comment thread nativelink-store/Cargo.toml
Comment thread nativelink-store/Cargo.toml Outdated
@vercel

vercel Bot commented Jun 11, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
nativelink Ready Ready Preview, Comment Jul 29, 2026 1:21pm
nativelink-aidm Ready Ready Preview, Comment Jul 29, 2026 1:21pm

Request Review

@palfrey

palfrey commented Jul 27, 2026

Copy link
Copy Markdown
Member Author

I think what's missing is libz, and something incorporating hermeticbuild/rules_rs#139 might be able to fix this?

@palfrey

palfrey commented Jul 29, 2026

Copy link
Copy Markdown
Member Author

I think what's missing is libz, and something incorporating hermeticbuild/rules_rs#139 might be able to fix this?

I'm now more uncertain about this. So, the reason I flagged libz is that the glibc version of rustc was trying to be run under the Nix LRE container, and it was failing due to a lack of libz. I had a fix for adding libz, but was still seeing the same problem. Now, ideally what we actually want is the musl version of rustc run under the container, but getting rules_rs to do that seems tricky, and I haven't figured that out yet. I think also that a lot of the problems here were caused by moving to the hermetic rust setup, which then pulled in it's own copy of Rust v.s. having previously the Nix version. I think it's fine to use the Bazel hermeticity for getting Rust itself, we just need to make sure it's compatible with what Nix wants to do...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant