-
Notifications
You must be signed in to change notification settings - Fork 410
chore: evaluate Remote Build Execution (RBE) @ Namespace #10579
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
basvandijk
merged 258 commits into
master
from
basvandijk/namespace-bazel-remote-execution
Jul 29, 2026
Merged
Changes from 236 commits
Commits
Show all changes
258 commits
Select commit
Hold shift + click to select a range
acc567a
Parse BRE worker digest from nsc upload output
basvandijk dd8e7fb
Potential fix for pull request finding
basvandijk 765211b
Grant id-token: write so nsc can authenticate via GitHub OIDC
basvandijk 41345a5
Revert id-token: write; Namespace runners already have a workspace id…
basvandijk d6a217a
Address review: least-privilege permissions and safe inputs reference
basvandijk e94973f
wip
basvandijk 464a535
Switch to namespace-profile-default
basvandijk a4d7525
Pass BRE test target patterns as a quoted array
basvandijk 7b3e520
Make update-image-references checkout robust on push/dispatch
basvandijk 44aa0e4
Merge remote-tracking branch 'origin' into basvandijk/namespace-bazel…
basvandijk 9c9e323
trigger container build
basvandijk 34c3300
grant ns runner all baseimage permissions
basvandijk dc11e43
Updating container images to tag: d395e4504a1bf0cf387cbf17335b6a6cf47…
9c34c77
run bazel inside the container
basvandijk 5f9d3e8
install nsc in the container
basvandijk a827076
fix
basvandijk db404e1
Updating container images to tag: a139b8d37688eb2a5e1bc4a85b730a3ea64…
ee81d00
reference ic-build image via output
basvandijk b8423ea
trigger
basvandijk 2ddf870
Updating container images to tag: a139b8d37688eb2a5e1bc4a85b730a3ea64…
dd831d4
bring nsc into scope
basvandijk fc4d909
debug
basvandijk d3c7adf
Install nsc to /bin
basvandijk 7758e4b
fix
basvandijk d182040
Updating container images to tag: 7838da7e90af323eb5aeb8463bc0346ef73…
d102776
fix
basvandijk e2c51dd
Updating container images to tag: 7a908e1407f45149e59440b14029c6b9937…
6b5be94
nsc auth exchange-github-token
basvandijk 109e28d
permissions: id-token: write
basvandijk 4fc49d1
use host's nsc auth
basvandijk cdab485
uses: namespacelabs/nscloud-setup@v0
basvandijk 5445b3f
Run skylib copy actions locally under Bazel remote execution
basvandijk 10dab84
nsc is not needed in the container since we use namespacelabs/nsclou…
basvandijk 8767ef3
Run non-remote-executable spawns locally under Namespace BRE
basvandijk b38fdf0
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk 8a70e66
Updating container images to tag: d395e4504a1bf0cf387cbf17335b6a6cf47…
3c155aa
Run local-only genrules locally under Namespace BRE
basvandijk 6d3eee7
Keep BRE worker src dynamic, immune to the update-image-references sed
basvandijk 7cde29f
Pass C++ .d files in memory under Namespace BRE
basvandijk 39242c0
Run Namespace BRE job in a privileged container for ic-os podman builds
basvandijk 2b0504f
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk f819061
trigger container rebuild
basvandijk 1dac48a
Updating container images to tag: 22378bb2ad2621b518f4000afdb1ebbe793…
75e5bf8
fix(bazel): ship rust-lld's gcc-ld self-contained linker to Rustc act…
basvandijk 521b674
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk c7f598d
ci(bre): drop obsolete --experimental_inmemory_dotd_files flag
basvandijk 085bb0a
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk 5439548
fix(bazel): keep rust-lld binary in sysroot for wasm canister links
basvandijk 3ef6265
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk 6c83df8
test(pocket-ic): split requires-network tests into arm64-darwin variants
basvandijk 20bc0f4
test(pocket-ic): set crate_root for multi-source -darwin test variants
basvandijk c06dcdc
fix: build artifact_bundle locally to avoid dangling symlinks under BRE
basvandijk 380255a
test: run permission-denied tests as an unprivileged user when root
basvandijk 24982cf
--skip_incompatible_explicit_targets
basvandijk 2801748
fix: compile test_utilities/privileges on macOS
basvandijk a99be41
test: run sns-cli permission-denied tests as unprivileged user when root
basvandijk ab1b797
Revert " --skip_incompatible_explicit_targets"
basvandijk 861e217
Revert "test(pocket-ic): set crate_root for multi-source -darwin test…
basvandijk eef3df9
Revert "test(pocket-ic): split requires-network tests into arm64-darw…
basvandijk 9d830fc
Skip pocket-ic:unix test_canister_http_in_live_mode on the BRE job
basvandijk f6db25c
test(driver): run libvirtd/QEMU/dnsmasq as nobody when the local back…
basvandijk 3e115bb
ci: mitigate remote CAS blob eviction on the BRE job
basvandijk b1a9eff
test(driver): let nobody traverse to the libvirtd binary and working dir
basvandijk 1794676
test(driver): run net-admin scripts directly (not via capsh) when root
basvandijk 41f73bb
Revert "Skip pocket-ic:unix test_canister_http_in_live_mode on the BR…
basvandijk 1351d03
disable //packages/pocket-ic:unix
basvandijk 0a654ba
BRE -> RBE
basvandijk 1d4ad17
--test_output=errors
basvandijk 015a779
enable //packages/pocket-ic:unix again but disable test_canister_http…
basvandijk 751728f
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk 5d996b8
start binary search for the hanging //packages/pocket-ic:unix
basvandijk a4fe22c
more ignores
basvandijk 20ebc34
more ignores
basvandijk 16270df
one more ignore
basvandijk bf8d8e5
more ignores
basvandijk 283f27e
run resume_killed_instance_strict
basvandijk 213e33b
run resume_killed_instance
basvandijk a41bc5f
only ignore resume_killed_instance
basvandijk f02fc80
reduce diff
basvandijk 6850827
experiment: net_admin
basvandijk 1348098
Revert "experiment: net_admin"
basvandijk 8f4f8d9
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk 0694570
Unshare a self-owned netns when root lacks CAP_NET_ADMIN over the cur…
basvandijk ac4c025
test: run permission-denied tests as an unprivileged user when root
basvandijk f914787
fix: compile test_utilities/privileges on macOS
basvandijk 0ff2873
test: run sns-cli permission-denied tests as unprivileged user when root
basvandijk 75903f4
fix: address review comments in run_as_nobody_if_root
basvandijk 39e8abf
Automatically fixing code for linting and formatting issues
071d0ee
docs: comment why run_as_nobody_if_root is needed at each call site
basvandijk e224f9f
feat: add #[as_nobody_when_root] attribute form of run_as_nobody_if_root
basvandijk ceaeddc
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk f073f6c
Merge branch 'basvandijk/run-permission-denied-tests-as-nobody' into …
basvandijk 8f6d88a
fix: unwrap double-applied run_as_nobody_if_root after merge
basvandijk 3856b8c
fix: shard zig cache per action under remote execution
basvandijk 42e0d14
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk e9f93c4
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk e7a44cb
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk fb8d89d
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk 5e37e0a
Refactor RBE workflow to build bazel args in a documented array
basvandijk 6bcb0ba
Revert changes in Cargo.toml
basvandijk 06f7220
Merge remote-tracking branch 'origin' into basvandijk/namespace-bazel…
basvandijk 38afa08
try resume_killed_instance again
basvandijk 7893858
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk 0c41013
rm comments
basvandijk 0bb6ad4
trigger image build
basvandijk d0f0132
refactor
basvandijk 318c9e5
tweaks
basvandijk d6b9892
Updating container images to tag: 8daf75b044f16ab4395e9746071e3d09115…
2deda1c
Merge branch 'master' into basvandijk/namespace-bazel-remote-execution
basvandijk 9839569
Merge branch 'master' into basvandijk/namespace-bazel-remote-execution
basvandijk 73235a5
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk 0e0a01a
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk 7c4490e
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk b19fed5
Prefer sandboxed strategy for local fallback in RBE namespace test
basvandijk 659209e
Don't run tests that require a network namespace remotely via RBE for…
basvandijk a08da84
bazel execution setup --enable_remote_asset_api
basvandijk 3742999
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk ea4fd81
Temporarily skip local_system_tests
basvandijk 2f99691
RBE: forward per-URL credentials to Namespace remote downloader
basvandijk 9764e21
fix
basvandijk e23aa0c
RBE: use --experimental_remote_downloader_propagate_credentials
basvandijk 5edfe3e
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk 0bdb7db
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk d7c2898
have a step running no system-tests and one running only system-tests
basvandijk f1436a2
fix
basvandijk c911ba7
ci(rbe): store Bazel BEP and profile in Namespace Artifact Storage
basvandijk 9dff17a
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk a38f55a
set a per step timeout to ensure the BEP file gets uploaded
basvandijk e4d33c1
trigger container rebuild
basvandijk 42830f0
Updating container images to tag: abbba8fa906b3d98553d787112d7ce0041a…
91be156
Apply suggestion from @basvandijk
basvandijk c4b2c15
Merge branch 'master' into basvandijk/namespace-bazel-remote-execution
basvandijk 607c2a0
Apply suggestion from @basvandijk
basvandijk 75e1a2a
test: end-to-end deposit-from-CEX EIP-7702 sweep demo against a local…
gregorydemay 22c883b
RBE tweaks based on EngFlow's Bazel Invocation Analyzer
basvandijk cf64ebe
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk ab84615
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk 37f8bcd
trigger
basvandijk 8c7dc46
Updating container images to tag: d6256361e7396874e20ee8297fb09b166a3…
b3e8a1a
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk 55bc803
--config=flaky_retry
basvandijk 317f89e
fix
basvandijk a6cab42
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk dca2194
trigger
basvandijk b2e3cb7
bump timeouts some more
basvandijk b357d46
Updating container images to tag: 90f50a9529f78a97ad0c37055d15a93cbf4…
b0d5b05
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk c184677
bump limits even more
basvandijk df56025
trigger container rebuild
basvandijk 14fdca7
Updating container images to tag: 285e2d5f1b7bc3735765c70b70504aad04c…
a50befa
--jobs=15
basvandijk f0a8f94
Bazel build separately, just like in ci-main
basvandijk bc9bfe6
Use --build_tag_filters in build step
basvandijk 80873b5
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk 836160b
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk ebf6354
trigger container rebuild
basvandijk 503f72a
--remote_default_exec_properties=namespace_action_isolation=sandboxed
basvandijk ef6eace
mv --remote_default_exec_properties=container-image=... to other --re…
basvandijk 1f3fd99
Remove tags = ["no-remote-exec"] to benefit from Namespace's new sand…
basvandijk 17ee3a3
Updating container images to tag: 19e9d2e4ce05cbdf61704b96509658e125b…
d4e27c7
Set exec_properties explicitly
basvandijk f98c9ee
fix: cpus has to be string
basvandijk 84c8fc8
Set `exec_properties = {"cpu": "n"}` for all `tags = ["cpu:n"]`
basvandijk 7ea1f48
rm `cpu:n`` tags since they conflict with `exec_properties = {"cpu": …
basvandijk 397ece2
Merge branch 'master' into basvandijk/namespace-bazel-remote-execution
basvandijk 1486a59
Try --jobs=64 again after the cpu reservation fixes by Namespace
basvandijk f8220bd
--jobs=120 for `bazel build`
basvandijk 6747c8b
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk 5295489
trigger
basvandijk a6d99b8
Do not run `@@+mainnet_icos_images+...//:guest-img` remotely
basvandijk 46ecc2f
Lower --jobs again to get to a passing job
basvandijk e21b54d
Updating container images to tag: d82f82fd3da23633ea37dee630d0eff9814…
b08b8d0
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk 696cf80
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk d31323c
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk 4f485b0
chore: set namespace_requires_network exec property on requires-netwo…
basvandijk 965bf16
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk ee953cf
fix //packages/pocket-ic:canister_snapshots on RBE
basvandijk 0fce252
Only test //packages/pocket-ic:canister_snapshots for now on RBE
basvandijk 6a76757
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk 4cfe7db
Revert "Only test //packages/pocket-ic:canister_snapshots for now on …
basvandijk d9c0f81
fix(system-tests): keep local backend dnsmasq non-root under RBE sandbox
basvandijk f9357b9
fix(test-utils): make nobody temp base reachable under sandboxed RBE
basvandijk 95ab262
fix(nns_delegation_manager): skip DNS resolver for literal API BN IPs
basvandijk 501d107
fix(system-tests): tag local backend TAP with the driver's inner uid
basvandijk 1223a2d
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk ab99512
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk 74125d5
ci(rbe-namespace): upload failed and flaky system-test logs
basvandijk c08c6c7
fix(nns_delegation_manager): skip DNS resolver for literal API BN IPs
basvandijk eb4d0ce
fix(system-tests): cap unstuck_subnet_test per-VM memory on local bac…
basvandijk 5fec32f
fix(system-tests): pin rent_subnet_test to the Farm backend
basvandijk d20e978
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk 7b90228
fix: pass dummy nameserver
pierugo-dfinity 8813a8d
fix: revert unused import
pierugo-dfinity 3d11f5c
Automatically fixing code for linting and formatting issues
5b5262d
test(nns): remove the 32 filler subnets from rent_subnet_test
basvandijk b1c8053
Merge branch 'basvandijk/nns-delegation-skip-dns-for-literal-ip' into…
basvandijk fc571ac
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk ce0662f
Merge remote-tracking branch 'origin/bas/rent-subnet-test-no-filler-s…
basvandijk 924a025
Apply suggestion from @basvandijk
basvandijk 72df15b
Automatically fixing code for linting and formatting issues
85eb461
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk e26ba87
chore: unify the bazel test invocation steps back into a single step …
basvandijk a8da15a
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk ee41031
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk a7419ea
Switch to namespace-profile-rbe-driver-amd64-linux-16x32
basvandijk e702535
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk 180c75b
remove comment
basvandijk 3285792
First stab at integration RBE (in eval mode) into regular CI
basvandijk 9c9ddc8
fix: infer-bazel-targets needs the checkout git with history
basvandijk c2a1c16
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk e21552b
Double --jobs for RBE
basvandijk cbd4c48
run infer-bazel-targets on namespace runner
basvandijk 36db159
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk ad22bd8
Remove some unneeeded "namespace_requires_network": "true" exec_prope…
basvandijk d48b708
layout
basvandijk 5fbb2b5
lower --jobs for bazel build from 128 to 96 to try to avoid queuing o…
basvandijk 29bd6a9
Just go for --jobs=120 (15 workers * 8 actions each)
basvandijk 2a0e8c6
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk c737b04
Also temporarily disable diff_only in ci-rbe-evaluation.yml
basvandijk 1991461
--jobs=240
basvandijk 87a2608
trigger
basvandijk a357cd4
feat(system-tests): write the JUnit XML report from the test driver
basvandijk 0a65772
fix(system-tests): report skipped tasks as status="notrun" in the JUn…
basvandijk f716137
fix: don't mix up bazel-targets and bazel-rbe-targets
basvandijk eb4aca2
oversubscribe cpus for local system-tests
basvandijk c8460f1
fix: validate cpus and cpus_oversubscription_factor, clarify docstrings
basvandijk e86f88c
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk ef97445
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk c374339
trigger container build
basvandijk dd6dd5b
Updating container images to tag: d82f82fd3da23633ea37dee630d0eff9814…
5f4c444
use tee
basvandijk f87d0c2
remove `|| github.ref_name`
basvandijk 2c9ff05
fix
basvandijk b143eb3
trigger container rebuild
basvandijk d64ef13
Updating container images to tag: d82f82fd3da23633ea37dee630d0eff9814…
cbabdf1
Merge branch 'master' into basvandijk/namespace-bazel-remote-execution
basvandijk cdd8bc2
Merge remote-tracking branch 'origin/master' into basvandijk/namespac…
basvandijk bdbbb19
trigger container rebuild
basvandijk 7ea2f9d
Simplify rbe-worker-image job
basvandijk 470eff0
simplify
basvandijk d9ba0ba
Updating container images to tag: d82f82fd3da23633ea37dee630d0eff9814…
7550fc9
fix
basvandijk 83e2018
trigger
basvandijk fe7ad4f
Updating container images to tag: d82f82fd3da23633ea37dee630d0eff9814…
a98d704
fix bazel/hermetic_cc_toolchain_cache_dir.patch
basvandijk 9460d94
fix
basvandijk d422ede
trigger
basvandijk c43ea6a
Updating container images to tag: d82f82fd3da23633ea37dee630d0eff9814…
6c41243
remove continue-on-error: true such that the workflow turns red when …
basvandijk File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.