forward the bootstrap `runner` to `run-make` #141856

folkertdev · 2025-06-01T13:33:05Z

The runner was already forwarded to compiletest, this just passes it on to run-make and uses it in the run functions.

The configuration can look like this

# in bootstrap.toml
[target.s390x-unknown-linux-gnu]
runner = "qemu-s390x -L /usr/s390x-linux-gnu"

Any C compilation automatically sets the correct target. Calls to rustc must use .target(target()). Then, a command like below will work by cross-compiling to the given target, and using the given runner for that target to execute the binary:

./x test tests/run-make/c-link-to-rust-va-list-fn --target s390x-unknown-linux-gnu

The runner can also be used for e.g. running with valgrind.

This PR also enables its use in the test case that I care about, hopefully that actually does work on the platforms that CI uses. We should probably run some try jobs to be sure?

r? @jieyouxu

try-job: test-various
try-job: armhf-gnu

rustbot · 2025-06-01T13:33:11Z

Some changes occurred in src/tools/compiletest

cc @jieyouxu

This PR modifies run-make tests.

cc @jieyouxu

The run-make-support library was changed

cc @jieyouxu

jieyouxu

Thanks, one nit

src/tools/run-make-support/src/run.rs

jieyouxu · 2025-06-01T13:35:09Z

tests/run-make/c-link-to-rust-va-list-fn/rmake.rs


 fn main() {
-    rustc().input("checkrust.rs").run();
+    rustc().input("checkrust.rs").target(target()).run();


Remark: this will not be needed if I finish auto-cross #139244

I removed target(target() bit now (yay!) and use the new needs-target-std in that test file.

jieyouxu · 2025-06-01T13:37:54Z

This PR also enables its use in the test case that I care about, hopefully that actually does work on the platforms that CI uses. We should probably run some try jobs to be sure?

Do you happen to know which CI jobs? armhf-gnu?

folkertdev · 2025-06-01T13:43:27Z

I really don't know what cross-compilation actually happens in our CI and would run this run-make test. I've locally run this test for powerpc and s390x. Testing it with a 32-bit arm sounds like a good idea at least.

jieyouxu · 2025-06-01T13:49:12Z

Let's give it a try.
@bors try

…r, r=<try> forward the bootstrap `runner` to `run-make` The runner was already forwarded to `compiletest`, this just passes it on to `run-make` and uses it in the `run` functions. The configuration can look like this ```toml # in bootstrap.toml [target.s390x-unknown-linux-gnu] runner = "qemu-s390x -L /usr/s390x-linux-gnu" ``` Any C compilation automatically sets the correct target. Calls to rustc must use `.target(target())`. Then, a command like below will work by cross-compiling to the given target, and using the given runner for that target to execute the binary: ``` ./x test tests/run-make/c-link-to-rust-va-list-fn --target s390x-unknown-linux-gnu ``` The runner can also be used for e.g. running with `valgrind`. This PR also enables its use in the test case that I care about, hopefully that actually does work on the platforms that CI uses. We should probably run some try jobs to be sure? r? `@jieyouxu` try-job: test-various try-job: armhf-gnu

bors · 2025-06-01T13:50:23Z

⌛ Trying commit ab6646a with merge 1d3fe3c...

jieyouxu · 2025-06-01T13:51:28Z

Reminder (for myself): if try job fails, use @bors2 try to try out new bors

rust-bors · 2025-06-01T13:51:31Z

Unknown command "try`".

bors · 2025-06-01T14:26:38Z

💔 Test failed - checks-actions

jieyouxu · 2025-06-01T14:35:16Z

You probably need sth like //@ ignore-unknown (needs target std)

folkertdev · 2025-06-01T14:56:16Z

Hmm, that directive is never actually used, and also does not solve the issue for that target. It is a known target, just not one that has std (just core I assume). None of the directives in the list seem to really capture this? Maybe needs-threads is a decent proxy but that's not ideal.

jieyouxu · 2025-06-01T15:16:05Z

Yeah, a proper fix is sth like //@ needs-std, but for this PR, //@ ignore-none is a proxy for ignoring the *-unknown targets. I suppose //@ ignore-nvptx is better tho.

EDIT: opened #141863

jieyouxu · 2025-06-01T15:16:28Z

Hmm, that directive is never actually used, and also does not solve the issue for that target. It is a known target, just not one that has std (just core I assume). None of the directives in the list seem to really capture this? Maybe needs-threads is a decent proxy but that's not ideal.

(Sorry, I wrote //@ ignore-unknown but I meant //@ ignore-none as a temporary directive to unblock this PR)

folkertdev · 2025-06-01T15:51:34Z

Apparently you need to explicitly exluce nvptx (several other tests do this).

//@ ignore-none
// Reason: no-std is not supported
//@ ignore-nvptx64-nvidia-cuda
// Reason: can't find crate for 'std'

[WIP] Enable automatic cross-compilation in run-make tests > [!CAUTION] > > Stacked on top of #142414, needs rebase. Supersedes #138066. Blocker for #141856. Based on #138066 with #139242 + #139239 cherry-picked in, plus `rustdoc()` cross-compile changes. r? `@ghost` try-job: armhf-gnu try-job: test-various

[WIP] Enable automatic cross-compilation in run-make tests > [!CAUTION] > > Conflicts with #142563. Supersedes #138066. Blocker for #141856. Based on #138066 with #139242 + #139239 cherry-picked in, plus `rustdoc()` cross-compile changes. r? `@ghost` try-job: armhf-gnu try-job: test-various

Enable automatic cross-compilation in run-make tests Supersedes #138066. Blocker for #141856. Based on #138066 plus `rustdoc()` cross-compile changes. ### Summary This PR automatically specifies `--target` to `rustc()` and `rustdoc()` to have `rustc`/`rustdoc` produce cross-compiled artifacts in run-make tests by default, unless: - `//@ ignore-cross-compile` is used, or - `bare_{rustc,rustdoc}` are used, or - Explicit `.target()` is specified, which overrides the default cross-compile target. Some tests are necessarily modified: - Tests that have `.target(target())` have that incantation removed (since this is now automatically the default). - Some tests have `//@ needs-target-std`, but are a necessary-but-insufficient condition, and are changed to `//@ ignore-cross-compile` instead as host-only tests. - A few tests received `//@ ignore-musl` that fail against `x86_64-unknown-linux-musl` because of inability to find `-lunwind`. AFAICT, they don't *need* to test cross-compiled artifacts. - Some tests are constrained to host-only for now, because the effort to make them pass on cross-compile does not seem worth the complexity, and it's not really *meaningfully* improving test coverage. try-job: armhf-gnu try-job: test-various

Enable automatic cross-compilation in run-make tests Supersedes #138066. Blocker for #141856. Based on #138066 plus `rustdoc()` cross-compile changes. ### Summary This PR automatically specifies `--target` to `rustc()` and `rustdoc()` to have `rustc`/`rustdoc` produce cross-compiled artifacts in run-make tests by default, unless: - `//@ ignore-cross-compile` is used, or - `bare_{rustc,rustdoc}` are used, or - Explicit `.target()` is specified, which overrides the default cross-compile target. Some tests are necessarily modified: - Tests that have `.target(target())` have that incantation removed (since this is now automatically the default). - Some tests have `//@ needs-target-std`, but are a necessary-but-insufficient condition, and are changed to `//@ ignore-cross-compile` instead as host-only tests. - A few tests received `//@ ignore-musl` that fail against `x86_64-unknown-linux-musl` because of inability to find `-lunwind`. AFAICT, they don't *need* to test cross-compiled artifacts. - Some tests are constrained to host-only for now, because the effort to make them pass on cross-compile does not seem worth the complexity, and it's not really *meaningfully* improving test coverage. try-job: dist-various-1 try-job: dist-various-1

Enable automatic cross-compilation in run-make tests Supersedes #138066. Blocker for #141856. Based on #138066 plus `rustdoc()` cross-compile changes. ### Summary This PR automatically specifies `--target` to `rustc()` and `rustdoc()` to have `rustc`/`rustdoc` produce cross-compiled artifacts in run-make tests by default, unless: - `//@ ignore-cross-compile` is used, or - `bare_{rustc,rustdoc}` are used, or - Explicit `.target()` is specified, which overrides the default cross-compile target. Some tests are necessarily modified: - Tests that have `.target(target())` have that incantation removed (since this is now automatically the default). - Some tests have `//@ needs-target-std`, but are a necessary-but-insufficient condition, and are changed to `//@ ignore-cross-compile` instead as host-only tests. - A few tests received `//@ ignore-musl` that fail against `x86_64-unknown-linux-musl` because of inability to find `-lunwind`. AFAICT, they don't *need* to test cross-compiled artifacts. - Some tests are constrained to host-only for now, because the effort to make them pass on cross-compile does not seem worth the complexity, and it's not really *meaningfully* improving test coverage. try-job: dist-various-1

jieyouxu · 2025-06-19T09:38:24Z

@rustbot review

The runner was already forwarded to `compiletest`, this just passes it on to `run-make` and uses it in the `run` functions.

Enable automatic cross-compilation in run-make tests Supersedes rust-lang/rust#138066. Blocker for rust-lang/rust#141856. Based on rust-lang/rust#138066 plus `rustdoc()` cross-compile changes. ### Summary This PR automatically specifies `--target` to `rustc()` and `rustdoc()` to have `rustc`/`rustdoc` produce cross-compiled artifacts in run-make tests by default, unless: - `//@ ignore-cross-compile` is used, or - `bare_{rustc,rustdoc}` are used, or - Explicit `.target()` is specified, which overrides the default cross-compile target. Some tests are necessarily modified: - Tests that have `.target(target())` have that incantation removed (since this is now automatically the default). - Some tests have `//@ needs-target-std`, but are a necessary-but-insufficient condition, and are changed to `//@ ignore-cross-compile` instead as host-only tests. - A few tests received `//@ ignore-musl` that fail against `x86_64-unknown-linux-musl` because of inability to find `-lunwind`. AFAICT, they don't *need* to test cross-compiled artifacts. - Some tests are constrained to host-only for now, because the effort to make them pass on cross-compile does not seem worth the complexity, and it's not really *meaningfully* improving test coverage. try-job: dist-various-1

jieyouxu · 2025-06-21T07:10:54Z

@bors2 try

rust-bors · 2025-06-21T07:10:58Z

⌛ Trying commit d023fe7 with merge f2b6f0e…

To cancel the try build, run the command @bors2 try cancel.

…r, r=<try> forward the bootstrap `runner` to `run-make` The runner was already forwarded to `compiletest`, this just passes it on to `run-make` and uses it in the `run` functions. The configuration can look like this ```toml # in bootstrap.toml [target.s390x-unknown-linux-gnu] runner = "qemu-s390x -L /usr/s390x-linux-gnu" ``` Any C compilation automatically sets the correct target. Calls to rustc must use `.target(target())`. Then, a command like below will work by cross-compiling to the given target, and using the given runner for that target to execute the binary: ``` ./x test tests/run-make/c-link-to-rust-va-list-fn --target s390x-unknown-linux-gnu ``` The runner can also be used for e.g. running with `valgrind`. This PR also enables its use in the test case that I care about, hopefully that actually does work on the platforms that CI uses. We should probably run some try jobs to be sure? r? `@jieyouxu` try-job: test-various try-job: armhf-gnu

rust-bors · 2025-06-21T08:49:41Z

☀️ Try build successful (CI)
Build commit: f2b6f0e (f2b6f0e02e171de99ddb43686feef374f1546f0d, parent: 15c701fbc995eb6c5b3a86021c18185f8eee020d)

jieyouxu · 2025-06-21T09:36:41Z

@bors r+ rollup=never

bors · 2025-06-21T09:36:44Z

📌 Commit d023fe7 has been approved by jieyouxu

It is now in the queue for this repository.

bors · 2025-06-21T13:05:16Z

⌛ Testing commit d023fe7 with merge 7d1b2c0...

…r, r=jieyouxu forward the bootstrap `runner` to `run-make` The runner was already forwarded to `compiletest`, this just passes it on to `run-make` and uses it in the `run` functions. The configuration can look like this ```toml # in bootstrap.toml [target.s390x-unknown-linux-gnu] runner = "qemu-s390x -L /usr/s390x-linux-gnu" ``` Any C compilation automatically sets the correct target. Calls to rustc must use `.target(target())`. Then, a command like below will work by cross-compiling to the given target, and using the given runner for that target to execute the binary: ``` ./x test tests/run-make/c-link-to-rust-va-list-fn --target s390x-unknown-linux-gnu ``` The runner can also be used for e.g. running with `valgrind`. This PR also enables its use in the test case that I care about, hopefully that actually does work on the platforms that CI uses. We should probably run some try jobs to be sure? r? `@jieyouxu` try-job: test-various try-job: armhf-gnu

rust-log-analyzer · 2025-06-21T14:47:00Z

The job arm-android failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

---- [run-make] tests/run-make/c-link-to-rust-va-list-fn stdout ----

error: rmake recipe failed to complete
status: exit status: 1
command: cd "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/c-link-to-rust-va-list-fn/rmake_out" && env -u RUSTFLAGS AR="llvm-ar" BUILD_ROOT="/checkout/obj/build/x86_64-unknown-linux-gnu" CARGO="/checkout/obj/build/x86_64-unknown-linux-gnu/stage1-tools-bin/cargo" CC="/android/ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/armv7a-linux-androideabi21-clang" CC_DEFAULT_FLAGS="-DANDROID -ffunction-sections -fdata-sections -fPIC" CXX="/android/ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/armv7a-linux-androideabi21-clang++" CXX_DEFAULT_FLAGS="-DANDROID -ffunction-sections -fdata-sections -fPIC" HOST_RUSTC_DYLIB_PATH="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" LD_LIBRARY_PATH="/checkout/obj/build/x86_64-unknown-linux-gnu/bootstrap-tools/x86_64-unknown-linux-gnu/release/deps:/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/lib/rustlib/x86_64-unknown-linux-gnu/lib" LD_LIB_PATH_ENVVAR="LD_LIBRARY_PATH" LLVM_BIN_DIR="/checkout/obj/build/x86_64-unknown-linux-gnu/ci-llvm/bin" LLVM_COMPONENTS="aarch64 aarch64asmparser aarch64codegen aarch64desc aarch64disassembler aarch64info aarch64utils aggressiveinstcombine all all-targets amdgpu amdgpuasmparser amdgpucodegen amdgpudesc amdgpudisassembler amdgpuinfo amdgputargetmca amdgpuutils analysis arm armasmparser armcodegen armdesc armdisassembler arminfo armutils asmparser asmprinter avr avrasmparser avrcodegen avrdesc avrdisassembler avrinfo binaryformat bitreader bitstreamreader bitwriter bpf bpfasmparser bpfcodegen bpfdesc bpfdisassembler bpfinfo cfguard cgdata codegen codegentypes core coroutines coverage csky cskyasmparser cskycodegen cskydesc cskydisassembler cskyinfo debuginfobtf debuginfocodeview debuginfodwarf debuginfogsym debuginfologicalview debuginfomsf debuginfopdb demangle dlltooldriver dwarflinker dwarflinkerclassic dwarflinkerparallel dwp engine executionengine extensions filecheck frontendatomic frontenddriver frontendhlsl frontendoffloading frontendopenacc frontendopenmp fuzzercli fuzzmutate globalisel hexagon hexagonasmparser hexagoncodegen hexagondesc hexagondisassembler hexagoninfo hipstdpar instcombine instrumentation interfacestub interpreter ipo irprinter irreader jitlink libdriver lineeditor linker loongarch loongarchasmparser loongarchcodegen loongarchdesc loongarchdisassembler loongarchinfo lto m68k m68kasmparser m68kcodegen m68kdesc m68kdisassembler m68kinfo mc mca mcdisassembler mcjit mcparser mips mipsasmparser mipscodegen mipsdesc mipsdisassembler mipsinfo mirparser msp430 msp430asmparser msp430codegen msp430desc msp430disassembler msp430info native nativecodegen nvptx nvptxcodegen nvptxdesc nvptxinfo objcarcopts objcopy object objectyaml option orcdebugging orcjit orcshared orctargetprocess passes powerpc powerpcasmparser powerpccodegen powerpcdesc powerpcdisassembler powerpcinfo profiledata remarks riscv riscvasmparser riscvcodegen riscvdesc riscvdisassembler riscvinfo riscvtargetmca runtimedyld sandboxir scalaropts selectiondag sparc sparcasmparser sparccodegen sparcdesc sparcdisassembler sparcinfo support symbolize systemz systemzasmparser systemzcodegen systemzdesc systemzdisassembler systemzinfo tablegen target targetparser telemetry textapi textapibinaryreader transformutils vectorize webassembly webassemblyasmparser webassemblycodegen webassemblydesc webassemblydisassembler webassemblyinfo webassemblyutils windowsdriver windowsmanifest x86 x86asmparser x86codegen x86desc x86disassembler x86info x86targetmca xray xtensa xtensaasmparser xtensacodegen xtensadesc xtensadisassembler xtensainfo" LLVM_FILECHECK="/checkout/obj/build/x86_64-unknown-linux-gnu/ci-llvm/bin/FileCheck" PYTHON="/usr/bin/python3" REMOTE_TEST_CLIENT="/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/remote-test-client" RUSTC="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" RUSTC_LINKER="/android/ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/armv7a-linux-androideabi21-clang" RUSTDOC="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustdoc" SOURCE_ROOT="/checkout" TARGET="arm-linux-androideabi" TARGET_EXE_DYLIB_PATH="/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/arm-linux-androideabi/lib" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/run-make/c-link-to-rust-va-list-fn/rmake"
stdout: none
--- stderr -------------------------------
command failed at line 16
Command { cmd: "/android/ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/armv7a-linux-androideabi21-clang" "-DANDROID" "-ffunction-sections" "-fdata-sections" "-fPIC" "test.c" "libcheckrust.a" "-o" "test" "-lm" "-lrt" "-ldl" "-lpthread", stdin_buf: None, stdin: None, stdout: None, stderr: None, drop_bomb: DropBomb { command: "/android/ndk/toolchains/llvm/prebuilt/linux-x86_64/bin/armv7a-linux-androideabi21-clang", defused: true, armed_location: Location { file: "/checkout/tests/run-make/c-link-to-rust-va-list-fn/rmake.rs", line: 12, col: 5 } }, already_executed: true }
output status: `exit status: 1`
=== STDOUT ===



=== STDERR ===
ld.lld: error: unable to find library -lrt

bors · 2025-06-21T14:47:37Z

💔 Test failed - checks-actions

folkertdev · 2025-06-21T15:53:35Z

Hmm, https://developer.android.com/ndk/guides/stable_apis#c_library makes me believe that maybe -lrt should not be passed on android?

Note that on Android, unlike Linux, there are no separate libpthread or librt libraries. That functionality is included directly in libc, which does not need to be explicitly linked against.

Should we dig into that, or just ignore android here? (at the moment, most run-make tests that compile C disable cross-compilation one way or another)

rustbot assigned jieyouxu Jun 1, 2025

jieyouxu approved these changes Jun 1, 2025

View reviewed changes

folkertdev force-pushed the run-make-forward-compiletest-runner branch from 3f206ce to ab6646a Compare June 1, 2025 13:39

jieyouxu mentioned this pull request Jun 1, 2025

New bors looks within markdown code blocks rust-lang/bors#295

Closed

This comment has been minimized.

Sign in to view

bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 1, 2025

This comment has been minimized.

Sign in to view

jieyouxu mentioned this pull request Jun 1, 2025

Consider adding a //@ needs-std directive for target std support #141863

Closed

folkertdev force-pushed the run-make-forward-compiletest-runner branch from ab6646a to e3fee6a Compare June 1, 2025 15:52

folkertdev force-pushed the run-make-forward-compiletest-runner branch from 272dfe8 to abbf4a9 Compare June 18, 2025 08:49

folkertdev force-pushed the run-make-forward-compiletest-runner branch from abbf4a9 to 8eef8cf Compare June 19, 2025 09:29

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-blocked Status: Blocked on something else such as an RFC or other implementation work. labels Jun 19, 2025

forward the bootstrap runner to run-make

d023fe7

The runner was already forwarded to `compiletest`, this just passes it on to `run-make` and uses it in the `run` functions.

folkertdev force-pushed the run-make-forward-compiletest-runner branch from 8eef8cf to d023fe7 Compare June 19, 2025 10:32

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jun 21, 2025

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Jun 21, 2025

forward the bootstrap runner to run-make #141856

Are you sure you want to change the base?

forward the bootstrap runner to run-make #141856

Conversation

folkertdev commented Jun 1, 2025 • edited by jieyouxu Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Jun 1, 2025

Uh oh!

jieyouxu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jieyouxu Jun 1, 2025

Choose a reason for hiding this comment

Uh oh!

folkertdev Jun 1, 2025

Choose a reason for hiding this comment

Uh oh!

folkertdev Jun 19, 2025

Choose a reason for hiding this comment

Uh oh!

jieyouxu commented Jun 1, 2025

Uh oh!

folkertdev commented Jun 1, 2025

Uh oh!

jieyouxu commented Jun 1, 2025

Uh oh!

bors commented Jun 1, 2025

Uh oh!

jieyouxu commented Jun 1, 2025

Uh oh!

rust-bors bot commented Jun 1, 2025

Uh oh!

This comment has been minimized.

bors commented Jun 1, 2025

Uh oh!

This comment has been minimized.

jieyouxu commented Jun 1, 2025

Uh oh!

folkertdev commented Jun 1, 2025

Uh oh!

jieyouxu commented Jun 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jieyouxu commented Jun 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

folkertdev commented Jun 1, 2025

Uh oh!

jieyouxu commented Jun 19, 2025

Uh oh!

jieyouxu commented Jun 21, 2025

Uh oh!

rust-bors bot commented Jun 21, 2025

Uh oh!

rust-bors bot commented Jun 21, 2025

Uh oh!

jieyouxu commented Jun 21, 2025

Uh oh!

bors commented Jun 21, 2025

Uh oh!

bors commented Jun 21, 2025

Uh oh!

rust-log-analyzer commented Jun 21, 2025

Uh oh!

bors commented Jun 21, 2025

Uh oh!

folkertdev commented Jun 21, 2025

Uh oh!

Uh oh!

forward the bootstrap `runner` to `run-make` #141856

forward the bootstrap `runner` to `run-make` #141856

folkertdev commented Jun 1, 2025 •

edited by jieyouxu

Loading

jieyouxu commented Jun 1, 2025 •

edited

Loading

jieyouxu commented Jun 1, 2025 •

edited

Loading