Skip to content

Remember LLVM_ENABLE_LIBCXX setting in installed configuration #134990

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

Meinersbur
Copy link
Member

@Meinersbur Meinersbur commented Apr 9, 2025

The buidbot flang-aarch64-libcxx is currently failing with an ABI issue. The suspected reason is that LLVMSupport.a is built using libc++, but the unittests are using the default C++ standard library, libstdc++ in this case. This predefined llvm_gtest target uses the LLVMSupport from find_package(LLVM), which finds the libc++-built LLVMSupport.

To fix, store the LLVM_ENABLE_LIBCXX setting in the LLVMConfig.cmake such that everything that links to LLVM libraries use the same standard library. In this case discussed in llvm/llvm-zorg#387 it was the flang-rt unittests, but other runtimes with GTest unittests should have the same issue (e.g. offload), and any external project that uses find_package(LLVM). This patch fixed the problem for me locally.

llvm_gtest defined here:

add_llvm_library(llvm_gtest

Used by flang-rt unittests here:

add_subdirectory("${LLVM_THIRD_PARTY_DIR}/unittest" "${CMAKE_CURRENT_BINARY_DIR}/third-party/unittest")

Used by offload here:

add_subdirectory(${LLVM_THIRD_PARTY_DIR}/unittest ${CMAKE_CURRENT_BINARY_DIR}/third-party/unittest)

Use by libc here (but I don't see where it is added):

@Meinersbur Meinersbur marked this pull request as ready for review April 9, 2025 11:31
@llvmbot llvmbot added the cmake Build system in general and CMake in particular label Apr 9, 2025
Copy link
Contributor

@nikic nikic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Meinersbur Meinersbur merged commit 785e7f0 into llvm:main Apr 10, 2025
15 checks passed
@llvm-ci
Copy link
Collaborator

llvm-ci commented Apr 10, 2025

LLVM Buildbot has detected a new failure on builder openmp-offload-amdgpu-runtime-2 running on rocm-worker-hw-02 while building llvm at step 5 "compile-openmp".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/10/builds/3180

Here is the relevant piece of the build log for the reference
Step 5 (compile-openmp) failure: build (failure)
...
2.213 [2025/130/716] Copying CXXABI header cxxabi.h
2.213 [2024/130/717] Copying CXX header __algorithm/ranges_find_end.h
2.215 [2023/130/718] Copying CXX header __atomic/to_gcc_order.h
2.219 [2022/130/719] Building CXX object libc/src/math/generic/CMakeFiles/libc.src.math.generic.f16fmaf128.dir/f16fmaf128.cpp.o
2.220 [2021/130/720] Building CXX object libc/src/sys/stat/linux/CMakeFiles/libc.src.sys.stat.linux.fchmod.dir/fchmod.cpp.o
2.222 [2020/130/721] Copying CXX header __bit/bit_cast.h
2.224 [2019/130/722] Building CXX object libc/src/math/generic/CMakeFiles/libc.src.math.generic.tan.dir/tan.cpp.o
2.225 [2018/130/723] Building CXX object libc/src/sys/auxv/linux/CMakeFiles/libc.src.sys.auxv.linux.getauxval.dir/getauxval.cpp.o
2.225 [2017/130/724] Building CXX object libc/src/termios/linux/CMakeFiles/libc.src.termios.linux.cfgetispeed.dir/cfgetispeed.cpp.o
2.228 [2016/130/725] Building CXX object libc/test/src/math/performance_testing/CMakeFiles/libc_diff_test_utils.dir/Timer.cpp.o
FAILED: libc/test/src/math/performance_testing/CMakeFiles/libc_diff_test_utils.dir/Timer.cpp.o 
/home/botworker/builds/openmp-offload-amdgpu-runtime-2/llvm.build/./bin/clang++ --target=x86_64-unknown-linux-gnu -DLIBC_NAMESPACE=__llvm_libc_21_0_0_git -I/home/botworker/builds/openmp-offload-amdgpu-runtime-2/llvm.src/libc -stdlib=libc++ -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -fdiagnostics-color -ffunction-sections -fdata-sections -O3 -DNDEBUG -UNDEBUG -MD -MT libc/test/src/math/performance_testing/CMakeFiles/libc_diff_test_utils.dir/Timer.cpp.o -MF libc/test/src/math/performance_testing/CMakeFiles/libc_diff_test_utils.dir/Timer.cpp.o.d -o libc/test/src/math/performance_testing/CMakeFiles/libc_diff_test_utils.dir/Timer.cpp.o -c /home/botworker/builds/openmp-offload-amdgpu-runtime-2/llvm.src/libc/test/src/math/performance_testing/Timer.cpp
/home/botworker/builds/openmp-offload-amdgpu-runtime-2/llvm.src/libc/test/src/math/performance_testing/Timer.cpp:12:10: fatal error: 'chrono' file not found
   12 | #include <chrono>
      |          ^~~~~~~~
1 error generated.
2.228 [2016/129/726] Building CXX object libunwind/src/CMakeFiles/unwind_shared_objects.dir/Unwind-EHABI.cpp.o
2.228 [2016/128/727] Building CXX object libc/src/termios/linux/CMakeFiles/libc.src.termios.linux.cfgetospeed.dir/cfgetospeed.cpp.o
2.229 [2016/127/728] Copying CXX header __bit/bit_ceil.h
2.229 [2016/126/729] Copying CXX header __atomic/atomic_init.h
2.229 [2016/125/730] Copying CXX header __bit/bit_floor.h
2.230 [2016/124/731] Building CXX object libc/src/sys/stat/linux/CMakeFiles/libc.src.sys.stat.linux.fchmodat.dir/fchmodat.cpp.o
2.230 [2016/123/732] Building CXX object libc/src/sys/stat/linux/CMakeFiles/libc.src.sys.stat.linux.chmod.dir/chmod.cpp.o
2.232 [2016/122/733] Copying CXX header __atomic/atomic_lock_free.h
2.233 [2016/121/734] Building CXX object libc/src/sys/stat/linux/CMakeFiles/libc.src.sys.stat.linux.mkdirat.dir/mkdirat.cpp.o
2.233 [2016/120/735] Copying CXX header __algorithm/set_difference.h
2.235 [2016/119/736] Copying CXX header __algorithm/move_backward.h
2.236 [2016/118/737] Copying CXX header __algorithm/ranges_fold.h
2.236 [2016/117/738] Building CXX object libunwind/src/CMakeFiles/unwind_static_objects.dir/Unwind-EHABI.cpp.o
2.237 [2016/116/739] Copying CXX header __algorithm/move.h
2.237 [2016/115/740] Building CXX object libc/src/math/generic/CMakeFiles/libc.src.math.generic.f16fmal.dir/f16fmal.cpp.o
2.238 [2016/114/741] Building CXX object libc/src/termios/linux/CMakeFiles/libc.src.termios.linux.cfsetispeed.dir/cfsetispeed.cpp.o
2.239 [2016/113/742] Building CXX object libunwind/src/CMakeFiles/unwind_shared_objects.dir/Unwind-seh.cpp.o
2.239 [2016/112/743] Copying CXX header __algorithm/ranges_upper_bound.h
2.241 [2016/111/744] Building CXX object libc/src/termios/linux/CMakeFiles/libc.src.termios.linux.cfsetospeed.dir/cfsetospeed.cpp.o
2.241 [2016/110/745] Copying CXX header __algorithm/mismatch.h
2.242 [2016/109/746] Building C object libunwind/src/CMakeFiles/unwind_shared_objects.dir/Unwind-sjlj.c.o
2.246 [2016/108/747] Building CXX object libc/src/sys/stat/linux/CMakeFiles/libc.src.sys.stat.linux.mkdir.dir/mkdir.cpp.o
2.250 [2016/107/748] Building C object libunwind/src/CMakeFiles/unwind_shared_objects.dir/Unwind-wasm.c.o
2.251 [2016/106/749] Building CXX object libunwind/src/CMakeFiles/unwind_static_objects.dir/Unwind-seh.cpp.o
2.255 [2016/105/750] Building C object libunwind/src/CMakeFiles/unwind_static_objects.dir/Unwind-wasm.c.o
2.256 [2016/104/751] Building C object libunwind/src/CMakeFiles/unwind_static_objects.dir/Unwind-sjlj.c.o
2.264 [2016/103/752] Building CXX object libc/src/termios/linux/CMakeFiles/libc.src.termios.linux.tcsendbreak.dir/tcsendbreak.cpp.o
2.266 [2016/102/753] Building CXX object libc/src/termios/linux/CMakeFiles/libc.src.termios.linux.tcdrain.dir/tcdrain.cpp.o
2.268 [2016/101/754] Building CXX object libc/src/math/generic/CMakeFiles/libc.src.math.generic.sincos.dir/sincos.cpp.o
2.269 [2016/100/755] Building CXX object libc/src/sys/uio/linux/CMakeFiles/libc.src.sys.uio.linux.readv.dir/readv.cpp.o
2.272 [2016/99/756] Building CXX object libc/src/sys/wait/linux/CMakeFiles/libc.src.sys.wait.linux.wait.dir/wait.cpp.o
2.273 [2016/98/757] Building CXX object libc/src/sys/time/linux/CMakeFiles/libc.src.sys.time.linux.utimes.dir/utimes.cpp.o
2.273 [2016/97/758] Building CXX object libc/src/sys/stat/linux/CMakeFiles/libc.src.sys.stat.linux.lstat.dir/lstat.cpp.o

@DavidSpickett
Copy link
Collaborator

Unfortunately still not working for flang: https://lab.llvm.org/buildbot/#/builders/89/builds/20320

Various headers are not found, kinda like the failure reported above but more of them:

FAILED: flang-rt/lib/runtime/CMakeFiles/flang_rt.runtime.static.dir/allocator-registry.cpp.o 
/home/tcwg-buildbot/worker/flang-aarch64-libcxx/build/./bin/clang++ --target=aarch64-unknown-linux-gnu -D_DEBUG -D_GLIBCXX_ASSERTIONS -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -I/home/tcwg-buildbot/worker/flang-aarch64-libcxx/llvm-project/flang-rt/include -I/home/tcwg-buildbot/worker/flang-aarch64-libcxx/llvm-project/flang-rt/../flang/include -I/home/tcwg-buildbot/worker/flang-aarch64-libcxx/build/runtimes/runtimes-bins/flang-rt -stdlib=libc++ -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -fdiagnostics-color -ffunction-sections -fdata-sections -O3 -DNDEBUG -std=gnu++17 -UNDEBUG -fno-lto -fno-exceptions -fno-rtti -funwind-tables -fno-asynchronous-unwind-tables -U_GLIBCXX_ASSERTIONS -U_LIBCPP_ENABLE_ASSERTIONS -MD -MT flang-rt/lib/runtime/CMakeFiles/flang_rt.runtime.static.dir/allocator-registry.cpp.o -MF flang-rt/lib/runtime/CMakeFiles/flang_rt.runtime.static.dir/allocator-registry.cpp.o.d -o flang-rt/lib/runtime/CMakeFiles/flang_rt.runtime.static.dir/allocator-registry.cpp.o -c /home/tcwg-buildbot/worker/flang-aarch64-libcxx/llvm-project/flang-rt/lib/runtime/allocator-registry.cpp
In file included from /home/tcwg-buildbot/worker/flang-aarch64-libcxx/llvm-project/flang-rt/lib/runtime/allocator-registry.cpp:9:
/home/tcwg-buildbot/worker/flang-aarch64-libcxx/llvm-project/flang-rt/include/flang-rt/runtime/allocator-registry.h:14:10: fatal error: 'cstdint' file not found
   14 | #include <cstdint>
      |          ^~~~~~~~~

I can test things locally if needed.

@jplehr
Copy link
Contributor

jplehr commented Apr 10, 2025

If you need help from my end for the failing AMDGPU buildbot, let me know.
Happy to help.

If this is not an easy fix, can we revert in the meantime?

@Meinersbur
Copy link
Member Author

Meinersbur commented Apr 10, 2025

It is a different error, previous was a linking error (libc++ symbol not found in libstdc++). Now it does not find the libc++ headers. Same with lab.llvm.org/buildbot#/builders/10/builds/3180.

If the bootstrapping compiler (clang 19) found them, and they are installed in /usr, shouldn't stage1 clang find them as well? It did so when I tested locally (libcxx via Ubuntu package).

Meinersbur added a commit that referenced this pull request Apr 10, 2025
Meinersbur added a commit that referenced this pull request Apr 10, 2025
#134990)"

This reverts commit 785e7f0.

It did not solve the problem with flang-aarch64-libcxx and caused
another failure with openmp-offload-amdgpu-runtime-2.
@jplehr
Copy link
Contributor

jplehr commented Apr 10, 2025

Hmm.. I just looked at the CMake config we use on that failing bot. I don't think we have libcxx installed on the machine and we build with GCC.

The CMake command does -DLIBCXX_INSTALL_LIBRARY=OFF -DLIBCXX_INSTALL_HEADERS=OFF. Not sure if this is helpful / related / the issue.

@Meinersbur
Copy link
Member Author

@jplehr When libc++ is not installed, why does it use LLVM_ENABLE_LIBCXX=ON?

The builder's configure output includes:

CMake Warning at cmake/modules/HandleLLVMStdlib.cmake:28 (message):
  Can't specify libc++ with '-stdlib='
Call Stack (most recent call first):
  cmake/config-ix.cmake:18 (include)
  CMakeLists.txt:986 (include)

@jplehr
Copy link
Contributor

jplehr commented Apr 10, 2025

I copied the flags we use downstream, not sure why they are the way they are TBH. Will look more into that.

On a separate note though, would these flags be doing anything during the ninja step? We do not do an intermediate ninja install or anything.

@Meinersbur
Copy link
Member Author

I can test things locally if needed.

Would be great because I do not know why stage1-clang does not find the libc++ headers

@DavidSpickett
Copy link
Collaborator

In our case, the libcxx is not system installed, it's part of the llvm release package we use. So stage 1 clang has no way of knowing where that is unless we tell it. Or enable libcxx in stage 1, but that feels like we're then going to mix 2 versions of libcxx.

I'll figure that out.

@DavidSpickett
Copy link
Collaborator

I've found the settings to get stage 1 clang to use the host compiler's libcxx:

  -DLLVM_RUNTIME_TARGETS=aarch64-unknown-linux-gnu \
  -DRUNTIMES_aarch64-unknown-linux-gnu_CMAKE_CXX_FLAGS="-I /usr/local/clang+llvm-19.1.7-aarch64-linux-gnu/include/c++/v1/ -I /usr/local/clang+llvm-19.1.7-aarch64-linux-gnu/include/aarch64-unknown-linux-gnu/c++/v1/ -L/usr/local/clang+llvm-19.1.7-aarch64-linux-gnu/lib/aarch64-unknown-linux-gnu/" \

Note that you have to set LLVM_RUNTIMES_TARGETS, or the pass through of options doesn't work, or at least, didn't seem to.
I also tried a single --sysroot but the llvm install package isn't in a layout that will work with that.

Needs some changes to be something I'm comfortable putting in a buildbot config, but at least I know it can work.

@llvm-ci
Copy link
Collaborator

llvm-ci commented Apr 10, 2025

LLVM Buildbot has detected a new failure on builder lld-x86_64-win running on as-worker-93 while building llvm at step 7 "test-build-unified-tree-check-all".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/146/builds/2681

Here is the relevant piece of the build log for the reference
Step 7 (test-build-unified-tree-check-all) failure: test (failure)
******************** TEST 'LLVM-Unit :: Support/./SupportTests.exe/82/95' FAILED ********************
Script(shard):
--
GTEST_OUTPUT=json:C:\a\lld-x86_64-win\build\unittests\Support\.\SupportTests.exe-LLVM-Unit-22640-82-95.json GTEST_SHUFFLE=0 GTEST_TOTAL_SHARDS=95 GTEST_SHARD_INDEX=82 C:\a\lld-x86_64-win\build\unittests\Support\.\SupportTests.exe
--

Script:
--
C:\a\lld-x86_64-win\build\unittests\Support\.\SupportTests.exe --gtest_filter=ProgramEnvTest.CreateProcessLongPath
--
C:\a\lld-x86_64-win\llvm-project\llvm\unittests\Support\ProgramTest.cpp(160): error: Expected equality of these values:
  0
  RC
    Which is: -2

C:\a\lld-x86_64-win\llvm-project\llvm\unittests\Support\ProgramTest.cpp(163): error: fs::remove(Twine(LongPath)): did not return errc::success.
error number: 13
error message: permission denied



C:\a\lld-x86_64-win\llvm-project\llvm\unittests\Support\ProgramTest.cpp:160
Expected equality of these values:
  0
  RC
    Which is: -2

C:\a\lld-x86_64-win\llvm-project\llvm\unittests\Support\ProgramTest.cpp:163
fs::remove(Twine(LongPath)): did not return errc::success.
error number: 13
error message: permission denied




********************


@DavidSpickett
Copy link
Collaborator

I'm not going to be able to work on this due to travel, so I've written up what I know in #135381 and will get a colleague to pick it up. Maybe some of those notes will be useful for the other bots.

I have moved our bot to staging (https://lab.llvm.org/staging/#/builders/46), so if you do find fixes for the other bots and want to reland this, go ahead.

var-const pushed a commit to ldionne/llvm-project that referenced this pull request Apr 17, 2025
…134990)

The buidbot
[flang-aarch64-libcxx](https://lab.llvm.org/buildbot/#/builders/89) is
currently failing with an ABI issue. The suspected reason is that
LLVMSupport.a is built using libc++, but the unittests are using the
default C++ standard library, libstdc++ in this case. This predefined
`llvm_gtest` target uses the LLVMSupport from `find_package(LLVM)`,
which finds the libc++-built LLVMSupport.

To fix, store the `LLVM_ENABLE_LIBCXX` setting in the LLVMConfig.cmake
such that everything that links to LLVM libraries use the same standard
library. In this case discussed in
llvm/llvm-zorg#387 it was the flang-rt
unittests, but other runtimes with GTest unittests should have the same
issue (e.g. offload), and any external project that uses
`find_package(LLVM)`.
var-const pushed a commit to ldionne/llvm-project that referenced this pull request Apr 17, 2025
llvm#134990)"

This reverts commit 785e7f0.

It did not solve the problem with flang-aarch64-libcxx and caused
another failure with openmp-offload-amdgpu-runtime-2.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cmake Build system in general and CMake in particular
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants